cancel
Showing results for 
Search instead for 
Did you mean: 

Netbackup Duplication Performance issue

krehman83
Level 3

Hi, 

We have a backup solution with Netbackup + DD890 (OST). The duplication jobs are running from Netbackup, but it looks like that there is no deduplication is being performed during the duplication jobs. Looks like a flat file is coming from Netbackup and going from one DD to the other DD. 

Is there any way to troubleshoot this? Any help will be much apprecaited.

Infra details: 

1) Netbackup version 7.1
2) DD-890 (Local) : OS 5.4.0.8-404909
3) DD-890 (Remote) : OS 5.4.0.8-404909
4) DD-BOOST is enabled
5) OST Plugin is installed at Media servers; details below. 

Plugin Name: libstspiproxyMT.so
        Prefix: *
        Label: STS-I Proxy
        Build Version: 10
        Build Version Minor: 1
        Operating Version: 10
        Vendor Version:

        Plugin Name: libstspinbostpxyMT.so
        Prefix: NewProxy:
        Label: NewProxy STS-I Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version:

        Plugin Name: libstspibasicdiskMT.so
        Prefix: STSBasicDisk:
        Label: Basic Disk Plugin
        Prefix: AdvancedDisk:
        Label: Advanced Disk Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version: 710000 NetBackup_7.1

        Plugin Name: libstspinearstoreMT.so
        Prefix: ntap:
        Label: Nearstore Plugin
        Build Version: 9
        Build Version Minor: 1
        Operating Version: 9
        Vendor Version: Protocol Version = 2

        Plugin Name: libstspicpMT.so
        Prefix: ClientDirect:
        Label: Copy STS-I Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version:

        Plugin Name: libstspishmemMT.so
        Prefix: ShMem:
        Label: Shared Memory STS-I Plugin
        Build Version: 0
        Build Version Minor: 0
        Operating Version: 0
        Vendor Version:

        Plugin Name: libstspipdMT.so
        Prefix: PureDisk:
        Label: OpenStorage PureDisk Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version: 6.0604.0111.1129 14:13:49 Nov

        Plugin Name: libstspiDataDomainMT.so
        Prefix: DataDomain:
        Label: Data Domain Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version: 2.5.0.3-314845

Thanks,
Khalid

 

1 ACCEPTED SOLUTION

Accepted Solutions

Nicolai
Moderator
Moderator
Partner    VIP   

Enabling DSP does not require a reboot.

The formal fix is either DDBOOST plugin version 2.6.2.1. EMC made a workround in this relase of BOOST

or

You open a support call with Symantec and reference Bug ID: 3416462. But since you are at 7.1 go for the update of the boost plugin.

View solution in original post

11 REPLIES 11

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

so if i understand correct... you have the backup copy in local DD, and duplicating it to remote DD...

if that is the case..what ever the data that you have in local DD is already deduplicated..... and then moving to another DD.

you dont find good deduplication factor the data which is already deduplicated.... its not possible.. 

let me know my understanding is not correct... 

Nicolai
Moderator
Moderator
Partner    VIP   

OST duplication from one Data Domain to another is performed optimized, meaning only uniquer data is moved between a set of Data Domains.

Netbackup backup are encapsulated in files for disk based backup, it is those files you see replicated.

If you are in doubt about the deduplication rate, use the Enterprise Manager web GUI and check the values. Point a browser to use IP used by one of the Data Domain and you should see a login screen.

http://www.emc.com/collateral/software/data-sheet/h6801-datadomain-enterprise-manager-ds.pdf

krehman83
Level 3

Many thanks Ram and Nicoali for responding on this. 

@ Ram Yes your understanding is correct it goes from local DD to remote DD. But the replicaiton/duplicaiton jobs are not controlled by Data Domain but infect Netbackup. 

@Nicolai  - I do login to the Enterprise Manager GUI for diff activities but since replication/duplicaiton is managed by Netbackup I don't see any rate there. 

My question again is; 

1) When we take the backup from database to local DD using Netbackup should it be deduplicated?  I don't think so that it will be and even if it does not I don't mind with that. 

2) Replication of the same backup to remote DD is carried out by Netbackup ( using SLPs) - I understand that this should be de-duped and less data should be transferred across th network to remote DD. But it looks like complete backup copy is being replicated as the timings are quite high. Or otherwise it should reduce with daily backup being almost the same. We don't have much change in the data on daily basis, still a lot of data is same over the days and should not be replicated across and should deduped. 

Is there anything we are missing here? Any help will be much apprecaited. 

Many Thanks,
Khalid

Nicolai
Moderator
Moderator
Partner    VIP   

Netbackup control the replication but Data Domain carry it out. OST  (BOOST) enabled Netbackup to request Data Domain operation and DD will then report back the result.

About the replication time you could be affected by the bug mentioned here:

https://www-secure.symantec.com/connect/forums/netbackup-imports-running-longer-when-using-airdata-domain

 

krehman83
Level 3

Many thanks Nicolai for pointing to the bug. Just a confirmation that all the previous releases are effected by this bug? 

"Makre sure you use DDBOOST plugin version 2.6.2.1 or install EEB for Symantec BUG id Symantec about bug 
Bug ID: 3416462. The fix is included in 7.6.0.2

This EEB alleviates an issue in Accelerator backups where the incorrect image type is passed to an open image call.  The EEB replaces bptm and bpdm, and will be disruptive to any running jobs utilizing these programs.  Pleasequiesce all jobs before installing.   This fix alleviates the noted issue and
also places the current img_saveas value on the opened image in the bptm/bpdm log.
"

Our system info; 

DD BOOST Plugin Version: 2.5.0.3-314845 ( same on remote & local DD)
NetBackup: 7.1.0.3
OS: Linux 2.6.16.60-0.117

 

Also I found the below settings; that this sound ok to you? Why DSP is enabled on remote data domain and disaled on local data domain? Enabling it at local data domain would change anything?

 

ddboost option show ( Remote)
Option                           Value
------------------------------   --------
distributed-segment-processing   enabled
virtual-synthetics               enabled
fc                               disabled


 ddboost option show (Local)
Option                           Value
------------------------------   --------
distributed-segment-processing   disabled
virtual-synthetics               enabled
fc                               disabled
------------------------------   --------

 

Again, many thanks for your help in looking into this.

Thanks,
Khalid

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Why DSP is enabled on remote data domain and disaled on local data domain?

We cannot tell you.
Ask the person who implemented this solution.

Enabling it at local data domain would change anything?

DSP helps with backup performance.

Look for this White Paper on EMC site: h7296-data-domain-boost-openstorage-wp.pdf
(EMC Data Domain Boost for Symantec NetBackup OpenStorage Best Practices Planning)

Extract:

DD Boost distributed segment processing
DD Boost distributed segment processing allocates part of the deduplication process to NetBackup media
servers, usually increasing the aggregate backup data transfer rate, and decreasing both NetBackup media server and Data Domain CPU loads. Additionally, LAN traffic is reduced between NetBackup media
servers and the Data Domain system, as only unique data is transferred over the backup network. For
additional information, please refer to the EMC Data Domain Boost for OpenStorage Administration
Guide.

DD Boost distributed segment processing is enabled by default on Data Domain systems where the initial
DD OS install occurred with version 4.8 or later. Data Domain systems upgraded to DD OS 4.8 from prior
DD OS versions will not have the distributed segment processing

*************************************************************************************************************

Look for other recommendations such as:

Segregate optimized duplication traffic onto a dedicated replication network when possible.

file-replication setting, etc...

Best to involve your DD support team and network team to troubleshoot performance.
 

krehman83
Level 3

Many Thanks Marianne for the comments - I will speak to the EMC support team.

Regards,

Khalid

Nicolai
Moderator
Moderator
Partner    VIP   

Yes you are affected by the bug. 

Enables DSP on the local data domain. This will release CPU resources and move some of that load to the media servers. As a side effect load on the network link to Data Domain will fall with a factor 90%

See http://www.mass.dk/netbackup/quick-hints/98-the-efficiencies-of-emc-boost-distributed-segment-processing.html

Possible the admin as disabled DSP because it caused CPU bottleneck at the media servers. Or I could be oversight in the configuration. I would enable DSP right away

krehman83
Level 3

Thanks Nicolai. There is nothing else running apart from the backups - so not sure why admin would disable it. Agree with you, I will enable it anyways. 

Regarding the BUG can you please share where I can find more details on this? I understand that either we need to upgrade to NB 7.6 or get DDBOOST plugin version 2.6.2.1 or install EEB for Symantec.

krehman83
Level 3

I am sure enabling DSP does not require a reboot- can you please confirm?

Nicolai
Moderator
Moderator
Partner    VIP   

Enabling DSP does not require a reboot.

The formal fix is either DDBOOST plugin version 2.6.2.1. EMC made a workround in this relase of BOOST

or

You open a support call with Symantec and reference Bug ID: 3416462. But since you are at 7.1 go for the update of the boost plugin.