Forum Discussion

krehman83's avatar
krehman83
Level 3
10 years ago

Netbackup Duplication Performance issue

Hi, 

We have a backup solution with Netbackup + DD890 (OST). The duplication jobs are running from Netbackup, but it looks like that there is no deduplication is being performed during the duplication jobs. Looks like a flat file is coming from Netbackup and going from one DD to the other DD. 

Is there any way to troubleshoot this? Any help will be much apprecaited.

Infra details: 

1) Netbackup version 7.1
2) DD-890 (Local) : OS 5.4.0.8-404909
3) DD-890 (Remote) : OS 5.4.0.8-404909
4) DD-BOOST is enabled
5) OST Plugin is installed at Media servers; details below. 

Plugin Name: libstspiproxyMT.so
        Prefix: *
        Label: STS-I Proxy
        Build Version: 10
        Build Version Minor: 1
        Operating Version: 10
        Vendor Version:

        Plugin Name: libstspinbostpxyMT.so
        Prefix: NewProxy:
        Label: NewProxy STS-I Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version:

        Plugin Name: libstspibasicdiskMT.so
        Prefix: STSBasicDisk:
        Label: Basic Disk Plugin
        Prefix: AdvancedDisk:
        Label: Advanced Disk Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version: 710000 NetBackup_7.1

        Plugin Name: libstspinearstoreMT.so
        Prefix: ntap:
        Label: Nearstore Plugin
        Build Version: 9
        Build Version Minor: 1
        Operating Version: 9
        Vendor Version: Protocol Version = 2

        Plugin Name: libstspicpMT.so
        Prefix: ClientDirect:
        Label: Copy STS-I Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version:

        Plugin Name: libstspishmemMT.so
        Prefix: ShMem:
        Label: Shared Memory STS-I Plugin
        Build Version: 0
        Build Version Minor: 0
        Operating Version: 0
        Vendor Version:

        Plugin Name: libstspipdMT.so
        Prefix: PureDisk:
        Label: OpenStorage PureDisk Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version: 6.0604.0111.1129 14:13:49 Nov

        Plugin Name: libstspiDataDomainMT.so
        Prefix: DataDomain:
        Label: Data Domain Plugin
        Build Version: 11
        Build Version Minor: 1
        Operating Version: 11
        Vendor Version: 2.5.0.3-314845

Thanks,
Khalid

 

7.1.x and Earlier
Backup and Recovery
Deduplication Option
Linux
NetBackup
Troubleshooting
  • Nicolai's avatar
    Nicolai
    10 years ago

    Enabling DSP does not require a reboot.

    The formal fix is either DDBOOST plugin version 2.6.2.1. EMC made a workround in this relase of BOOST

    or

    You open a support call with Symantec and reference Bug ID: 3416462. But since you are at 7.1 go for the update of the boost plugin.

11 Replies

Sort By
  • RamNagalla's avatar
    RamNagalla
    Moderator
    10 years ago

    so if i understand correct... you have the backup copy in local DD, and duplicating it to remote DD...

    if that is the case..what ever the data that you have in local DD is already deduplicated..... and then moving to another DD.

    you dont find good deduplication factor the data which is already deduplicated.... its not possible.. 

    let me know my understanding is not correct... 

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    OST duplication from one Data Domain to another is performed optimized, meaning only uniquer data is moved between a set of Data Domains.

    Netbackup backup are encapsulated in files for disk based backup, it is those files you see replicated.

    If you are in doubt about the deduplication rate, use the Enterprise Manager web GUI and check the values. Point a browser to use IP used by one of the Data Domain and you should see a login screen.

    http://www.emc.com/collateral/software/data-sheet/h6801-datadomain-enterprise-manager-ds.pdf

  • krehman83's avatar
    krehman83
    Level 3
    10 years ago

    Many thanks Ram and Nicoali for responding on this. 

    @ Ram Yes your understanding is correct it goes from local DD to remote DD. But the replicaiton/duplicaiton jobs are not controlled by Data Domain but infect Netbackup. 

    @Nicolai  - I do login to the Enterprise Manager GUI for diff activities but since replication/duplicaiton is managed by Netbackup I don't see any rate there. 

    My question again is; 

    1) When we take the backup from database to local DD using Netbackup should it be deduplicated?  I don't think so that it will be and even if it does not I don't mind with that. 

    2) Replication of the same backup to remote DD is carried out by Netbackup ( using SLPs) - I understand that this should be de-duped and less data should be transferred across th network to remote DD. But it looks like complete backup copy is being replicated as the timings are quite high. Or otherwise it should reduce with daily backup being almost the same. We don't have much change in the data on daily basis, still a lot of data is same over the days and should not be replicated across and should deduped. 

    Is there anything we are missing here? Any help will be much apprecaited. 

    Many Thanks,
    Khalid

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    Netbackup control the replication but Data Domain carry it out. OST  (BOOST) enabled Netbackup to request Data Domain operation and DD will then report back the result.

    About the replication time you could be affected by the bug mentioned here:

    https://www-secure.symantec.com/connect/forums/netbackup-imports-running-longer-when-using-airdata-domain

     

  • krehman83's avatar
    krehman83
    Level 3
    10 years ago

    Many thanks Nicolai for pointing to the bug. Just a confirmation that all the previous releases are effected by this bug? 

    "Makre sure you use DDBOOST plugin version 2.6.2.1 or install EEB for Symantec BUG id Symantec about bug 
    Bug ID: 3416462. The fix is included in 7.6.0.2

    This EEB alleviates an issue in Accelerator backups where the incorrect image type is passed to an open image call.  The EEB replaces bptm and bpdm, and will be disruptive to any running jobs utilizing these programs.  Pleasequiesce all jobs before installing.   This fix alleviates the noted issue and
    also places the current img_saveas value on the opened image in the bptm/bpdm log.    "

    Our system info; 

    DD BOOST Plugin Version: 2.5.0.3-314845 ( same on remote & local DD)
    NetBackup: 7.1.0.3
    OS: Linux 2.6.16.60-0.117

     

    Also I found the below settings; that this sound ok to you? Why DSP is enabled on remote data domain and disaled on local data domain? Enabling it at local data domain would change anything?

     

    ddboost option show ( Remote)
    Option                           Value
    ------------------------------   --------
    distributed-segment-processing   enabled
    virtual-synthetics               enabled
    fc                               disabled


     ddboost option show (Local)
    Option                           Value
    ------------------------------   --------
    distributed-segment-processing   disabled
    virtual-synthetics               enabled
    fc                               disabled
    ------------------------------   --------

     

    Again, many thanks for your help in looking into this.

    Thanks,
    Khalid

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago

    Why DSP is enabled on remote data domain and disaled on local data domain?

    We cannot tell you.
    Ask the person who implemented this solution.

    Enabling it at local data domain would change anything?

    DSP helps with backup performance.

    Look for this White Paper on EMC site: h7296-data-domain-boost-openstorage-wp.pdf
    (EMC Data Domain Boost for Symantec NetBackup OpenStorage Best Practices Planning)

    Extract:

    DD Boost distributed segment processing
    DD Boost distributed segment processing allocates part of the deduplication process to NetBackup media
    servers, usually increasing the aggregate backup data transfer rate, and decreasing both NetBackup media server and Data Domain CPU loads. Additionally, LAN traffic is reduced between NetBackup media
    servers and the Data Domain system, as only unique data is transferred over the backup network. For
    additional information, please refer to the EMC Data Domain Boost for OpenStorage Administration
    Guide.

    DD Boost distributed segment processing is enabled by default on Data Domain systems where the initial
    DD OS install occurred with version 4.8 or later. Data Domain systems upgraded to DD OS 4.8 from prior
    DD OS versions will not have the distributed segment processing

    *************************************************************************************************************

    Look for other recommendations such as:

    Segregate optimized duplication traffic onto a dedicated replication network when possible.

    file-replication setting, etc...

    Best to involve your DD support team and network team to troubleshoot performance.
     

  • krehman83's avatar
    krehman83
    Level 3
    10 years ago

    Many Thanks Marianne for the comments - I will speak to the EMC support team.

    Regards,

    Khalid

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    Yes you are affected by the bug. 

    Enables DSP on the local data domain. This will release CPU resources and move some of that load to the media servers. As a side effect load on the network link to Data Domain will fall with a factor 90%

    See http://www.mass.dk/netbackup/quick-hints/98-the-efficiencies-of-emc-boost-distributed-segment-processing.html

    Possible the admin as disabled DSP because it caused CPU bottleneck at the media servers. Or I could be oversight in the configuration. I would enable DSP right away

  • krehman83's avatar
    krehman83
    Level 3
    10 years ago

    Thanks Nicolai. There is nothing else running apart from the backups - so not sure why admin would disable it. Agree with you, I will enable it anyways. 

    Regarding the BUG can you please share where I can find more details on this? I understand that either we need to upgrade to NB 7.6 or get DDBOOST plugin version 2.6.2.1 or install EEB for Symantec.

  • krehman83's avatar
    krehman83
    Level 3
    10 years ago

    I am sure enabling DSP does not require a reboot- can you please confirm?