Forum Discussion

childress's avatar
childress
Level 2
16 years ago

NDMP slow performace

After upgrade from Netbackup v6.0.4 to v6.5.4 our NDMP backups are taking twice to three times longer to run.  We moved from a IBM to HP server.
Two fiber cables attached to each head of the Netapps then to a brocade switch which then connects to the Netbackup server.  Answers???

7.1.x and Earlier
Backup and Recovery
NetBackup
Performance
  • childress's avatar
    childress
    16 years ago

    Answer was found, when we upgraded it seems the NBLOG.CONF file had Diagnostic and Debug logging level set to 6, not sure how this happened as not that experienced in logging, but after it was changed with the help on Symantec the NDMP backups ran at great speed.  One thing that we noticed that bought us to this conclusion was that we had 200 gigs of logs.  ATTENTION GETTER.

    Hopes this help when you have the blind leading the blind when you upgrade from 6.0 to 6.5.4.

    Be careful not to change file without support or if you know how to do it from command line is the only way it is suppose to be done. 

  • Deepak_W's avatar
    Deepak_W
    Level 6
    16 years ago
    Can you provide some details as mentioned below...

    What is the size of the NDMP data that is being backed up ??

    Are you using local NDMP backup or Remote ??

    I had faced NDMP slow backup issue, but got resolved after splitting the backup selection into different policies as my NDMP backup size was more than 2TB

    Hope this helps you...

  • childress's avatar
    childress
    Level 2
    16 years ago
    Using NDMP Remote
    We have two Netapp heads, each has a fiber connection to a Brocade Silkworm 300 16 port Fiber Channell Switch then connected by fiber to a Sun Storage Teck SL 500.

    We upgraded from V6.0.4 running on an IBM 8670 Server to V6.5.4 on a HP Proliant DL380  G5.
    Just one Volume on our Netapps is about 596 gigs of small files which prior to upgrade ran around for about 4hours at approximatly 40,000 Kbytes/sec now with the upgrade the same Vol runs at 10,000 Kbytes/sec and runs 15hours same size.
    This is only on one vol on one head, in the past I ran 7 volumes (2 of them very large with many little files that are images) and it would run for about 15 hours in total.

    Any suggestions would help, could it be server performance?  Clueless and so is Symantec and NetApps.
  • Android's avatar
    Android
    Level 6
    16 years ago
    Have you looked into whether this is related to your network adapter?  Are the old and new adapter the same speed?  Did you have any sort of trunking set up or tuning in place on the old server that is not in place on the new one? 

    Have you checked to be sure the interface is communicating at full duplex at the highest speed?  Is the new server connected to the Netapp and the SL500 in the same way that the old server was?

    When you installed NBU on your new server did you create all of the policies and schedules from scratch?  If so, have you looked at whether or not you were previously using multiplexing or something in the previously existing policy that would have an impact on performance? 



  • childress's avatar
    childress
    Level 2
    16 years ago

    Answer was found, when we upgraded it seems the NBLOG.CONF file had Diagnostic and Debug logging level set to 6, not sure how this happened as not that experienced in logging, but after it was changed with the help on Symantec the NDMP backups ran at great speed.  One thing that we noticed that bought us to this conclusion was that we had 200 gigs of logs.  ATTENTION GETTER.

    Hopes this help when you have the blind leading the blind when you upgrade from 6.0 to 6.5.4.

    Be careful not to change file without support or if you know how to do it from command line is the only way it is suppose to be done. 

  • Claudio_Veronez's avatar
    Claudio_Veronez
    Level 6
    16 years ago
    Maybe we should turn off all log while we don have problems?

    I had made that with   bpcd   just renaming for DISABLE.bpcd


    bpkar and the others is actually running.


    I'll open another discussion about it,.



  • Android's avatar
    Android
    Level 6
    16 years ago
    When making changes to your logging levels daemons/services always need to be restarted for those changes to take.  Bounce NBU and the changes to your logging will take.
  • Jay_Son's avatar
    Jay_Son
    Level 5
    16 years ago
    Hi Childress

    Our NDMP (VMWare)  backups are at 3-4 times LESS the speed it was after we did an upgrade, as well. I've went ahead and chenged the unified logging to 0. But, I dont see a difference.

    I was wondering if you had restarted your master server daemons (and media servers) before starting you backups again? Did you set DebugLevel and DiagnosticLevel to 0?

    Thanks
  • childress's avatar
    childress
    Level 2
    16 years ago
    Yes we  reset the DebugLevel and Diagnostic Level to 0. 
    we did a bpdown then bpup after we changed the setting

    below is the command line that was entered by the Symantec support

    vxlogcfg -a -p 51216 -o Default  -s DiagnosticLevel=0 -s DebugLevel=0

    Note the DiagnosticLevel and DebugLevel is case sensitive so it must be spelled exactly the way it is above per support