cancel
Showing results for 
Search instead for 
Did you mean: 

Poor NDMP backup performance while backing up EMC isilon

nairaj
Level 3

We have a new EMC 3 node Isilon and we are using (3-way) NDMP. Isilon has multiple 10G network but our backup speed is 3MB/s.

We have NetApps with similar setup and it gives better backup speed.

Please advice if any of you has encountered similar issues.

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

nairaj
Level 3

This is resolved with the help of EMC support.

Resolution was to change the gateway priority of the multiple 1G network (subnet0) to 2 so that the backup traffic will go through the multiple 10G network.

View solution in original post

6 REPLIES 6

mnolan
Level 6
Employee Accredited Certified

So in ALL performance troubleshooting you have to look at ALL theorecticals.

Read Speed - Transfer Speed - Write Speed

 

If you feel that transfer network and storage speed is ok, I would focus on the read speed (As perhaps your NetApp is a better tiered device.)

From:

    How to troubleshoot NDMP Backups failures when status code 99 (or other NDMP backup failure) is reported. Includes logging instructions.

Article: TECH56492

Updated: February 24, 2015

Article URL: http://www.symantec.com/docs/TECH56492

7.  For performance issues, try backing up the path in the policy to a null device and note the timing:

NetApp:  dump 0f null /vol/volname
EMC Celerra:  server_archive <datamover name> -w -f /dev/null/ -J <file path>

 

If the Celerra command is not applicible for the Isilon, please see EMC for the correct command to attempt the dump to null to test local device speeds.

elanmbx
Level 6

Does the data have to be rehydrated prior to being backed up?  I'd guess it does and perhaps this is contributing to the poor backup performance?

a_la_carte
Level 5

NDMP local is the fastest way among 3-way NDMP backup and remote NDMP backup.

However, seggregating the backup into different backup selection lists, depanding upon the Q-tree structure at the filer end (Guessing that EMC Isilon has the similar "Q-tree" concept as NetApp filers.) helps speeding up the overall backup.

Rather than giving all the data into a single backup selection list such as /vol/vol0/xxx, if this can be split into multiple q-trees and hence different backup selections, that would definately help a bit such as /vol/vol0/xx, /vol/vol1/yy etc.

Dump test would be the first step towards troubleshooting slower NDMP backup as discussed by mnolan above.

 

 

 

Nicolai
Moderator
Moderator
Partner    VIP   

Let EMC have a look at the issue - they may be able to help you. 3MB/sec is ridiculous low.

Are you using the Isilon NDMP accelerator appliance ?

What NDMP environment variables are used - if any ?

Page 69 : https://www.emc.com/collateral/TechnicalDocument/docu50233.pdf

nairaj
Level 3

This is resolved with the help of EMC support.

Resolution was to change the gateway priority of the multiple 1G network (subnet0) to 2 so that the backup traffic will go through the multiple 10G network.

mnolan
Level 6
Employee Accredited Certified

So basically my previous comment at looking at all three areas was pretty spot on?