cancel
Showing results for 
Search instead for 
Did you mean: 

Advice on using NDMP Accelerated backup or not

Lbellefeuille
Level 4

Hello Everyone, 

I am seeking for some advice on what should i be using to backup an emc Isilon using NDMP protocol. On this isilon i have millions of small files that i need to backup. About 70 % of those files are "cold files" but need to retain forever.

The isilon support NDMP level 10 (incremental forever) and since i am using version 8.1.1 i was wondering of what should be the backup setup for this ;

1) NDMP with accelerator and  a Force rescan every 6 month + incermental for the rest ? 

2) NDMP non accelerated backup with a full every year and incremental With level 10 the reste of the year  ?

Should both FULL and Incermental be configured on the same policy or should i use one policy for each ?  Any variable i should set in the policy ?

Thanks for sharing your thoughts on this.   

1 ACCEPTED SOLUTION

Accepted Solutions

Nicolai
Moderator
Moderator
Partner    VIP   

With small files, it is not very likely you will be able to archive a high transferspeed. There may be speed gains by setting the OneFS NDMP maximum tape block size to 256K instead of default value. I would say a backup time of 554 hours for 33 TB is near unpractical setup. Furthermore - the speed from each share is about 45MB/sec, meaning you will wear tape drives down because of shoe shining. 

Always keep full and incremental schedules in same policy, else Netbackup won't be able to refer the incremental backup to the full backup.

For reference about NDMP on OneFS: https://www.emc.com/collateral/TechnicalDocument/docu65070.pdf

View solution in original post

5 REPLIES 5

Nicolai
Moderator
Moderator
Partner    VIP   

What is the size of data you are requested to backup ?

If you take isilon data size and divide it by combined tape drive bandwidth, does it add up or does a full backup take 3 months to complete ?

Has archiving been considered , what are the legal requirements ?

 

Hello Nicolai, sorry for the delay.

The total amount of Data is close to 200 TB and its devided into 6 seperate parent folders. I know that this is not ideal and this has been adressed with the architect and the client for a complete reorganization of the data and the way they work with it but in the mean time i have to back this up in the most effective way.

From the first tests, a full backup of a 33 TB folder is taking 554 hours to complete (accelerated). Not very fast but can i expect more speed since it's compose of millions of very small image (jpg...) files ?

I will try a non accelerated backup but should i create 2 different job in this case, one for the full and one for the incremental ?

I plan to include these options (if both Full and incremental are on the same job) :

SET BACKUP_MODE=SNAPSHOT
SET DIRECT=Y
SET LEVEL=10
SET HIST=F

SET SNAPSURE=Y

Does it make sense ?

  

Nicolai
Moderator
Moderator
Partner    VIP   

With small files, it is not very likely you will be able to archive a high transferspeed. There may be speed gains by setting the OneFS NDMP maximum tape block size to 256K instead of default value. I would say a backup time of 554 hours for 33 TB is near unpractical setup. Furthermore - the speed from each share is about 45MB/sec, meaning you will wear tape drives down because of shoe shining. 

Always keep full and incremental schedules in same policy, else Netbackup won't be able to refer the incremental backup to the full backup.

For reference about NDMP on OneFS: https://www.emc.com/collateral/TechnicalDocument/docu65070.pdf

Thanks Nicolai for your valuable input.  It's more that obvious that we need to change the type of Data we save on this NAS, implement archiving and find some other backup target (disk pool) for this. 

Thanks again.  

Nicolai
Moderator
Moderator
Partner    VIP   

If pureply for DR - consider to deploy a secondary Isilon system where snapshot are transfered too in a seperate location.