Forum Discussion

Lbellefeuille's avatar
Lbellefeuille
Level 4
6 years ago

Advice on using NDMP Accelerated backup or not

Hello Everyone,  I am seeking for some advice on what should i be using to backup an emc Isilon using NDMP protocol. On this isilon i have millions of small files that i need to backup. About 70 % o...
8
Not Applicable
  • Nicolai's avatar
    Nicolai
    6 years ago

    With small files, it is not very likely you will be able to archive a high transferspeed. There may be speed gains by setting the OneFS NDMP maximum tape block size to 256K instead of default value. I would say a backup time of 554 hours for 33 TB is near unpractical setup. Furthermore - the speed from each share is about 45MB/sec, meaning you will wear tape drives down because of shoe shining. 

    Always keep full and incremental schedules in same policy, else Netbackup won't be able to refer the incremental backup to the full backup.

    For reference about NDMP on OneFS: https://www.emc.com/collateral/TechnicalDocument/docu65070.pdf

Nicolai's avatar
Nicolai
Moderator
6 years ago

What is the size of data you are requested to backup ?

If you take isilon data size and divide it by combined tape drive bandwidth, does it add up or does a full backup take 3 months to complete ?

Has archiving been considered , what are the legal requirements ?

 

  • Lbellefeuille's avatar
    Lbellefeuille
    Level 4
    6 years ago

    Hello Nicolai, sorry for the delay.

    The total amount of Data is close to 200 TB and its devided into 6 seperate parent folders. I know that this is not ideal and this has been adressed with the architect and the client for a complete reorganization of the data and the way they work with it but in the mean time i have to back this up in the most effective way.

    From the first tests, a full backup of a 33 TB folder is taking 554 hours to complete (accelerated). Not very fast but can i expect more speed since it's compose of millions of very small image (jpg...) files ?

    I will try a non accelerated backup but should i create 2 different job in this case, one for the full and one for the incremental ?

    I plan to include these options (if both Full and incremental are on the same job) :

    SET BACKUP_MODE=SNAPSHOT
    SET DIRECT=Y
    SET LEVEL=10
    SET HIST=F

    SET SNAPSURE=Y

    Does it make sense ?

      

    • Nicolai's avatar
      Nicolai
      Moderator
      6 years ago

      With small files, it is not very likely you will be able to archive a high transferspeed. There may be speed gains by setting the OneFS NDMP maximum tape block size to 256K instead of default value. I would say a backup time of 554 hours for 33 TB is near unpractical setup. Furthermore - the speed from each share is about 45MB/sec, meaning you will wear tape drives down because of shoe shining. 

      Always keep full and incremental schedules in same policy, else Netbackup won't be able to refer the incremental backup to the full backup.

      For reference about NDMP on OneFS: https://www.emc.com/collateral/TechnicalDocument/docu65070.pdf

      • Lbellefeuille's avatar
        Lbellefeuille
        Level 4
        6 years ago

        Thanks Nicolai for your valuable input.  It's more that obvious that we need to change the type of Data we save on this NAS, implement archiving and find some other backup target (disk pool) for this. 

        Thanks again.