Forum Discussion

Sagar_K's avatar
Sagar_K
Level 3
11 years ago

Backing up big data through NDMP

Hi Guys,

We have volume size of 42TB data to be backup with local NDMP backup going to tape library, it is taking more than 20 days to complete, luckly we don't have much network issues.

I was wondering if some one suggest me having succesful backup in minimum time? any other methods?

 

Netbackup master: 7.1.0.3

Media: 7.1.0.3

Netapp filer: Ontap 8.1.2P2

I am taking backup using snapshot ex: /vol/TP_MYDOCS/.snapshot/weekly.0

We have snap mirror configured from Production to DR filer and NDMP backups are being taken from DR filer.

Please let me know if any aditional details.

 

Thank you so much in advance.

Sagar

 

7.1.x and Earlier
Backup and Recovery
NDMP Option
NetBackup
Windows Server (2003-2008)
  • RamNagalla's avatar
    RamNagalla
    Moderator
    11 years ago

    how many tape drives you are using?

    what the type of take Drive (lto 4 or lto5) ?

    what is the average speed that you are getting when sending it to tape?

  • Michael_G_Ander's avatar
    Michael_G_Ander
    Level 6
    11 years ago

    One thing that can help is to tune the buffer size for ndmp backup, SIZE_DATA_BUFFERS_NDMP. I have used the same value as for normal tape backup in SIZE_DATA_BUFFERS.

    But the ndmp process on NetApp is a down prioritized process, so other things can slow down the backup

    Another way to minimize the backup time could be to split the backup in mutliple streams if possible

    Hope this helps

        

  • Sagar_K's avatar
    Sagar_K
    Level 3
    11 years ago

    Hi Nagala/Michael,

    Thank for your reply, I am using LTO 5 tape drives and maximum speed was 30MBPS.

    We have 11 tape drives and tape drive availablity is not an issue, it will always get free tape drive.

    Below are the buffer values which are already set.

    NUMBER_DATA_BUFFERS - 32

    SIZE_DATA_BUFFERS - 262144

    SIZE_DATA_BUFFERS_NDMP - 65536

    I can't split the backup as it single volume, it is actually user profile share which is something can't be changed.

     

    Any further suggestion will be great help.

  • sdo's avatar
    sdo
    Moderator
    11 years ago

    FYI - just some simple calculations:

    Actual  
    42 TB
    43,008 GB
    44,040,192 MB
    20 days
    1,728,000 seconds
    25 MB/s
       
    Desired  
    100 MB/s
    440,402 seconds
    122 hours
    5.1 days

     

    Even if the NetApp was able to 100% consistently send the data at maximum near wire speed of a 1 Gb/s interface, then it would still take 5 days.

    I'd start by looking at the source system, and seeing whether you have any performance issues, i.e. inability to sustain traffic.

    How many active NICs do you have on the media server that is ingesteing the data?

    Does the media server have to perform any other backups?

  • sdo's avatar
    sdo
    Moderator
    11 years ago
    Are you using direct NDMP from NetApp to tape? Or three-way NDMP from NeApp to NetBackup to tape?
  • sdo's avatar
    sdo
    Moderator
    11 years ago
    Nagalla's question of average speed would be good to check/know, especially during times when the NetBackup environment is quiet, and nothing else is running. Does the intermittent average speed increase above the overall average of 25MB/s? And Michael's suggestion of splitting the backup into smaller pieces is worth looking at, especially if you have qtrees within the snapshot - and if so, then you could multi-stream and multi-plex the NDMP backup streams (assuming that the NDMP data is being ingested by the NetBackup media server (I.e. three-way NDMP) and not direct NDMP from NetApp to SAN attached tape).
  • Sagar_K's avatar
    Sagar_K
    Level 3
    11 years ago

    Thanks Nagalla, I have 11 LTO5 tape drives and max speed i saw was 25MB.

  • Sagar_K's avatar
    Sagar_K
    Level 3
    11 years ago

    Thanks Michael,

    Below are the buffer size set already set, please let me if any changes required.

    NUMBER_DATA_BUFFERS-32
    SIZE_DATA_BUFFERS - 262144
    SIZE_DATA_BUFFERS_NDMP - 65536

    Unfortunately, we can't split the backup as it is single volume, it is actually a user profile shared volume and it is not that easy to break into multiple volume/streams.

  • Sagar_K's avatar
    Sagar_K
    Level 3
    11 years ago

    Thanks, I have 2 NICs with teaming of 10GB speed, i don't have much experience on NetApp i will request my storage to team verify the performance issues on filer.

    Yes, we do have other backups running for other server backup such as Windows, Flash backup and Standard type but they are not running all the time, they scheduled to run  for 12 hours.

    We have 5 media servers shared with 2 libraries, 1 library dedicated for NDMP backups and other one for client backups.