duplication to tape

MSDP images have to re-hydrate before being put to tape, therefore the duplication will be slow as the "restore" part (the re-hydration) will be slower.

Why only 1 tape if you have the need to duplicate lots of backups?

Lots of random reads when re-hydrating from MSDP.  Have you checked your disk volume for how 'active' it is?  use Windows Resource Monitor, in the 'Disk' tab, in the 'Storage' panel at the bottom, you can see how busy a disk volume is.  If it's 99% all the time, then you know where your first (only?) bottleneck is.

It does not make sense to have only 1 tape drive if you intent to duplicate most, if not all, data to tape. Add more tape drives.

Even Symantec NetBackup Appliances (which are carefully specified, built, and tuned) are only 'unofficially' capable of driving two tape drives, and this is also 'unofficially' only when they are quiet - i.e. not having to perform backups, and are thus free to duplicate (re-hydrate) to tape.

To build your own server capable of reaching these levels, or exceeding them, requires very careful system design, and a pocket full of money to specify/purchase/build a powerful server.

Here's a simple calculation, which assumes that your server is even capable of this... so if you have 10 TB to duplicate to tape, and you have have a server capable of re-hydrating (sustained random read IOPs/throughput) and thus capable of reading MSDP disk at 120 MB/s - to match the typical minimum sustained write throughput of 120 MB/s to one LTO6 tape drive - then if you had 10TB to duplicate to tape, it would take 24 hours:

How much data do you have which must be duplicated?

And you need to be mindful that if a server (and it's storage (which MSDP resides upon)) is only capable of, for example sustained random read of 60 MB/s, then a duplication will take on average 48 hours - but then this also assumes that the tape drive itself is capable of 'speed streaming matching' down to 60 MB/s.  If the tape drive is not capable of physically running slower in 'streaming mode' - then shoe shining will take place, which will greatly increase the elapsed duration of the duplication.

FYI - FH (full height) and HH (half height) LTO tape drives have quite different capabilities and charactertistics.  Also, IBM LTO drives and HP LTO drives and Quantum LTO drives (etc...) have different 'speed matching' capabilities - e.g. some can speed match to 60 or 80 or 100 or 120 or 140 or 160 MB/s - whereas some can only speed match to 75 or 150 MB/s.  So, if one has a tape drive which has a minimum speed match capability of 75 MB/s and a server only capable of sustaining 60 MB/s, then shoe shining will occur for the entire duration of the duplication, and massivley increase the duration of the duplication job - and wear out both the tape drive head itself and the tape media at a much much higher rate.

Hello, 

Sorry for not getting back sooner. I had a support call about this issue. We found out that the disks are fast enough. One issue we found was that we have some very slow backups. Some systems take more then 20 hours. Support told me that those slow backups to disk will also be slow duplicated to tape. 

Support recommended me to investigate what the buffer-size should be and configure it accordingly 
https://support.symantec.com/en_US/article.TECH18422.html

So now I need to find out what the buffer-size should be for LTO3, LTO4 and LTO5 tapedrives. 

Also indeed they said that just 1 tape-drive is a little bit less. 

Would it not be possible to have multiple stream from the disk to tape while duplicating? 

Best regards, 

Rob

Why do you need to know typical buffer for three variants of LTO?  You only have one tape drive, so which version of LTO is it that you have?

I think support were a little unfair.  A slow backup does not necessarily equal a slow duplication to tape.  It is true that if the backup is large, then the duplication will be large.  And if the backup finishes late, then teh duplication willbe late because it cannot start until teh backup completes.  But, if the backup is at 5MB/s and finishes late, then on a quietish MSDP pool the duplication to tape should be much faster than 5MB/s - nonetheless, it will still be late.

Duplication to tape, from MSDP, is always an effective single stream and cannot be multi-plexed.  This is why it is really important to have reasonably fast disk under MSDP.

.

You say your disk is fast enough.  How did you prove this?  What random speeds did you get?  Are you aware of the recommended minimum disk speeds for MSDP?  Would you like some steps/commands/process to test disk speed?

Well this is an example site. I have multiple masterservers, the setup is the same. The difference is that they can have a LTO3, 4 or 5. 

The disk speed was tested by support, and they told the bottleneck is not there. If you can send some steps to test the speed of the disk that would be awsome.

Ok here's some steps:

The above are just basic single activity tests - and do not corresepond to real world - but they can serve as a minimum base line.

To really get a handle on what the storage is capable of, one would have to use something like the IOMeter tool and model a set of worker threads which emulate the IO patterns of MSDP - but that would take some time.

.

Also, are you aware that throughput to physical HDD spindles really tails off rather quickly as they go above 85% occupied.  It is a well known phenomenom.  So, if your storage array is at or over 85% full then performance can tail off quite dramatically.

.

Usually a typical buffer size for LTO4 and LTO5 is 256KB - but I'm not sure what LTO3 was generally capable of.  Not all drives are the same.  It could be true that some LTO3 drives use 64KB buffer, whereas some other may use 128KB.  Research vendor documentation and/or perform some test backups and test restores to/from test/scratch media when the systems are quiet - or make them quiet by disabling all SLPs and backup policies (except for your test policy) and then do some testing.

Hello All, 

Sorry for not getting back. The company I am working is doing a lot of migrations and as you all know the backup is the first to know about changes. I've been doing a lot of trouble shooting the past weeks. 

This issue, what we speak of in this thread, The feeling I get is that the data is indeed to much for one drive. I cannot really pin point it tho. The problem is a lack of time. So now we thought of a different solution. 

I configured a laptop as a Netbackup Mediaserver with an external 4TB USB disk. Last week I went to one of the remote sites, this site is small and has only vm's to backup. I did a full backup there to the laptop with accelarator enabled. The laptop is now sent courier to our datacenter where a Symantec appliance is. 

There I want to duplicate the images of the backup of the remote site to the appliance. And then I want to start doing the backups over the one connection from the remote site to the appliance in the data center. 

If this works we will get ride of tapes. This should work not? 

Regards, 
Rob