I had an eventful couple of weeks figuring out how to get our duplicate jobs to our tape library to improve speeds so I thought I could save someone many man hours + of trail and error by reporting my findings and resolutions.
We use Backup Exec 2010 R3 on a 2008 R2 virtual server. We have two backup to disk drives connected to the virtual server that are attached through dedicated 10G iSCSI - they are 2.5 Terabytes each. The LTO4 tape library is attached through a 1G iSCSI interface card on the library over the same dedicated iSCSI network. All Backup Exec drive settings were set to default (64k block, 64k buffer). We were seeing speeds of 100-250 MB/Min while writing our duplicate jobs to tape and determined they should be faster. The actual backup to disk speeds were faster, but nothing we'd call "good" - around 500-750 MB/Min.
We started out by calling the library manufacturer to just ask them about ideas. They pointed us to an IBM tape speed testing utility called ITDT that works to test speeds to various IBM based tape drive hardware. The ITDT software proved to be invaluable in showing us speeds from our backup server to the tape drives at all the various supported block levels. For us the 256k block size/compressed showed to have the fasted results within the ITDT tests (at around 2700 MB/Min) so we set our drives accordingly. We increased the Buffer to 1 MB. Since we have a shared storage device the tape library is reading from (the iSCSI drives) we checked the Write single block mode and Write SCSI pass-through mode. If you have direct attached/fiber channel backup to disk storage my understanding is you should leave all those Drive Property boxes unchecked.
Changing those setting by themselves did not help us and we still saw similar speeds. But we had proven that the backup infrastructure was capable of more and was not bottlenecking the tape drives.
Next we began looking at the backup to disk iSCSI drives themselves. It quickly became apparent that there were serious fragmentation issues. Both drives had hundreds of 4 Gig backup to disk files that Backup Exec had created and was rotating through as they became over writable. Trying to defragment the drives through Windows proved impossible due to the high level of fragmentation + the size of the disks. It would never budge past the analyzing stage. Our once a week defragment schedule was not cutting it at all.
After verifying the backup to disk jobs were properly backed up to tape we erased ALL Backup to Disk files through Backup Exec, leaving only small, empty 2k files on the disk. We then defragmented the backup to disk drives through the Windows defrag utility. After this the backups ran much better. Both backups to disk and duplicates to tapes saw increased speed. The duplicate to tape was particularly significant with most speeds double or more than what they previously were running at. We have now set the defragment schedule to run once a day, and so far it has been able to keep up. Time will tell if eventually as the disk space use grows back to normal levels we will have to restort to clearing out the backup to disk files on some regular basis.
When using backup to disk and duplicating those backups, make sure you keep fragmentation under control. Those backup to disk files can really choke the system up badly if they are fragmented. Hopefully a few people find this usefull!
