Duplication speed with staging / parallel process

[cstaubliew]

Hi all

I've a question regarding duplication speed between media servers. Our setup is as follows:

Clients do backup to different media server, storage is basic disk. This usually happens during night time.

At each storage unit there is a staging schedule defined, which will duplicate all images to another media server with similar storage (same size, same type, same raid level etc) during day time. This works fine, dupliction is started according to schedule, and according to number of maximum concurrent jobs. On source server, the one sending images to destination server, many processes of bpdm are started, all with network activity towards destination server. On destination side however there's a single pbx_exchange process started receiving all this traffic and storing it to disk.

Now my question is as follows: both source and destination server are connected to each other with dedicated 10Gpbs links, but with duplication process we're unable to reach speeds beyond 2Gbps. First we've checked server and storage itself, with multithreaded file copy from source to destination storage (robocopy images from source to destination) we're able to reach much higher speeds, up to 8Gbps over smb file share. Is there a setting which allows more parallel processes or streams to be run with duplication? On source side I see many number of bpdm processes, but just a single one on destination.

NetBackup master, media (both source and destionation):

OS: Windows 2008 R2

NetBackup: 7.6

Thanks in advance for any insight

Christian

[Marianne · Accepted solution]

The pbx process is not responsible for read or write processing.

It will be bptm process on the destination server as well as bptm processes on the source server.

You may want to create bptm log folders on both servers to see what the current buffer sizes are and on which side the 'wait for full' or 'wait for empty' buffers are.

About Buffer Tuning, see:

NetBackup Backup Planning and Performance Tuning Guide, Release 7.5 and Release 7.6 : 
http://www.symantec.com/docs/DOC7449

[RiaanBadenhorst · Accepted solution]

These might be a good start (on both source and destination)

echo 128 > INSTALL_PATH\Veritas\NetBackup\db\config\NUMBER_DATA_BUFFERS
echo 128 > INSTALL_PATH\Veritas\NetBackup\db\config\NUMBER_DATA_BUFFERS_DISK
echo 262144 > INSTALL_PATH\Veritas\NetBackup\db\config\SIZE_DATA_BUFFERS
echo 524288 > INSTALL_PATH\Veritas\NetBackup\db\config\SIZE_DATA_BUFFERS_DISK

Play around with the numbers after you've read the guide that Marianne posted.

[cstaubliew]

I've started to read this document, it's quite large... :)

For beginning, I've setup the values Riaan stated and configured logs for bptm. Will knew more tomorrow

[Yasuhisa_Ishika]

BasicDisk staging may not be able to run in parallel. Please consider to duplicate images by script or SLP.

[cstaubliew]

Initial results are promising, with 128 buffers and size of 512KB/1024KB we're able to reach 4 Gbit. For next cycle I've doubled buffers to 256

[RiaanBadenhorst]

Great news.
 

[cstaubliew]

256 buffers did not improve speed, but with this guide I can now tune other settings like size as well. Thanks alot!