Netbackup duplication data chunk size

It looks like you've got default buffers configuration and your environment is not tuned. All settings are described in this document: http://www.symantec.com/business/support/index?page=content&id=TECH1724

You can try with:

SIZE_DATA_BUFFERS = 262144

NUMBER_DATA_BUFFERS = 128

SIZE_DATA_BUFFERS_DISK = 1048576

NUMBER_DATA_BUFFERS_DISK = 16

"For example we are currently writing backups to advanced disk pools and we can see on the disk array that data is being written in 512k chuncks." - It's configured in SIZE_DATA_BUFFERS_DISK file.

"However when we duplicate to tape the array is receiving read requests to pass the data in 64k chunks" - It's configured in SIZE_DATA_BUFFERS file. Be careful, SIZE_DATA_BUFFERS is related to tape drives. In other words - during duplication disk read buffer is equal to tape write buffer. You can't set disk read buffer to 1024k and tape write buffer to 512k (for example).

marekk

No these buffer files are in place.

These buffer files determine how Netbackup buffers the information before passing it to disk/tape and we are using 32 x 1mb buffers for disk backups and 32 x 128k buffers for tape backups. Now when backing up direct to tape or disk we can see in bptm that it is using these values and we get great performance. Its only duplications that are slow.   

the array manufacturer has examined the  performance data and can see that writes to the array occur in 512k blocks (so i am assuming that when Netbackup buffers the data into our 1mb buffers its filled with 512k chunks as used by the bpbkar process).

However when the reads are coming off the array for duplications they are in 64K chunks which symantec support tell me is the default for bpduplicate operations. However they said they hadn't a clue as to how it could be changed.

I know its not the array because if  i backup data from the array it reads it in 512K chunks inline with the bpbkar process.

There is no option to change this, I was trying to find a solution for this problem few months ago but only way to incerase performance is to add more physical tape drives because there is no multiplexing during duplication. Duplication is similar to restore process, NB reads each image file by file. If backup has a lot of small files duplication is slow, if backup has large files - duplication is fast.

I have found a parameter in the bptm log called DUP_BLKSIZE and this is set to 65536 (64K)

I have asked symantec support to speak with engineering to see if this can be modified

curious to know where the netbackup picks up the DUP_BLKSIZE setting.