netbackup buffers and Solaris restrictions

[mpatt]

1) What good is a high NET_BUFFER_SZ (262144) when tcp_recv_hiwat and tcp_xmit_hiwat are set to low values (49152 or 48 Kb). Solaris kernal has the default at 49152. Symantec in the following tech note states NET_BUFFER_SZ is no good unless those 2 values are also set to a higher value.

http://www.symantec.com/business/support/index?page=content&id=TECH28339

(FYI I have SIZE_DATA_BUFF = 131072 and NUMBER_DATA_BUFF=128 and tcp_time_wait_interval=1000)

I dont have an issue changing them on my Master n Media servers, but the catch is that Symantec recommends the NET_BUFFER_SZ to be the same on Media and Client servers to improve the performance. Most of my client servers are Oracle DB or Apps servers and OS is perf tuned for those apps. The DB Admin are not sure if increasing  tcp_recv_hiwat and tcp_xmit_hiwat also increase the memory utilization. (RMAN itself puts them on the edge given the load it takes along with the nightly batch jobs)

I need an expert opinion about setting tcp_recv/xmit_hiwat to be the same as NET_BUFFER_SZ although i am trying this out on 6 clients to see the perf change.

2) Symantec in its performance tuning guide 6.5 page 98, says default SIZE_DATA_BUFF for tape is 64 Kb and advises to change it based on the tape drive specs. We have HP LTO-4 and LTO-5 whose spec's state that BUFFER SIZE is 64 MB and 256 MB (not KB). To what extent can we increase these values?

[Yasuhisa_Ishika]

and is also used as I/O block size. I suppose what you said(64MB) is physical buffer size in tape drives. Do not use this for reference of tuning SIZE_DATA_BUFFERS. I prefer to leave SIZE_DATA_BUFFERS default(do not create SIZE_DATA_BUFFERS file), and set NUMBER_DATA_BUFFERS 64. This is a bit conservative setting comparing with those gurus here used. But it is enough to get 200~250MB/s transfer speed per stream.

[Yasuhisa_Ishika]

tcp_recv_hiwat and tcp_xmit_hiwat must be set equal with or grater than NET_BUFFER_SZ. Otherwise, size of send and receive buffer allocated to bptm is limited by those values, and tuning of NET_BUFFER_SZ does not make sense.

[Nicolai]

I use 256 or higer for NUMBER_DATA_BUFFERS and NUMBER_DATA_BUFFERS_RESTORE. SIZE_DATA_BUFFERS is set to 262144 (256K). This values work really good for a Linux box.

See: http://www.mass.dk/netbackup/guides/49-netbackup-buffer-tuning.html

[mpatt]

I may have maxed out on the values. When i increased my number data buffers beyond 128 (to 256), i started gettng System failure errors (i think they were status 13 and 50). So i cut down on those values.

I am still not clear about the impact of increasing the tcp_recv_hiwat and tcp_xmit_hiwat values on other applications. Assuming that i set these buffers at 262144 along with NET_BUFFER_SZ, does that mean that 256 KB of buffer is allocated for every TCP connection that is made to/from that server? Assuming that /dev/tcp tcp_conn_req_max_q is set to 256, the worst case memory allocation for TCP Buffers would be 256*256 KB ??

Am i reading this right?

[Omar_Villa]

All block sizes must match to improve performance other way you will be waiting to fill buffers at some level, like seting up your network buffers to 64k in your client it means the media server will wait twice to fill the 128k buffer you have. So if everything is 256k will improve a lot. About the tx value of solaris it will depend of the nic speed so we can tune it at the best way.

[Omar_Villa]

Here is a good way to isolate nbu resources from oracle and you will be able to tune even more of tcp settings http://veritas-nbu.blogspot.com/2010/01/solaris-10-tuning.html Regards

[mpatt]

Thanks Omar,

We do have Gig-e network on most servers. I have increased incresead the tcp_tx and rx values on a few client systems. I will have to monitor their impact both at the backup level and Application level (Oracle DB/Web).