cancel
Showing results for 
Search instead for 
Did you mean: 

child bptm, shared memory and parent bptm

santoshpawar
Level 4
Hello, I need to know about shared memory. What is it exactly. During backup bpbkar sends data strems to child bptm. Then child bptm store that strems in shared memory and from shared memory parent bptm took the data and write to media. So will you please help me to understand about shared memory. What is it... Or it is just disk storage from our hard drive. Which get in use as shared memory for temporary basis to store the data strems.
1 ACCEPTED SOLUTION

Accepted Solutions

mph999
Level 6
Employee Accredited

 

Hi Santosh,

Your understanding is quite close, let me explain.

Data is sent from the client to a 'buffer'.  From here it is sent to storage (tape of disk).

Think of data as 'water' and the buffer as a 'bucket' (in fact multiple buckets).  The data (water) fills up the bucket, as you say, a tempoary storage area, before the water is tipped out of the bucket to the storage.  Fairly simple.

The buffer is in fact 'shared memory' actual RAM.  The size of these buffers is controlled by SIZE_DATA_BUFFERS config file, and the number of them by NUMBER_DATA_BUFFERS.  The larger these values, the more memory you need.  Shared memory is not unique to NBU, it is one of the ways that processes can use to talk to each other.  Other ways include message queues, sockets, named pipes (there are others but that's all I can think of at the moment).

The config files above are for tape, there are other files for disk eg. SIZE_DATA_BUFFERS_DISK which usually works well with higher values, and also files for fibre transport (SAN clients).  LTO7 drives are about to be released and I suspect we'll be able to use higher values than 'normal' for these as they are very fast, though don't quote me on this as I haven't got my mitts on these yet so it's an educated guess ... 

Which processes write / read from the data buffer changes, depending on a couple of factors - this is where it gets a bit confusing.

1.  Remote client (eg over the network).  bptm child and parent process

bpbkar sends the data from the client to a tcp socket on the media server.  A child bptm process sends it from here to the shared memory.  A separate bptm process then reads it from shared memory and sends it to the OS, which then sends it to storage (I could resist mentioning it - NBU does NOT write to storage, the OS does, which is why I've mentioned bptm sends the data to the OS).  For the record, NBU does read data from storage either, guess what, yep - it's that pesky OS again ...

2.  Local client (eg media server backing itself up)  (Only bptm parent process)

Here bpbkar sends the data direct to the shared memory.  There is no need to goto a tcp socket first as we're not going over the network, so this is more efficient.  Then, like before, bptm reads it from shared memory.

If you look in the manuals, you will see we don't talk about process names (bpbkar etc), we talk about a data 'provider' and 'data consumer'.  In (1), bptm is the provider and consumer, in (2) bpbkar is the provider and bptm the consumer  (so which process 'provides the data to shared mem and which process 'consumes' the data from shared mem.

There is a 3rd possibility.

3.  Local client (eg media server backing itself up) but with NOshm touch file

There is a belief that the NOshm means shared memory is not used, which is understandable, given the name of the touchfile.  Unfortunately, that is not true (apologies if I've just shattered the dreams and beliefs of anyone reading this ;0) )

What NOshm does, is make a local client behave like a remote client.  It means in other words, that you will get another bptm process.  So like a remote client, bpbkar sends the data to a tcp socket (even though we're not going over a network) and a child bptm process is used to send it to the shared memory. I only even use this for trooubleshooting.

There are a few other misconceptions about shared memory, or buffers as we generally call it.  The sizre and number of buffers affects performance.  Generally for LTO drives, a  size of 262144 is good (= 256k) and a number of 256.  Increasing these generally is a waste of memory (you need more memory remember if these increase) as if it's running at max performance (that is the data is filling the buckets about the same rate as it is leaving the buckets) then having more, or bigger buckets isn't going to help, as either the storgae can't write any quicker than it is, or the client (bpbkar) can't send the data any quicker (usually a netwrk limitation, less usually, a disk read speed limitation).

Nicolai is good with these performance settings, he has a web page (will have to look up the address, don't have it to hand) where he has performed a life long study on this subject, and helpfully, has written it down.

The amount of shared memory required can be calculated:

Total memory = (size data buffer x number data buffer) x mpx_value x number_drives   - so we use a chunk of memory for each data stream sent to each tape drive, it soon adds up if you start using mpx.  In fact, this equation is not quite write (it's close enough TBH, and is what is in the manuals), but for the full story ...

NBU rounds up the number of streams to the nearest factor of 4..

It's easier to show by example:

From my bptm log, we see I have x12 data buffers, each with a size of 131072 ...

08:55:45.665 [18200] <2> io_init: using 131072 data buffer size
08:55:45.665 [18200] <2> io_init: CINDEX 0, sched Kbytes for monitoring = 60000
08:55:45.665 [18200] <2> io_init: using 12 data buffers

... therefore, each tape drive, or each stream to a tape drive will require 131072 x 12 = 1572864

Now this example is actually from a MPX backup with 2 streams (mpx = 2), so you might think that the amount of shared memory will be 1572864 x2 = 3145728 

Now, here is the catch ...

Looking down my bptm log I find these lines ...

08:55:45.748 [18200] <2> mpx_setup_shm: buf control for CINDEX 0 is ffffffff79a00000
08:55:45.748 [18200] <2> mpx_setup_shm: shared memory address for group 0 is ffffffff76800000, size is 6291456, shmid is 117440636
08:55:45.748 [18200] <2> mpx_setup_shm: shared memory address for CINDEX 0 is ffffffff76800000, group 0, num_active 1

So we see the amount of memory is 6291456

Now, 6291456/ 1572864 = 4

So, what has happened, is even though I have one tape drive, and 2 streams the amount of memory NBU will allocate is the amount of memory required by x4 streams - NBU has 'rounded it up' .  In fact, it will round up to the nearest factor of 4, so it you have 5 streams, it will allocated the same amount of memory as if it was 8 streams.  I cannot remember the exact explanation for this,  NBU has always worked this way and it is done for efficiency (though as I say, i can;t remember the exact detailed explanation).

Hope this helps.

View solution in original post

10 REPLIES 10

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Nicolai
Moderator
Moderator
Partner    VIP   

Besides shared memory, setting semaphore is just as important. Every Netbackup upgrade actual check for proper values.

Recommended NetBackup *NIX semaphore tuning values 

http://www.symantec.com/docs/TECH203066

A Netbackup SORT risk assesment is also highly recommended, use the sort data collector from this location:

https://sort.symantec.com/data_collectors

 

mph999
Level 6
Employee Accredited

 

Hi Santosh,

Your understanding is quite close, let me explain.

Data is sent from the client to a 'buffer'.  From here it is sent to storage (tape of disk).

Think of data as 'water' and the buffer as a 'bucket' (in fact multiple buckets).  The data (water) fills up the bucket, as you say, a tempoary storage area, before the water is tipped out of the bucket to the storage.  Fairly simple.

The buffer is in fact 'shared memory' actual RAM.  The size of these buffers is controlled by SIZE_DATA_BUFFERS config file, and the number of them by NUMBER_DATA_BUFFERS.  The larger these values, the more memory you need.  Shared memory is not unique to NBU, it is one of the ways that processes can use to talk to each other.  Other ways include message queues, sockets, named pipes (there are others but that's all I can think of at the moment).

The config files above are for tape, there are other files for disk eg. SIZE_DATA_BUFFERS_DISK which usually works well with higher values, and also files for fibre transport (SAN clients).  LTO7 drives are about to be released and I suspect we'll be able to use higher values than 'normal' for these as they are very fast, though don't quote me on this as I haven't got my mitts on these yet so it's an educated guess ... 

Which processes write / read from the data buffer changes, depending on a couple of factors - this is where it gets a bit confusing.

1.  Remote client (eg over the network).  bptm child and parent process

bpbkar sends the data from the client to a tcp socket on the media server.  A child bptm process sends it from here to the shared memory.  A separate bptm process then reads it from shared memory and sends it to the OS, which then sends it to storage (I could resist mentioning it - NBU does NOT write to storage, the OS does, which is why I've mentioned bptm sends the data to the OS).  For the record, NBU does read data from storage either, guess what, yep - it's that pesky OS again ...

2.  Local client (eg media server backing itself up)  (Only bptm parent process)

Here bpbkar sends the data direct to the shared memory.  There is no need to goto a tcp socket first as we're not going over the network, so this is more efficient.  Then, like before, bptm reads it from shared memory.

If you look in the manuals, you will see we don't talk about process names (bpbkar etc), we talk about a data 'provider' and 'data consumer'.  In (1), bptm is the provider and consumer, in (2) bpbkar is the provider and bptm the consumer  (so which process 'provides the data to shared mem and which process 'consumes' the data from shared mem.

There is a 3rd possibility.

3.  Local client (eg media server backing itself up) but with NOshm touch file

There is a belief that the NOshm means shared memory is not used, which is understandable, given the name of the touchfile.  Unfortunately, that is not true (apologies if I've just shattered the dreams and beliefs of anyone reading this ;0) )

What NOshm does, is make a local client behave like a remote client.  It means in other words, that you will get another bptm process.  So like a remote client, bpbkar sends the data to a tcp socket (even though we're not going over a network) and a child bptm process is used to send it to the shared memory. I only even use this for trooubleshooting.

There are a few other misconceptions about shared memory, or buffers as we generally call it.  The sizre and number of buffers affects performance.  Generally for LTO drives, a  size of 262144 is good (= 256k) and a number of 256.  Increasing these generally is a waste of memory (you need more memory remember if these increase) as if it's running at max performance (that is the data is filling the buckets about the same rate as it is leaving the buckets) then having more, or bigger buckets isn't going to help, as either the storgae can't write any quicker than it is, or the client (bpbkar) can't send the data any quicker (usually a netwrk limitation, less usually, a disk read speed limitation).

Nicolai is good with these performance settings, he has a web page (will have to look up the address, don't have it to hand) where he has performed a life long study on this subject, and helpfully, has written it down.

The amount of shared memory required can be calculated:

Total memory = (size data buffer x number data buffer) x mpx_value x number_drives   - so we use a chunk of memory for each data stream sent to each tape drive, it soon adds up if you start using mpx.  In fact, this equation is not quite write (it's close enough TBH, and is what is in the manuals), but for the full story ...

NBU rounds up the number of streams to the nearest factor of 4..

It's easier to show by example:

From my bptm log, we see I have x12 data buffers, each with a size of 131072 ...

08:55:45.665 [18200] <2> io_init: using 131072 data buffer size
08:55:45.665 [18200] <2> io_init: CINDEX 0, sched Kbytes for monitoring = 60000
08:55:45.665 [18200] <2> io_init: using 12 data buffers

... therefore, each tape drive, or each stream to a tape drive will require 131072 x 12 = 1572864

Now this example is actually from a MPX backup with 2 streams (mpx = 2), so you might think that the amount of shared memory will be 1572864 x2 = 3145728 

Now, here is the catch ...

Looking down my bptm log I find these lines ...

08:55:45.748 [18200] <2> mpx_setup_shm: buf control for CINDEX 0 is ffffffff79a00000
08:55:45.748 [18200] <2> mpx_setup_shm: shared memory address for group 0 is ffffffff76800000, size is 6291456, shmid is 117440636
08:55:45.748 [18200] <2> mpx_setup_shm: shared memory address for CINDEX 0 is ffffffff76800000, group 0, num_active 1

So we see the amount of memory is 6291456

Now, 6291456/ 1572864 = 4

So, what has happened, is even though I have one tape drive, and 2 streams the amount of memory NBU will allocate is the amount of memory required by x4 streams - NBU has 'rounded it up' .  In fact, it will round up to the nearest factor of 4, so it you have 5 streams, it will allocated the same amount of memory as if it was 8 streams.  I cannot remember the exact explanation for this,  NBU has always worked this way and it is done for efficiency (though as I say, i can;t remember the exact detailed explanation).

Hope this helps.

GulzarShaikhAUS
Level 6
Partner Accredited Certified

Hi Martin,

This explaination is worth an article. May be you can publish it.

mph999
Level 6
Employee Accredited

Oh, if you think so, sure ....

 

GulzarShaikhAUS
Level 6
Partner Accredited Certified

Please let me know once you publish it

mph999
Level 6
Employee Accredited

I've submitted, with a few changes, my post as an Article.  It is waiting for approval, and is titled:

'A closer look at shared memory and buffers'

watsons
Level 6

Great explanation... Martin, will you be including the concept of NET_BUFFER_SZ  in it? 

Or maybe a different article, because sometimes mixing with the normal BUFFER touch file this can complicate things up?

mph999
Level 6
Employee Accredited

No, I've not covered NET_BUFFER_SZ, generally these days it is recommened to set it to 0

https://support.symantec.com/en_US/article.TECH28339.html

santoshpawar
Level 4

Wow Great explanation...mph999.  After reading your post, now the concept of shared memory, child bptm parent bptm is very clear...

Thanks to all... for your valuable reply.

 

Regards,

Santosh Pawar.