cancel
Showing results for 
Search instead for 
Did you mean: 

memory

NIKHIL234656595
Level 6

Data are received by a bptm process and written into memory. The bptm process works on a block basic. Once a block is full, it's written to tape by another bptm process. Since the memory block size is equal to the SCSI block size on tape this, we need to take care !!. By using bigger block size we can improve tape writes by reducing system overhead.

http://www.mass.dk/netbackup/guides/49-netbackup-buffer-tuning.html

the memory block size is equal to the SCSI block size on tape

 

What above  means?

1 ACCEPTED SOLUTION

Accepted Solutions

mph999
Level 6
Employee Accredited

As Marianne pointed out - it uses default values.  Something like 64k size and 16 buffers I think.

It does not really matter what the  defaults are, as in my experience the tuning must be applied to give acceptable performance when using modern tape drives (that is drives that perform at similar speeds to LTO drives.  A good starting point is 262144 (size) and 32 or 64 (number) but it is a case of 'try it and see'.

 

View solution in original post

12 REPLIES 12

V4
Level 6
Partner Accredited

Very simple

Lower block size, more time to fill it complete MEMORY BLOCK and wait till next block fills up with same small blocks

Larger block size . lesser time to wait

Though quote itself is self explanatory , 

 

NIKHIL234656595
Level 6

the memory block size is equal to the SCSI block size on tape

What  is this memory block size and SCSI block size on tape?

 

mph999
Level 6
Employee Accredited

If I am going to be picky, this is not always true.

"Data are received by a bptm process and written into memory."

If the backup is local, (media server backing itself up) the bpbkar process sends the data directly to te memory buffer, bptm process then reads it and sends it to the OS, which writes it to tape.

If the backup is remote (eg. client over network) bpbkar on the client sends the data to a TCP socket.  The child bptm process then reads ir from the socket and puts it in the memory buffer, the parent bptm process then reads it from the memory buffer and sends it to the OS.  At no time ever, does NBU write to tape.

Anyway, I got distracted ...

The memory buffer = a bucket

Data = water

The bucket has a size eg, 256k 128k etc ...  this is specified in the SIZE_DATA_BUFFERS file.

The rule is that the bucket has to be full of water before it can be tipped out ...

So if the bucket is size 256k (= 1024 x 256 - 262144 in SIZE_DATA_BUFFER file) the 256k worth of water is sent at a time to the drives (via os of course ...).

In real life, it is data, so 256k of data is sent, we call this a block of data, so the size of the block = the size of the buffer.

So if my data size is 512k and my block size is changed to be 128k 

512/128 = 4 so a total of 4 blocks would be sent.

So a block is simply a 'chunk' of data.

Before anyone asks, If you had total data of size of 520k, you would write 5 blocks of data :

128k , 128k , 128k , 128k , 8k = 520k

When before I said the bucket (buffer) has to be full, there is an exception if the last bit of data isn't enough to fill it (like above example).  Here we write a 'short block' which simply means the block sent is less than the specified size.

This is why NBU requires the drives to be in variable block mode, the block size can change during a backup .

Martin

 

 

 

 

CRZ
Level 6
Employee Accredited Certified

Wouldn't you have been better off PM'ing Nicolai about his blog entry than starting a thread here?

(Maybe Nicolai will show up and have more patience than I have, though...)

NIKHIL234656595
Level 6

Thanks Martin.

 

What is this SCSI block size on tape ?

 

mph999
Level 6
Employee Accredited

It is what I have explained it as, just an amount of data of a specific size.

I don't really know how else to explain it ...

If I have data which is represented by x and the 'block is represented by and each x = 64k of data I have made the SIZE_DATA_BUFFERS file with 256k  (would be 262144)

So data on disk xxxx xxxx xxxx xxxx xx

I then send the data to tape 

|xxxx|

 

|xxxx|

|xxxx|

|xxxx|

|xx|

So the data fills the buffer, where the buffer is 256k, so therefore 256k is sent to tape, and simply, we call this a block.  Therefore my data files in 5 blocks, 4 blocks are 256k in size and the last block is 128k , as there wasn't enough data to fill it completely.

So the data on the tape looks like this :

 

(MH)(BH)(TM)|xxxx||xxxx||xxxx||xxxx||xx|(TM)(EH)(TM)(TM)

MH = Tape label, or media header

BH = Backup header

TM = Tape mark

So the data can be seen to be written to the tape in 'blocks' which are the same as what was sent from NBU.

Martin

 

 

 

NIKHIL234656595
Level 6

So we defined this SCSI block size on the tape?

mph999
Level 6
Employee Accredited

Yes

NIKHIL234656595
Level 6

Thaks Martin.

 

One last query:

If we dont have  SIZE_DATA_BUFFERS and NUMBER_DATA_BUFFERS file on our media servers,Then what will happen in that case?

/usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS and 

/usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS

 

Marianne
Level 6
Partner    VIP    Accredited Certified
Nikhil, you are really something else... With all your activity on Connect and (apparent) reading of documentation, have you NEVER heard of the Planning and Performance Tuning Guide? And that this guide tells you what the defaults are? And how to use bptm logs on media servers to what actual buffer sizes and numbers are used as well as the net effect on backups?

mph999
Level 6
Employee Accredited

As Marianne pointed out - it uses default values.  Something like 64k size and 16 buffers I think.

It does not really matter what the  defaults are, as in my experience the tuning must be applied to give acceptable performance when using modern tape drives (that is drives that perform at similar speeds to LTO drives.  A good starting point is 262144 (size) and 32 or 64 (number) but it is a case of 'try it and see'.

 

Marianne
Level 6
Partner    VIP    Accredited Certified
Defaults are different for different NBU versions. All documented in the Tuning Guides for the different versions. Also very clear in bptm log when job starts.