cancel
Showing results for 
Search instead for 
Did you mean: 

media multiplexing

raffl
Level 4

i am duplicating images from tape to disk and getting 20-30 GB/Hour. These are LTO 4 tapes and duplicating them to advanced disk.

If these tapes are multiplexed will it imact the duplication ?I see in tape report that they were multiplexed but doesn't show what multiplexing number was used.

Pls help me with my performnce.

I have 1 master and 4 media server. 1 server reading 4 drives.

number data buffers = 32

size data buffer = 262144

 

 

30 REPLIES 30

I saw the logs and found that empty buffer below, what exactly is happening can you tell.

17:17:09.644 [13144.14056] <2> read_data: Total_Kbytes_Read = 27200000
17:17:10.331 [6896.9512] <2> fill_buffer: [14184] socket is closed, waited for empty buffer 251987 times, delayed 9745910 times, read 331150956 Kbytes
17:17:10.456 [9856.15168] <2> read_data: Total_Kbytes_Read = 82200064
17:17:10.674 [9596.12072] <2> set_job_details: Tfile (4443317): LOG 1501028230 4 bptm 9596 completed reading backup image

mph999
Level 6
Employee Accredited

If you're not filling buffers quickly enough, increasing the number of them makes no difference, apart from using more memory ....

mph999
Level 6
Employee Accredited

There should be a line that reads "read_buffer: waited for empty buffer xx times, delayed yy times"

You do not show this, altough I expect the numbers will be low (this would show that the job is reading from the tape)

There will be anther line that you didn't show "write_data: waited for full buffer xx ...." - this is the one we really want to see, though I suspect the numbers may be higher that what we see in the fill_buffer line.

Here is the snippet Martin, I tried serveral docs and really confused how to interpret this. Can you pls explain a bit.

1:55:10.736 [25808.14272] <2> read_data: stopping mpx read because 5610752 Kbytes of 5610752 were read
01:55:10.736 [25808.14272] <2> io_ioctl: command (1)MTFSF 1 0x0 from (bptm.c.14997) on drive index 2
01:55:10.752 [25808.14272] <2> read_data: waited for empty buffer 1608 times, delayed 17063 times
01:55:24.846 [4392.24788] <2> fill_buffer: [21596] socket is closed, waited for empty buffer 204870 times, delayed 2190809 times, read 516244904 Kbytes
01:55:24.846 [21596.7104] <2> write_data: writing block shorter than BUFF_SIZE, 172032 bytes
01:55:24.908 [21596.7104] <2> write_data: writing short block, 172032 bytes, remainder 0
01:55:24.908 [21596.7104] <2> write_data: waited for full buffer 365295 times, delayed 1395403 times

mph999
Level 6
Employee Accredited

01:55:10.752 [25808.14272] <2> read_data: waited for empty buffer 1608 times, delayed 17063 times

01:55:24.908 [21596.7104] <2> write_data: waited for full buffer 365295 times, delayed 1395403 times

 
For a duplication, bptm does both the read and the write.

So we have:

Source side  data   --->  Buffers  ---->  Duplication side

The source side fills up the buffer, and was delayed from doing so 17063 times, each delay is I think 15 milli seconds, so in the dupration of the job, there was a (15 x 17063)/ 1000 second delay = about 255 seconds, where the read side process was doing nothing (well it was waiting for a buffer to become free).  Presuming the job length was 'kinda long' , 255 sec isn't worth bothering about (in fact, you usually will get some delays, even in the best tuned system), as it is a small % of the totakl job length.  that said, I don;t know what the job length was, so you have to make a judgement as to how significant the delay is.

The write side is the bigget problem:

01:55:24.908 [21596.7104] <2> write_data: waited for full buffer 365295 times, delayed 1395403 times

Here bptm, waited 1395403 for data to be duplicated to be written to the buffer - so although the bptm read process was going through the tape, it was only finding relevant data 'every now and then', so the write process bptm had to wait until the buffer was full of data relevant to the backup being duplicated.

(15 x 1395403 )/ 1000 = 20931 sec = 348 minutes = 5.8 hours.

So, the mpx of the backup, may be contributing up to 5.8 hours of delays, in this specific job.

Thanks for the explaination Martin.

so if i understand correctly. bptm will read from tape and fill the buffer( buffer means memory or cache).

When the buffer is full it will write that data to disk( in my case). So you are saying before writing to disk again it will write it to buffer and that is taking time. Can you please explain this concept as i am getting confused here.

 

mph999
Level 6
Employee Accredited

It is confusing, I agree.

Think of water as being data, and buffers being buckets.

We can adjust the size of the buckets, and the number.

Data flows from Client to Media server (water in this case) but to get there has to go via a bucket.

The rules

You can only start to fill an empty bucket (cleint sending data)

You can only empty a full bucket (media server writing data to storage)

If you have poor read from client due to network or whatever, the water will only fill the bucket slowlty - hence bptm saying 'waiting for full' as it can only 'empty' a full bucket. (the exception being the end of backup where there may be less data than a full buffer).  You can also se how increseing the number of buffers (buckets) makes no difference, as you still can't fill them quick enough.

Of course, the idea is we have many buckets (typically 128 or 256 works well with LTO drives) and there are of size 256k, so there is a whole load of data flowing, into various buckets.

Does that help ?

mph999
Level 6
Employee Accredited

Thanks Marianne, mind you, I don't even remeber writing that ....

It really helped me a lot. Thanks so much for explaning things in detail and devoting your precious time in explaing the things. I really like here where everyone is trying to help.

I also tried to open this link as suggested by Marianne but when i click on view more it doesn't show or load the page. Can you paste that article here if possible.

https://vox.veritas.com/t5/Articles/A-closer-look-at-shared-memory-and-buffers/ta-p/809945#M931

 

 

 

 

sdo
Moderator
Moderator
Partner    VIP    Certified