cancel
Showing results for 
Search instead for 
Did you mean: 

30TB of raw data backed up to a single LTO4 tape

edla_shravya
Level 4

Hi All,

 

We are taking backup of raw devices where we have found 30TB of raw data backed up to a single LTO4 tape.LTO4 tape capacity has a capacity of 800GB uncompressed data , It takes upto 1.6TB of compressed data  but  30TB of data has been written onto the tape. Eventhough we have 20 Tapes assigned to that specific pool   only two Tapes are being utilized. If raw data will the capacity of tape differ. Could anyone please clarify?

 

Thanks in advance

1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

There is no such doc from Veritas.

You may want to ask your tape vendor.

View solution in original post

13 REPLIES 13

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Hi,

 

That is quite a bit of compression. It is possible to write more than the rated 1.6TB on a media. It all depends on the data that you're backing up and how compressable it is. Are you referring to a RAW data backup as a backup of the RAW disk device?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

This is not related to tape capacity, but with compressibility of data.

edla_shravya
Level 4

Hi,

 

Yes  I am referring to a RAW data backup as a backup of the RAW disk device. That means it does not depend on the tape capacity for raw disk device. There will not be any limit of capacity? Could you plesae let me know

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

It just means, what ever you're backing up is unique. If you backup a bunch of picture you'll get bad compression, if you backup a oracle data file you'll get better compression. In most cases that is partly because it is a sparse file (not completely filled). So when you're backing up the device RAW you're just going beneath the file system, but in the end, you're still backing up the blocks.

 

You really should not worry about this, the drive will tell you (NetBackup) when the tape is full and will request the next tape. You get what you get, you can be glad you're getting 30 and not 3.

 

To put you're mind at ease, perform a restore as a test.
 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

If the disk/lun is still fairly empty and/or thin provisioned, then the unoccupied space will be compressed very well and thin provisioned disk will only back up occupied space but report on entire allocated space.

Genericus
Moderator
Moderator
   VIP   

What Marianne said - depends on the data. 

I have some file systems that the program creates a big file (400GB) and then starts filling it up.

SO, to start, it is mostly empty, and takes no space to back up.

 

I am writing to LTO5 tapes, I generally max out at about 5-6 TB per tape, but some backups are encrypted or don't compress, and I can get full tapes that show as 1TB...

 

 

 

 

NetBackup 8.3.0.1 on Solaris 11, writing to Data Domain 9800 6.2.1.0
duplicating via SLP to LTO5 in SL8500 via ACSLS

sdo
Moderator
Moderator
Partner    VIP    Certified
Sounds like a raw read of a thin prov LUN is returning nearly all blocks not yet written by anything as blocks containing all zeroes - so, yes, LTO native compression is doing its job.

edla_shravya
Level 4

Hi Everyone,

Thanks a lot for your clarification. If this is the case

1.  after compression, how does a user check that exactly how much data has been written to the Tapes

2.If the compression rate depends on the type of data we backup then  How can we estimate regarding the number of media required for backup. 

 

 

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

You can see the amount of data on a tape via CLI (bpmedialist).

 

You'll have to perform this estimate yourself. If you've got various types of data, and you're sending them to different volume pools you'll be able to use bpmedialist to work out the average compression for the pool.

sdo
Moderator
Moderator
Partner    VIP    Certified

1) The media summary report will show how much is on each tape.

2) You can't - unless you write some fairly meaty scripts to post-process image catalog and media catalog and then determine based on policy and schedule which pools are likley to be used by which clients/policies - and then estimate based on previous results.  This is one of those questions that just doesn't make sense to experienced backup admins - i.e. we don't think like this.  There's no way of knowing before a backup how much space on tape will be consumed - there are multiple variables - not just compression - such as multi-plexing - inter-record gaps if streaming is not achieved - re-writes if the media or drive are a little dirty.

.

My advice, install OpsCenter and track growth and consumption of media using the reporting features.

edla_shravya
Level 4

Hi,

 

Can we get any document for  the compression rate for raw backups for LTO4 -tapes

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

There is no such doc from Veritas.

You may want to ask your tape vendor.

sdo
Moderator
Moderator
Partner    VIP    Certified

I haven't found yet, which compression algorithms have been licensed (used) by the LTO consortium for LTO4 technology, but I did find a SpectraLogic page:

https://www.spectralogic.com/features/lto/

...which says that LTO6 uses a form of compression named:

ECMA-321

...and their doc is here:

http://www.ecma-international.org/publications/standards/Ecma-321.htm

.

However, I seem to remember that LTO6 implemented a different compression algorithm to the earlier generations of LTO.

.

So, if you can find out which compression algorithm(s) are/were licensed for use within LTO4 drives - then maybe you can trace the algorithm owner's manual - i.e. the OEM of the compression technology to find out more about how the algorithm works - and/or likely compression rates - and/or how the algorithm behaves with a stream of bytes that are all the same - e.g. all zeroes.