Forum Discussion

Tauhid's avatar
Tauhid
Level 4
10 years ago

Problem of Tape Cartridge usability

Dear All,

I'am facing a problem with proper utilization of tape cartridge.I've windows 2008 Master & Media Server and HP 6000 Tape LIbrary. We use 800 GB Data Cartridge.

see the below ouput of bpmedialist -U.

Here some cartridge is Full before reaching the 800GB. Again some cartridge is not still Full though it exceeds the 800 GB.

For example Tape 1147L3 shows Full though it contains only 419045888KB (419GB)

Again Tape 1100L3 is not full though it contains 1418911808KB (1.4TB).

My queston is that,

Why 1147L3 is Full before reaching 800GB and How 1100L3 writes more than 800GB data?

********************************

Server Host = hodbkp2-svr

 id     rl  images   allocated        last updated      density  kbytes restores
           vimages   expiration       last read         <------- STATUS ------->
--------------------------------------------------------------------------------
1080L3   6     13   12/14/2014 12:54  03/08/2015 08:00  hcart3    69571773     5
               13   09/10/2015 08:00  03/04/2015 11:15 

1081L3   8     17   11/12/2014 19:27  11/15/2014 14:09  hcart3   539941003     0
               17   11/15/2015 14:09        N/A         FULL

1082L3   1*   189   01/30/2015 09:20  02/13/2015 07:30  hcart3   621511597     0
               14   03/16/2015 07:30        N/A         FULL

1083L3   1*   165   01/10/2015 20:00  01/30/2015 09:20  hcart3   624348320     0
               65   03/02/2015 08:00        N/A         EXPIRED FROZEN

1084L3   8     20   11/14/2014 23:39  11/29/2014 16:18  hcart3   537401679     0
               19   11/29/2015 16:18        N/A         FULL

1085L3   8     15   11/15/2014 14:09  11/29/2014 10:03  hcart3   503002236     0
               14   11/29/2015 10:03        N/A         FROZEN

1086L3   1*   136   05/10/2014 18:30  05/26/2014 17:00  hcart3   699342124     0
               38   05/26/2015 14:30        N/A         FROZEN

1087L3   8     50   11/05/2014 11:01  11/13/2014 16:24  hcart3   543183721     0
               47   11/13/2015 16:24        N/A         FROZEN

1088L3   8*    69   11/02/2014 11:09  11/12/2014 19:27  hcart3   640309199     0
               61   11/12/2015 19:27        N/A         FULL

1090L3   1*   141   02/16/2015 18:30  03/07/2015 18:30  hcart3   686745863     0
              109   04/06/2015 07:30        N/A         FULL

1092L3   8     28   11/09/2014 14:18  11/14/2014 23:39  hcart3   576338149     0
               28   11/14/2015 23:39        N/A         FULL

1093L3   1     22   03/05/2015 20:00  03/10/2015 20:30  hcart3     6573940     0
               22   03/24/2015 20:30        N/A        

1095L3   8      7   01/28/2015 16:41  01/29/2015 12:36  hcart3    45770137     0
                7   01/29/2016 12:36        N/A        

1096L3   1     49   02/27/2015 11:07  03/10/2015 20:30  hcart3   190522651     0
               49   03/24/2015 20:30        N/A        

1097L3   1*   135   02/20/2015 08:02  02/27/2015 11:07  hcart3   605680806     0
               45   03/30/2015 07:30        N/A         FULL

1098L3   8*   160   10/28/2014 10:32  11/05/2014 20:30  hcart3   696836972     1
               19   10/28/2015 17:19  11/05/2014 11:54  FROZEN

1099L3   8      6   01/28/2015 16:41  01/29/2015 12:35  hcart3    32286592     0
                6   01/29/2016 12:35        N/A        

1100L3   1      4   02/16/2015 17:05  03/06/2015 00:30  hcart3  1418911808     4
                3   03/20/2015 00:30  02/25/2015 15:14 

1101L3   1*   136   07/20/2014 17:00  07/26/2014 20:30  hcart3   709503714     0
               10   07/22/2015 10:35        N/A         FROZEN

1102L3   3*    41   11/04/2013 13:41  12/10/2013 19:40  hcart3    68137125     0
                0   01/08/2014 14:35        N/A         EXPIRED FROZEN

1104L3   0     37   05/27/2014 03:10  06/02/2014 03:10  hcart3   444569920     0
                0   06/09/2014 03:10        N/A         EXPIRED FULL FROZEN

1107L3   6*    70   12/24/2013 11:37  12/14/2014 08:00  hcart3   402494685    26
               18   06/18/2015 08:00  01/14/2015 16:06  FULL

1108L3   1      6   12/18/2013 13:02  12/19/2013 10:51  hcart3    10092165     0
                0   01/02/2014 10:51        N/A         EXPIRED FROZEN

1110L3   8     28   04/15/2014 10:25  04/16/2014 14:15  hcart3   190809499     0
               28   04/16/2015 14:15        N/A        

1112L3   1      1   03/06/2015 08:30  03/06/2015 08:30  hcart3   508474752     0
                1   03/20/2015 08:30        N/A         FULL

1113L3   1      8   08/22/2014 08:30  08/29/2014 08:30  hcart3   471251787     0
                0   09/12/2014 08:30        N/A         EXPIRED FROZEN

1114L3   1      6   03/06/2015 08:30  03/11/2015 00:10  hcart3   166761088     0
                6   03/25/2015 00:10        N/A        

1115L3   1*    11   05/31/2014 00:10  06/06/2014 08:30  hcart3  1498746834     0
                0   07/06/2014 13:00        N/A         EXPIRED FROZEN

1116L3   1      7   02/20/2015 08:40  02/26/2015 00:10  hcart3   558822778     0
                1   03/12/2015 00:10        N/A         FULL

1118L3   1      2   02/26/2015 00:10  02/27/2015 08:30  hcart3   509743429     0
                2   03/13/2015 08:30        N/A         FULL

1119L3   1      8   02/27/2015 08:30  03/06/2015 08:30  hcart3   538890620     0
                8   03/20/2015 08:30        N/A         FULL

1121L3   0    160   09/01/2014 10:57  03/10/2015 17:00  hcart3         640     0
               60   03/17/2015 17:00        N/A        

1122L3   1*    17   04/06/2014 10:48  04/13/2014 13:44  hcart3   247117326    29
                7   04/13/2015 13:44  04/30/2014 10:39 

1123L3   8*    10   04/13/2014 12:21  06/18/2014 12:04  hcart3   250671682    17
                8   04/13/2015 13:44  06/18/2014 23:37 

1124L3   0      3   03/11/2015 06:00  03/11/2015 08:00  hcart3    28671008     0
                3   03/18/2015 08:00        N/A        

1125L3   0     40   02/27/2015 03:10  03/05/2015 06:00  hcart3   445809312     0
                5   03/12/2015 06:00        N/A         FULL

1126L3   0     37   03/05/2015 06:00  03/11/2015 06:00  hcart3   444576768     0
               37   03/18/2015 06:00        N/A         FULL

1139L3   1    209   12/27/2014 18:42  03/10/2015 19:13  hcart3   224387616     0
               54   03/24/2015 19:13        N/A        

1140L3   0*    24   05/13/2014 18:00  06/02/2014 10:42  hcart3  1225120128     0
                3   06/02/2015 10:42        N/A         FULL

1147L3   0      1   03/10/2015 11:08  03/10/2015 11:08  hcart3   419045888     0
                1   03/17/2015 11:08        N/A         FULL

1150L3   1*   201   02/17/2015 19:00  03/05/2015 20:00  hcart3   615691826     0
              130   03/23/2015 07:03  03/09/2015 11:00  FULL

1154L3   0*    19   02/22/2015 18:00  03/10/2015 18:00  hcart3  1275621323     0
               10   03/20/2015 09:30        N/A        

1156L3   8*    27   06/02/2014 11:08  06/11/2014 10:37  hcart3   954386909     0
               17   06/11/2015 10:37        N/A         FULL

1157L3   8*    26   06/08/2014 15:02  06/26/2014 18:00  hcart3  1300169855     0
                8   06/11/2015 10:38        N/A         FULL

1159L3   0      2   03/10/2015 11:08  03/10/2015 11:08  hcart3    86717324     0
                2   03/17/2015 11:08        N/A        

1160L3   0      7   03/03/2015 17:30  03/10/2015 17:30  hcart3    89715073     0
                6   03/17/2015 17:30        N/A        

********************************

Backing Up
Backup and Recovery
NetBackup
Windows Server (2003-2008)
  • Andy_Welburn's avatar
    Andy_Welburn
    10 years ago

    Comes up time & again on the forum - essentially boils down to the type of data being backed up & how compressible it is:

    https://www-secure.symantec.com/connect/forums/netbackup-able-backup-25-tb-data-tape-drive-800gb

  • Nicolai's avatar
    Nicolai
    10 years ago

    All tape drives on the market today compress before writing data to tape. So if a tape can store more  than the native capacity it simply means data was highly compressible.

    We need to clarify one thing before we can conclude you have a capacity problem. You state you use 800GB tapes. But is that LTO3 with 400G/800GB per tape or LTO4 with 800/1,6TB per tape ?

    I see you use L3 bar codes labels which indicate LTO3 and thus 400/800Gb per tape. The 800 GB per tape require a 2:1 compression.

    If you use LTO3 tape you get what is expected from LTO3 technology.

    Based on the "tape full" variation you seems to have both compressible backup data (tapes with >400GB per tape) and not so compressible data (~400GB per tape). This is quite normal.

    Best Regards

    Nicolai

  • Andy_Welburn's avatar
    Andy_Welburn
    Level 6
    10 years ago

    Comes up time & again on the forum - essentially boils down to the type of data being backed up & how compressible it is:

    https://www-secure.symantec.com/connect/forums/netbackup-able-backup-25-tb-data-tape-drive-800gb

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    All tape drives on the market today compress before writing data to tape. So if a tape can store more  than the native capacity it simply means data was highly compressible.

    We need to clarify one thing before we can conclude you have a capacity problem. You state you use 800GB tapes. But is that LTO3 with 400G/800GB per tape or LTO4 with 800/1,6TB per tape ?

    I see you use L3 bar codes labels which indicate LTO3 and thus 400/800Gb per tape. The 800 GB per tape require a 2:1 compression.

    If you use LTO3 tape you get what is expected from LTO3 technology.

    Based on the "tape full" variation you seems to have both compressible backup data (tapes with >400GB per tape) and not so compressible data (~400GB per tape). This is quite normal.

    Best Regards

    Nicolai

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago

    The amount of data that will fit onto a tape depends on the compressibility of data.
    Another factor that plays a role is the other images on the tape that may have expired in the meantime.

    Consider this for a full tape (tape filled up with image3):
    image1 | image2 | image3 

    Now image1 and image2 expire.
    The space from expired images cannot be overwritten until image3 also expires.
    bpmedialist will only report the valid image data.
    (probably the explanation for 1150L3 where you have multiple retention levels on same media).

    Another example: 
    2 multiplexed backups run simultaneously to a tape.
    1 backup fails 80% into the backup but the other one continuous to write and fills up the tape.
    Only the successful backup is stored in the catalog and reported by bpmedialist. 
    The space written by the failed backup cannot be overwritten until the successful backup has expired. (probably the explanation for 1147L3).

    It is NEVER a good idea to allow multiple retentions on media and must be manually enabled (not the default). 
    It only solves media availability in the short term.
    In the long run, you are stuck with multiple FULL tapes that cannot be overwritten because some images have not yet expired.

    Please read through these excellent blogs:

    Understanding how NetBackup writes to a tape.

    NetBackup and just how Multiple streams to a tape drive works.

  • Tauhid's avatar
    Tauhid
    Level 4
    10 years ago

    Many many Thaks.

    Excellent post by Marianne. I got many concepts.

    But one thing.

    I didn't check compression option for any policy (see the image). After that How 1100L3 writes more than 800GB data?

    Does any other compression technique is working beside this.?

    compression.JPG

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    The compression option is the policy is Netbackup software compression. Tape drives compress data internal by hardware (as I metioned in my last post) , Netbackup does not know about this - nor does it require to do so.

    Do not enable software compression in the policy - you will get a decrease in performance and the capacity on tapes will drop becuase double compression does not improve anything. Like ziping a zip file ...

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago

    As per Nicolai's excellent post - compression is by default enabled on the tape drives.
    We call this hardware compression.

    You will probably find this information in the documentation for your tape library and tape drives.

  • Tauhid's avatar
    Tauhid
    Level 4
    10 years ago

    Many Thanks for sharing compression concept.

    So lastly,I make a concept that,  a 800GB LTO3 data cartridge can contain maximum 1.6TB data with compression (as said 2:1 compression)

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    No - LTO3 has a native capacity of 400G (800G with 2:1 compression).

    For LTO capacity see - under "generation"

    http://en.wikipedia.org/wiki/Linear_Tape-Open 

     

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago

    It seems that you are getting 1:3 compression.

    Native capacity for LTO3 is 400GB.
    1:2 compression will give you 800GB.

    PS: 
    You may want to Google 'lto 3 cartridge capacity'.

  • jim_dalton's avatar
    jim_dalton
    Level 6
    10 years ago

    Compression is data dependant and algorithm dependant, there are no hard and fast rules, the numbers quoted are "typical", but if you have highly compressed data you wont get the "typical". Conversely if its highly compressable you can get way more data on there than 2:1. You can test this if you feel the urge by using /dev/zero and you will find you can fit a staggering amount of data on your 400G media. Jim