What's taking up space in my "disk pools"

[manatee]

***** Moved to new discussion from https://www-secure.symantec.com/connect/forums/what-disk-pool-name  *****

the NBU environment was turned over (more like dropped on my lap) so i'm still learning the ropes.

i don't know how to answer you but i do know right after a disk backup, it gets dedup to tapes. i also know that images stay for at least a week on disk before expiring.

what i'm after is, if the images expire after a week, what's taking up space in my "disk pools"? isn't it that when a policy mentions expire after a year, that refers to the backup on tape? and not on disk?

[Mark_Solutions]

We need a bit more information about your system to know what is happening here.

Is your disk basic, advanced or de-dupe (PureDisk or MSDP)?

Are you using Storage Lifecycle Policies to backup to disk and duplicate to tape or are they just basic disk with temproary staging setup.

This makes all the difference in being able to answer your questions as the disk capacity can be managed in many ways - most of which like to keep the disk as full as possible.

The output of the following commands will help us (both in veritas\netbackup\bin\admincmd\ directory):

bpstulist -g -U

bpschedule -U

nbstl -L

Thanks

[manatee]

sorry for the very late reply.

we use PureDisk.

we use SLP to backup to disk then dedup to tapes.

bpstulist -g -U

Label:                ovmmanager-hcart2-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      ovmmanager
Number of Drives:     2
On Demand Only:       no
Max MPX/drive:        1
Density:              hcart2 - 1/2 Inch Cartridge 2
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB

Label:                ovmmanager-pd-1
Storage Unit Type:    Disk
Storage Unit Subtype: DiskPool (6)
Host Connection:      _STU_NO_DEV_HOST_
Concurrent Jobs:      16
On Demand Only:       yes
Max MPX:              1
Max Fragment Size:    51200 MB
Block Sharing:        yes
Ok On Root:           no
Disk Pool:            NBU-PureDisk-1
Snapshots:            no
Replication Primary:  no
Replication Source:   no
Replication Target:   no
Mirror            :   no
Independent       :   no
__GROUPS__

bpschedule -U -> no output

nbstl -L
                                Name: Monthly_PureDisk-to-Tape
                 Data Classification: Gold
            Duplication job priority: 0
                               State: active
                             Version: 2
 Operation  1              Use for: 0 (backup)
                             Storage: ovmmanager-pd-1
                         Volume Pool: (none specified)
                        Server Group: (none specified)
                      Retention Type: 0 (Fixed)
                     Retention Level: 0 (1 week)
               Alternate Read Server: (none specified)
               Preserve Multiplexing: false
      Enable Automatic Remote Import: false
                               State: active
                              Source: 0 (client)
                        Operation ID: (none specified)
                     Operation Index: 1
 Operation  2              Use for: 1 (duplication)
                             Storage: ovmmanager-hcart2-robot-tld-0
                         Volume Pool: NetBackup
                        Server Group: Any
                      Retention Type: 0 (Fixed)
                     Retention Level: 8 (1 year)
               Alternate Read Server: (none specified)
               Preserve Multiplexing: true
      Enable Automatic Remote Import: false
                               State: active
                              Source: Operation 1 (backup:ovmmanager-pd-1)
                        Operation ID: (none specified)
                     Operation Index: 2
------------------------------------------------------------------------
                                Name: SLP-PureDisk-to-Tape
                 Data Classification: Gold
            Duplication job priority: 0
                               State: active
                             Version: 5
 Operation  1              Use for: 0 (backup)
                             Storage: ovmmanager-pd-1
                         Volume Pool: (none specified)
                        Server Group: (none specified)
                      Retention Type: 0 (Fixed)
                     Retention Level: 0 (1 week)
               Alternate Read Server: (none specified)
               Preserve Multiplexing: false
      Enable Automatic Remote Import: false
                               State: active
                              Source: 0 (client)
                        Operation ID: 0
                     Operation Index: 1
 Operation  2              Use for: 1 (duplication)
                             Storage: ovmmanager-hcart2-robot-tld-0
                         Volume Pool: Monthly
                        Server Group: Any
                      Retention Type: 0 (Fixed)
                     Retention Level: 3 (1 month)
               Alternate Read Server: ovmmanager
               Preserve Multiplexing: false
      Enable Automatic Remote Import: false
                               State: active
                              Source: Operation 1 (backup:ovmmanager-pd-1)
                        Operation ID: 0
                     Operation Index: 2
------------------------------------------------------------------------
                                Name: Service_Desk_To_Tape
                 Data Classification: Gold
            Duplication job priority: 0
                               State: active
                             Version: 0
 Operation  1              Use for: 0 (backup)
                             Storage: ovmmanager-pd-1
                         Volume Pool: (none specified)
                        Server Group: (none specified)
                      Retention Type: 0 (Fixed)
                     Retention Level: 0 (1 week)
               Alternate Read Server: (none specified)
               Preserve Multiplexing: false
      Enable Automatic Remote Import: false
                               State: active
                              Source: 0 (client)
                        Operation ID: (none specified)
                     Operation Index: 1
 Operation  2              Use for: 1 (duplication)
                             Storage: ovmmanager-hcart2-robot-tld-0
                         Volume Pool: NetBackup
                        Server Group: Any
                      Retention Type: 0 (Fixed)
                     Retention Level: 3 (1 month)
               Alternate Read Server: (none specified)
               Preserve Multiplexing: true
      Enable Automatic Remote Import: false
                               State: active
                              Source: Operation 1 (backup:ovmmanager-pd-1)
                        Operation ID: (none specified)
                     Operation Index: 2
------------------------------------------------------------------------
                                Name: one-time
                 Data Classification: (none specified)
            Duplication job priority: 0
                               State: active
                             Version: 1
 Operation  1              Use for: 0 (backup)
                             Storage: ovmmanager-pd-1
                         Volume Pool: (none specified)
                        Server Group: (none specified)
                      Retention Type: 0 (Fixed)
                     Retention Level: 1 (2 weeks)
               Alternate Read Server: (none specified)
               Preserve Multiplexing: false
      Enable Automatic Remote Import: false
                               State: active
                              Source: 0 (client)
                        Operation ID: (none specified)
                     Operation Index: 1
 Operation  2              Use for: 1 (duplication)
                             Storage: ovmmanager-hcart2-robot-tld-0
                         Volume Pool: Monthly
                        Server Group: Any
                      Retention Type: 0 (Fixed)
                     Retention Level: 9 (infinity)
               Alternate Read Server: ovmmanager
               Preserve Multiplexing: false
      Enable Automatic Remote Import: false
                               State: active
                              Source: Operation 1 (backup:ovmmanager-pd-1)
                        Operation ID: (none specified)
                     Operation Index: 2
 

[watsons · Accepted solution]

With your first post saying 1 year expiration, I am taking you refer to this SLP:

                Name: Monthly_PureDisk-to-Tape
 Operation  1              Use for: 0 (backup)
                             Storage: ovmmanager-pd-1
                     Retention Level: 0 (1 week)
 Operation  2              Use for: 1 (duplication)
                             Storage: ovmmanager-hcart2-robot-tld-0
                     Retention Level: 8 (1 year)

Looking at this setting, your backup (1st disk copy) is kept for 1 week, then duplicate to tape (2nd copy) and keep for 1 year. The concept with SLP is that you need to make sure 2nd operation "duplicate to tape" completed successfully, only then your (1st) disk copy will be expired after a week, if duplicate fails the 1st copy will always be there and won't get expired. 

From your setting, there won't be a disk copy you keep for 1 year long. The most is 2 weeks for this SLP: "one-time"

[Marianne]

So, if we go back to your question:  what's taking up space in my "disk pools"?

The 1st thing to check as per watsons excellent post is whether duplications are successful.

If so, you may want to read up on the expiration/cleanup process on dedupe.

Some basics explained here: 

 basics of deduplication:  http://www.symantec.com/docs/HOWTO36285

Catalog references to expired backups is removed, but as long as there are links to actual data from unexpired backups, the deduped data is not removed.

This TN describes the dedupe cleanup process:  http://www.symantec.com/docs/TECH124914 

There are all sorts of patches and TNs about cleanup not working the way it should, so, important to ensure that you are on latest patch levels to ensure that you have all fixes and enhancements.

[Mark_Solutions]

Do bear in mind that as you use de-duplication that expiry is not the same as normal disk expiry.

So say we backup 10TB the first week .. this creates it de-dupe database / files etc and maybe writes 8TB actually on disk.

The next week we run the 10TB of backups again .. but most of that is already known to the de-dupe database so we actuallt write very little.

Now at this point if we expire the first weeks backups we will not free up any disk space as all of the pointers are included in the second weeks backups.

Generally de-dupe tends to stay fairly constant in size once you have thrown a few backups at it... and generally grows a little over time if all clients / backup types stay the same.

The other point is that a full cleanup / compaction / internal defrag etc. only occurrs on a one month cycle so if you expire after 1 week you may not get that space back for another 3 weeks anyway .. for this reason I never advise my customers to have such a small retention period on disk .. 28 days is a minimum.

If you want to run the cleanup processes yourself on a weekly basis you can do and this will recoup your disk space .. but i would advise a longer retention on de-dupe to start with.

Hope this makes sense and helps

[manatee]

thanks all for the replies. that's a lot to digest :)