Compression of catalogs not occurring

[Matt_Musgrove]

Hi,

I am in a pickle because I have nearly run out of space on /usr/openv on my Red Hat NetBackup 6.0 master/media server.

I have tried to implement catalog compression by updating the "Compress catalog interval" in global attributes but it is not occurring.  I discovered there were problems with compression with NBU 6.0 MP5 so I have upgraded to MP7 but the catalogs are still not compressing when the backup session completes. 

I have also tried on command line:

bpimage  -compress -client <client name>

I have also tried used the allclients option with the command above but nothing happens.

In order to kickstart the catalog compression I have been running a small backup job and waiting for it to complete, is this the right approach?

I would upgrade to 6.5 but I am concerned I don't have enough disk space to do this.

I would also like to prune the catalogs of backups older than 1 year.  I have changed the retention period for my monthlies from 3 years to 1 year but of course this doesn't affect backups that have already run.

many thanks
Matt

[Andy_Welburn]

Have you tried bpimage -cleanup -allclients ? Does that error out? This should manually run the scheduled process & appear as a job in the Activity Monitor (I presume that was the case for 6.0 MPx?)

Have you a file system that you could move the image catalog to?

Are your logs in the same directory? They can also be moved if necessary.

[Matt_Musgrove]

bpimage -cleanup is a NBU 6.5 command unfortunately for me. 

My catalogs are all in one location : /usr/openv/netbackup/db/images.  I guess it would be OK if I create a symbolic link for images to a shared storage device?

[Andy_Welburn]

Really?  (EDIT: altho' I do tend to be a bit blinkered sometimes relating everything to my environment!)

GENERAL ERROR: bpdbm dumping core during cleanup
http://seer.entsupport.symantec.com/docs/294354.htm


There is a T/N for moving catalog but for some reason the UNIX one is only for 6.5.x ???
http://seer.entsupport.symantec.com/docs/323700.htm

whereas the Windows one includes 6.0 also.

What about your logs?

[Marianne · Accepted solution]

Around since NBU 3.4:

http://seer.entsupport.symantec.com/docs/238023.htm

Details:
The bpimage command was changed in NetBackup DataCenter 3.4 so that in order to expire images from all the clients, it is necessary to execute "bpimage -cleanup -allclients", or run the "bpimage -cleanup" command on the specific client that also contains the images to expire.

[Matt_Musgrove]

I am pretty sure that command is 6.5 only.  In Activity Monitor I never get a cleanup job appear like I see in my 6.5 environments.

Nothing in problem reports relating to my compression problem. Nothing useful in the /usr/openv/logs either.

Thanks for the link, I think I'll copy the images off; upgrade to 6.5; compress the catalogs (hopefully!); and copy the images back. 

Was there an answer to pruning catalogs of the older images which I don't need? Will using bpexpdate to delete the media from the database also prune the image catalog?

Matt

[Andy_Welburn]

Maybe that bit came into effect after 6.0 ? Should still be a valid command tho'.

Expiring images/tapes should reduce the size of the image d/b as far as I'm concerned. But that could also require the image cleanup to be run - certainly in 6.5 it compresses plus deletes expired images, old TIR files etc.

[Matt_Musgrove]

Ah yes, the bpimage -cleanup -allclients is valid, it just doesn't appear in bpimage -help.  Doesn't reduce the size of my catalogs though :(

[Marianne]

*** EDIT ***
I see you have tried the command while I was busy typing this message...
**************************************************************************************
If you STILL don't believe us, here's another NBU 5.1 TechNote:
http://seer.entsupport.symantec.com/docs/236274.htm

To initiate manual cleanup, perform the following:

# /usr/openv/netbackup/bin/admincmd/bpimage -cleanup -allclients


Why not TRY the command??

Catalog cleanup should be logged in bpdbm log.

Please see what 'bpconfig -L' command produces (I don't have a 6.0 server around to test). I'm curious to know if the 'DB Clean Interval' and 'DB Clean Wait Time' are present.

Mail Admin:             *NULL*
Job Retry Delay:        10 minutes
Max Jobs/Client:        4
Backup Tries:           1 in 1 hours
Keep Logs:              20 days
Max drives/master:      0
Compress DB Files:      (not enabled)
Media Mnt Timeout:      0 minutes (unlimited)
Display Reports:        24 hours ago
Keep TIR Info:          1 days
Prep Interval:          4 hours (default)
Max Backup Copies:      2
DB Clean Interval:      12 hours
DB Clean Wait Time:     60 minutes
Policy Update Interval: 10 minutes

[Andy_Welburn]

If it makes you feel any better it doesn't on 6.5 either!

[Marianne]

Run a small test backup to kick-start compression...

Found this in Admin Guide I:

If you choose to compress the image catalog, NetBackup uses the compress command on the server to perform compression after each backup session. It does not make a difference to NetBackup if the backup session was successful. The operation occurs while NetBackup expires backups and before it runs the session_notify script and the backup of the NetBackup catalogs.

[Matt_Musgrove]

It works :)

I lowered the interval from 730 days to 500 days and reran bpimage -cleanup -allclients.
Disk space is being reclaimed as I type.  I was mislead by the output from bpcatlist which shows older backups. I don't see how they could have already been compressed? Any how, I don't really care.  Many thanks for your help :) 

I attach the output of bpconfig -L for completeness

Mail Admin:             *NULL*
Job Retry Delay:        10 minutes
Max Jobs/Client:        5
Backup Tries:           2 in 12 hours
Keep Logs:              28 days
Max drives/master:      0
Compress DB Files:      older than 500 days
Media Mnt Timeout:      0 minutes (unlimited)
Shared Timeout:      0 minutes(unlimited)
Display Reports:        24 hours ago
Keep TIR Info:          1 days
Prep Interval:          4 hours (default)
Max Backup Copies:      2
DB Clean Interval:      12 hours
Policy Update Interval: 10 minutes