Netbackup catalog big and no change in catalog size after compression activated

[Noor_Toorabally]

Hello everyone,

I am facing an issue as follows on netbackup 6.5.6:

My catalog size is 323GB. I previously activated catalog compression for backups older that 30days, about 10,000 images were compresed according to the image cleanup job but the size of the catalog remained exactly the same.

Some information:

Number of images:

[root@sesegx10 images]# bpimagelist -d 10/10/2000 -e 10/10/2015 | wc -l
63822
[root@sesegx10 images]# du -sh /u030/netbackup/db/images/
323G    /u030/netbackup/db/images/
 

Is there any way to solve this?

Rgds,

Noor

[RamNagalla]

hi,

I would recommend you to go for catalog archiving, 

Catalog archiving is meant to be used to reclaim disk space used by the files file of images that backed up many thousands of files and that also have infinite or multi-year retention periods.  This allows the large files files for these images to be removed from disk while the image header files remain in the image catalog. It should not be used as a method to reclaim disk space when a catalog filesystem fills up. For those cases, investigate catalog compression or adding disk space and growing the filesystem.

check the below T/N

http://www.symantec.com/business/support/index?page=content&id=TECH36433

http://www.symantec.com/business/support/index?page=content&id=TECH36434

[Nicolai]

My recommendation would be to extend the catalog area.

Both HP-UX, AIX Solaris and Linux has tool for extending file system on the fly.

[Noor_Toorabally]

So from what i understand, if the compression do not change the size of my catalog, I have to resort to archiving or extending the filesystem? But it is weird that the compression don't change the size at all as if it did not take effect. I would be grateful for any other suggestion, because the size of the catalog concerns us due to the length and space occupied by the catalog backup.

[Yasuhisa_Ishika]

Please look under images directly if catalog files are actualy compressed.
If I remember right, you can distinguish compress catalog file from file's extension.

[Jaykullar]

Sorry to Hi-Jack, but do the NBU services on the master require a restart when enabling compression or changing the interval field? In NBU 7.1

Thanks,

[Stumpr2]

Noor,

Did you run bpimage -cleanup -allclients?

[Noor_Toorabally]

Hello,

How can i distinguish if it is compressed, example:

OLD images that shoul have been compressed:

[root@sesegx10 1265000000]# pwd
/usr/openv/netbackup/db/images/seppux06/1265000000
[root@sesegx10 1265000000]# ls -lrt
total 16
-rw------- 1 root root   72 Feb  4  2010 LESS3-APPS_1265316545_FULL.f
-rw------- 1 root root   72 Feb  4  2010 LESS3-APPS_1265316153_FULL.f
drwxr-xr-x 3 root root 4096 Feb 12  2010 tmp
drwx------ 2 root root 4096 Jun  1  2010 catstore
-rw-rw-rw- 1 root root    0 Feb 27 13:19 LESS3-APPS_1265316545_FULL.lck
-rw-rw-rw- 1 root root    0 Feb 27 13:19 LESS3-APPS_1265316153_FULL.lck
[root@sesegx10 1265000000]# du -sh *
95M     catstore
4.0K    LESS3-APPS_1265316153_FULL.f
0       LESS3-APPS_1265316153_FULL.lck
4.0K    LESS3-APPS_1265316545_FULL.f
0       LESS3-APPS_1265316545_FULL.lck
8.0K    tmp
 

NEW images that should not have been compressed:

[root@sesegx10 1362000000]# pwd
/usr/openv/netbackup/db/images/seppux06/1362000000
[root@sesegx10 1362000000]# ls -lrt
total 452
-rw-rw-rw- 1 root root      0 Feb 28 01:54 linux_seppux06_apps_1362031839_INCR.lck
-rw------- 1 root root 445235 Feb 28 01:54 linux_seppux06_apps_1362031839_INCR.f
-rw-rw-rw- 1 root root      0 Feb 28 11:55 DBA-LPRE01-PGI-PRE-PROD-APPS_1362070466_FULL.lck
drwxr-xr-x 3 root root   4096 Feb 28 12:29 tmp
-rw------- 1 root root     72 Feb 28 12:29 DBA-LPRE01-PGI-PRE-PROD-APPS_1362070466_FULL.f
drwx------ 2 root root   4096 Feb 28 12:29 catstore
[root@sesegx10 1362000000]# du -sh *
95M     catstore
4.0K    DBA-LPRE01-PGI-PRE-PROD-APPS_1362070466_FULL.f
0       DBA-LPRE01-PGI-PRE-PROD-APPS_1362070466_FULL.lck
440K    linux_seppux06_apps_1362031839_INCR.f
0       linux_seppux06_apps_1362031839_INCR.lck
8.0K    tmp
 

[Noor_Toorabally]

Hello,

Yes i ran bpimage -cleanup -allclients and also bpimage -prunetir -cleanup -allclients:

Details status in activity monitor shows that 2471 images are compressed but the catalog size is unchanged.

Another point is that if i run the command again after some seconds, i get the same output with the same amount of images compressed:

02/28/2013 11:28:44 - Info bpdbm (pid=30289) image catalog cleanup
02/28/2013 11:28:44 - Info bpdbm (pid=30289) Cleaning up tables in the relational database
02/28/2013 11:28:44 - Info bpdbm (pid=30289) deleting images which expire before Thu Feb 28 11:28:43 2013 (1362068923)
02/28/2013 11:30:19 - Info bpdbm (pid=30289) deleted 0 expired records, compressed 2471, tir removed 0, deleted 0 expired copies
the requested operation was successfully completed  (0)

Thanks

[Noor_Toorabally]

Hello,

No restart of the service is required, but you should run the command for the image cleanup job to kick in:

bpimage -cleanup -allclients

Thanks

[Stumpr2]

I would reboot the master. Thats what I would do.

[Noor_Toorabally]

We  rebooted in vain

[Yasuhisa_Ishika · Accepted solution]

Sorry to be late. image db files seems not being compressed.

Do you run any script like reporting that refers contents of each backup images, or daily operation that looks into backup contents? Each time someone looks into contents of backup image NetBackup uncompress image db files related to the request.

For confirmation, please tell us what value you set to Compress Catalog Interval.

Any chance your system does not have compress command in it. Accoring to the statement in Administrator's Guile Volume I, NetBackup uses compress command to compress db files.

[Marianne]

I agree with Yasuhisa - compression is not a long-term solution. 

In an ever-growing environment, best to add more space. 

PS: PLEASE upgrade your environment! NBU 6.x is no longer supported.

[Noor_Toorabally]

Thank you all for your reply, the netbackup GUI even in 7.5.0.4 shows that the images are successfully compressed which was in fact not true.

The netbackup images cleanup job, with compression enabled, uses the UNIX compress command to make the catalog images compression.

This command did not exist on the server. After installing the ncompress rpm, I ran the images cleanup manually again and my catalog size decreased by more than half its size. The compression is still in progress, I shall post the final size once the job is complete.

Bottom line: In order for Netbackup catalog compression to work, the ncompress rpm has to be installed if it does not exist.

Thank you Yasuhisa Ishikawa