Cant compress Netbackup Catalog DB

[Raghav_tv]

Hello Everyone,

We had a problem with free space on a Local Drive of a Master Server, hence we had to migrate few images to another drive allocated by SAN Team. ALTPATH has also been set on the source locartion.

Now the problem is after Catalog DB is migrated, data is in Uncompressed state and even after running ---> bpimage -compress -allclients or bpimage -compress -client <client name>    Catalog DB is still in uncomressed state.. Can someone help ?

Regards,
Raghav

[Marianne]

What is the value in Host Properties -> Master -> Global Attributes -> Compress catalog interval ?

Only images older than the value in this field will be compressed.

Compression will automatically be done next time Image Cleanup job runs (default is every 12 hours).

[SymTerry]

Hello,

HOWTO34441 talks about just this. Please note that Symantec discourages manually compressing or decompressing catalog backups using bpimage -[de]compress or any other method. Manually compressing or decompressing a catalog backup while any backup (regular or catalog) is running results in inconsistent image-catalog entries.

As Marianne said, Control image-catalog compression by setting the Global Attributes property, Compress Catalog Interval

[Raghav_tv]

Thanks for your response Marianne

Compress Catalog Intervel is set to 1 and several image cleanup jobs ran as well, we had moved images on last friday (11th July14). still image folder size is uncompressed on the new Disk drive.

[Marianne]

Please create bpdbm log folder under netbackup\logs and restart NBU Database Service.

After next Image Cleanup job, please copy bpdbm log file to bpdbm.txt and upload as File attachment.

PS:
If images have been moved to a new drive with more space, why do still need to compress images?

[Raghav_tv]

Size of the folder at source location was 470GB in size and compressed size was around 170GB.

Size of the new drive is 500GB only.

[Walker_Yang1]

Hi, Raghav_tv

Here are my suggestions:

1. If possbile, please restart NBU services. Not sure if it's necessary after setting Compress catalog interval property.

2. Please try to run manually the following command to kick off the compression operation. From job detail, you could see some info related to image compression.

bpimage -cleanup -allclients

---cleanup
Deletes expired images, compresses the images that are scheduled to be
compressed, and prunes the TIR information from the specified images.

3. Please check related compress tool on master server, as netbackup will use compression command on OS, if no any compression tool on OS, it should not compress image actually, though you'll see compression info in step 2. e.g. gzip, ncompress, zip, etc.

Please share it, if you have any progress.

Thanks

[Raghav_tv]

We did restarted NBU services and ran bpimage cleanup -allclients as well but there is no change in size of images folder on the new Disk. Both Size and Size on Disk are still same (470GB in Size)

Before we moved the image folder, size and size on Disk were different. Size was around 470GB and size on Disk was 170GB in size.

[Raghav_tv]

Compress catalog is set since long time and no changes were made to it.

And its not the complete image folder that was moved to newly allocated Disk, but only one image folder from c:\program files\veritas\netbackup\db\images was moved to new Disk.

[Walker_Yang1]

Hi, Raghav_tv As Marianne said, could you post bpdbm log? where it explicitly log why (or why not) we are compressing an image Run "bpimage -cleanup -allclients" or wait next image cleanup job, and collect bpdbm log on master. In addition, if compression image is successful, all *.f files will be stored as *.f.Zl, Zl means the image was compressed. do you see this type of files? for 76, it uses embedded tool to compress image. for 75, it uses OS' tool or external tool. in my 7602 environment, the images older than 86400 seconds have been compressed to *.Zl on new location. Thanks

[Raghav_tv]

Hello Walker,

I do see some errors in BPDBM log file. I've enclosed the attachment.

And compressed image files are still being stored as *.f only.

Thanks

[Marianne]

Best to log a support call.

Errors like this one says that file cannot be found:

00:17:06.234 [9052.7164] <16> compress_dbfile: cannot lstat() Q:\Netbackup\Catalog\Images\hoidpns02dm4\1405000000\catstore\HO_EMC-SAN-HOIDPNS02DM4_EV10_OPEN_VAULT_1405235313_FULL.f_imgRecord0: Invalid argument (22)

00:25:51.201 [12280.11092] <16> read_info_file: EXCHANGE_SOURCE corrupt: expected range 0-3 got -1
00:25:51.217 [12280.11092] <16> cred_allowed_by_image: get_dir_list(C:\Program Files\Veritas\NetBackup\db\images) failed: 227
00:25:51.217 [12280.11092] <16> cred_allowed_by_snapshot: get_dir_list(C:\Program Files\Veritas\NetBackup\db\snapshot\homwplev01.ad.bgep.co.uk) failed: 227

This does not look good either:

08:16:59.691 [10328.8960] <16> read_info_file: EXCHANGE_SOURCE corrupt: expected range 0-3 got -1

Here we see that compression of some files worked, but logging level is probably too low to tell us which image files:

00:21:12.128 [9052.7164] <4> ImageDelete::~ImageDelete: deleted 86 expired records, compressed 21, tir removed 0, deleted 67 expired copies

08:17:06.289 [10328.8960] <4> ImageDelete::~ImageDelete: deleted 50 expired records, compressed 5, tir removed 0, deleted 39 expired copies

So, please log a call.

[Walker_Yang1]

hi, Raghav_tv

Sorry for late response.

In support of Marianne's suggestion.

The error code "Invalid argument" for lstat, it seems to be a rare error condition, don't know why.

In addition, it indeed shows that some images are getting compressed.

00:21:12.128 [9052.7164] <4> ImageDelete::~ImageDelete: deleted 86 expired records, compressed 21, tir removed 0, deleted 67 expired copies

don't you have any images get compressed?

Thanks

[Raghav_tv]

We have logged call with Symantec and its with Backline..Still inprogress...!!

Thanks everyone for your time.

[Raghav_tv · Accepted solution]

Images get decompressed during copy operation. Netbackup is unaware of this NTFS compression. Assuming that the image is already compressed(going with image header), NBU doesnt attempt to compress it again. So, we first have to decompress the backup images and then kick off compression again.

Step 1:- bpimage -decompress -client <clientname>

Step 2:- bpimage -compress -client <clientname>

Now, all images are compressed :)