cancel
Showing results for 
Search instead for 
Did you mean: 

volume_cleanup issued on valid copy

alokjha
Level 3

Greetings,

I am seeing something peculiar in my NBU restores that has me puzzled. We are going through multiple backup-recover cycles and I don't see this happening for every restore but it has happened more than once.

We have an SLP (non-AIR) on the source side that backups up the primary copy to a local DataDomain storage. The duplication job that's part of the same SLP creates a 2nd copy of the backed up file at a remote DataDomain. Both the backup and duplication jobs seem to complete successfully.

After the SLP has completed successfully, I take a FULL catalog backup on the source side and restore this catalog to the remote side. The catalog restore completes successfully, but I am unable to verify copy 2 of a few backup IDs. Verify on those backup ids fails with the following error - 

07:03:44 INF - from host backup1.dmotorworks.com, no data in archive
07:03:44 INF - Verify of policy RMAN_prod_pdb01, schedule App90 (racdb01-backup.dmotorworks.com_1402716314) failed, tar had an unexpected error.

Further investigation (in the bpdm logs) revealed that during the catalog restore, this backup id was issued a volume_cleanup. 

10:04:26.884 [24949] <2> volume_cleanup_delete_fragment: deletion of racdb01-backup.dmotorworks.com_1402716314, copy number 2, fragment number 1, resume number 1 media id @aaaa@ succeeded

Running nbstlutil list on this backup id confirms that the copy should have a retention of 1 month and is not set to expire till July 14. I checked the date/time on the remote master server it has the right date/time.

Copy:
   Master Server       : backup1.dmotorworks.com
   Backup ID           : racdb01-backup.dmotorworks.com_1402716314
   Copy Number         : 2
   Copy Type           : 1 (DUPLICATE) 
   Expire Time         : 1405394714 (Mon Jul 14 22:25:14 2014)
   Expire LC Time      : 1405394714 (Mon Jul 14 22:25:14 2014)
   Try To Keep Time    : 1405394714 (Mon Jul 14 22:25:14 2014)
   Residence           : dc2cbu-dd02
   Copy State          : 5 (COPYCOMPLETE)

Any ideas on what could possibly be causing valid copies to get cleaned up on the target side during catalog restore? Thank you all!

 

1 ACCEPTED SOLUTION

Accepted Solutions

mph999
Level 6
Employee Accredited

I've not had a min to read this 'fully' (have to go out).

If your analysis is correct and at a very fast read through it looks like you have made a superb effort, they log a call very fast - as it's potentially data loss.

I would recommend to put /usr/openv/netbackup/bin/NOexpire touch file in place in the meantime.

View solution in original post

2 REPLIES 2

mph999
Level 6
Employee Accredited

I've not had a min to read this 'fully' (have to go out).

If your analysis is correct and at a very fast read through it looks like you have made a superb effort, they log a call very fast - as it's potentially data loss.

I would recommend to put /usr/openv/netbackup/bin/NOexpire touch file in place in the meantime.

alokjha
Level 3

Thanks Martin,

I put /usr/openv/netbackup/bin/NOexpire in place on both source and destination master servers per your suggestion.

Upon reading up a little bit about NOexpire, I took it out of the source side. Images are retaining upto their retention period without issues on the primary side. It is on the remote (DR) site that the anomalous behavior is being exhibited - so I left it on there.