cancel
Showing results for 
Search instead for 
Did you mean: 

How do I delete backups on an old DiskPool

khemmerl
Level 5

Hello everyone,

I hope you can help me out.  Several months ago, my company added a new MSA disk pool to replace our aging DataDomain disk pool.  All backup policies were reconfigured to stop using DataDomain and, originally, the usage on DataDomain dropped as backups aged and expired.  The problem is that the Data Domain usage only dropped to 14% and has been stuck there for over a month.  When I go into the NetBackup GUI (version 7.1) and navigate to "Media and Device Management" > Devices > Disk Pools, it displays 2.5677 TB used oiut of 18.1005 TB total (about 14%). 

I'm not sure why these backups are not expiring.  After doing some reading, I ran the following from D:\Program Files\Veritas\NetBackup\bin\admincmd on my NetBackup admin host:

bpimmedia -L -stype DiskPool -dp dd670
Backup-ID             Policy     ScheduleType
Copy  Frag  Expires     DiskType  DiskPoolName  DiskVolume  StorageServerName
-----------------------------------------------------------------------------
APP-SRV29_1357349450  APP_SRV29  FULL
1     1     1360027850  DiskPool  dd670         DDSTU1      inf-srv17
APP-SRV29_1357349451  APP_SRV29  FULL
1     1     1360027851  DiskPool  dd670         DDSTU1      inf-srv17
... (and on for hundreds of entries)

The server APP_SRV29 no longer exists so we certainly don't need to keep these backups (because all backups are for disaster recovery, not historical snapshots).  After some more reading, I found documents talking about bpexpdate.  Unfortunatetly, using this command gives me "no entry was found":

bpexpdate  -backupid APP-SRV29_1357349450 -d 0
Are you SURE you want to delete APP-SRV29_1357349450 y/n (n)? y
no entity was found

Oddly, even though the bpimmedia command shows the Policy of APP_SRV29, the bpimagelist shows nothing:

bpimagelist -policy APP_SRV29
no entity was found

I am very frustrated.  I have 2.6 TB of backups sitting in my disk pool that I can see but not clean up.  Can anyone offer me help on how to delete these backup IDs?  Any help would be greatly appreciated.

Ken

24 REPLIES 24

mph999
Level 6
Employee Accredited

Just had a thought :

If you can, stop NBU and copy all the files in /usr/openv/db/data and upload these.

Upload these files also (this is a 'hot' backup of the NBDB, same as what the catalog backup does.

The idea for this is it allows your DB to be started up on a test server - you have to add an alias of your histname to /etc/hosts, or pref rename the test server to have the same servername as yours, then edit the following files to have the same hostname:

/usr/openv/netbackup/bp.conf
/usr/openv/var/global/server.conf
/usr/openv/db/data/vxdbms.conf
/usr/openv/db/bin/servername

This was support can investigate what's going on by running commands directly without having to bother you via email/ webex - hopefully ...

 

 

mph999
Level 6
Employee Accredited

OK, so it seems fairly clear that although there are entries in the image/ copy/ frag tables, bpimagelist is not reporting them - log call time I think ( please post case number up here )

Call details to include:

 

Issue:

Several months ago, my company added a new MSA disk pool to replace our aging DataDomain disk pool.  All backup policies were reconfigured to stop using DataDomain and, originally, the usage on DataDomain dropped as backups aged and expired.  The problem is that the Data Domain usage only dropped to 14% and has been stuck there for over a month.  When I go into the NetBackup GUI (version 7.1) and navigate to "Media and Device Management" > Devices > Disk Pools, it displays 2.5677 TB used oiut of 18.1005 TB total (about 14%). 

Investigations and research so far:

(For explanation purposes, backupid APP-SRV29_1357349450 is used, though many others are affected)

bpimagelist -backupid <backupid> shows no entity found, but image is referenced in Image / Copy/ Frag tables of NBDB (808 /810/ 811.dat)  This was determined by nbdb_unload on NBDB and then searching .dat files for the ctime.

And

bpimagelist -client APP_SRV29 -d 01/01/2013 -U
no entity was found

 

 bpimagelist -l -d 01/04/13 -e 01/06/13

This returns some rows but not any for APP-SRV29.

bpimagelist -l -d 01/04/13 -e 01/06/13

IMAGE INF-SRV42 0 0 8 INF-SRV42_1357434038 INF-SRV42 13 *NULL* root
Yearly 0 3 1357434038 6640 1420506038 0 0 30119133 539 1 1 0 INF-SRV
42_1 0 3 0 0 0 *NULL* 0 0 0 0 0 0 0 *NULL* 0 0 0 *NULL* 633683 2 0 6
5251 0 0 *NULL* Disk-Disk-Tape-Yearly 3 1357434002 0 0 *NULL* *NULL*
HISTO -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
FRAG 3 1 30119133 0 2 6 2 APA290 inf-srv17 65536 4713499 1357413731
11 0 *NULL* 1420506038 0 65546 0 0 0 1 1420506038 1357441439 0 *NULL
* *N
IMAGE INF-SRV42 0 0 8 INF-SRV42_1357434037 INF-SRV42 13 *NULL* root
Yearly 0 3 1357434037 4639 1420506037 0 0 13360959 55285 1 1 0 INF-S
RV42L* 0 3 0 0 0 *NULL* 0 0 0 0 0 0 0 *NULL* 0 0 0 *NULL* 633682 2 0
 7246877 0 0 *NULL* Disk-Disk-Tape-Yearly 3 1357434002 0 0 *NULL* *N
ULL*
HISTO -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
FRAG 3 1 13360959 0 2 6 5 APA071 inf-srv17 65536 430877 1357350254 9
 0 *NULL* 1420506037 0 65546 0 0 0 1 1420506037 1357439355 0 *NULL*
*NUL
IMAGE INF-SRV42 0 0 8 INF-SRV42_1357434036 INF-SRV42 13 *NULL* root
Yearly 0 3 1357434036 2996 1420506036 0 0 6757899 35919 1 1 0 INF-SR
V42_* 0 3 0 0 0 *NULL* 0 0 0 0 0 0 0 *NULL* 0 0 0 *NULL* 633681 2 0
4714550 0 0 *NULL* Disk-Disk-Tape-Yearly 3 1357434002 0 0 *NULL* *NU
LL*
HISTO -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
FRAG 3 1 6757899 0 2 6 4 APA071 inf-srv17 65536 325282 1357350254 10
 0 *NULL* 1420506036 0 65546 0 0 0 1 1420506036 1357437398 0 *NULL*
*NUL
IMAGE INF-SRV42 0 0 8 INF-SRV42_1357434035 INF-SRV42 13 *NULL* root
Yearly 0 3 1357434035 6301 1420506035 0 0 14874879 7923 1 1 0 INF-SR
V42_* 0 3 0 0 0 *NULL* 0 0 0 0 0 0 0 *NULL* 0 0 0 *NULL* 633680 2 0
992471 0 0 *NULL* Disk-Disk-Tape-Yearly 3 1357434002 0 0 *NULL* *NUL
L*
HISTO -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
FRAG 3 1 14874879 0 2 6 6 APA071 inf-srv17 65536 639644 1357350254 0
 0 *NULL* 1420506035 0 65546 0 0 0 1 1420506035 1357440916 0 *NULL*
*NUL

 

bpimmedia does however show the images, but it cannot be expired with bpexpdate:

bpimmedia -L -stype DiskPool -dp dd670
Backup-ID             Policy     ScheduleType
Copy  Frag  Expires     DiskType  DiskPoolName  DiskVolume  StorageServerName
-----------------------------------------------------------------------------
APP-SRV29_1357349450  APP_SRV29  FULL
1     1     1360027850  DiskPool  dd670         DDSTU1      inf-srv17
APP-SRV29_1357349451  APP_SRV29  FULL
1     1     1360027851  DiskPool  dd670         DDSTU1      inf-srv17
... (and on for hundreds of entries)

 

The image is also found in nbstlutil stlilist output

"nbstlutil stlilist":

V7.0.1 I APP-SRV29_1357349450 Disk-Disk-Tape-Monthly 3
V7.0.1 I APP-SRV29_1357349451 Disk-Disk-Tape-Monthly 3

Though as the status is 3, it is no longer under SLP control, this of course should not affect the output of bpimagelist.

The server APP_SRV29 no longer exists so we certainly don't need to keep these backups (because all backups are for disaster recovery, not historical snapshots).  After some more reading, I found documents talking about bpexpdate.  Unfortunatetly, using this command gives me "no entry was found":

bpexpdate  -backupid APP-SRV29_1357349450 -d 0
Are you SURE you want to delete APP-SRV29_1357349450 y/n (n)? y
no entity was found

 

Please assist in determining why bpexpdate and bpimagelist don't find images that are contained in the image/ copy /frag tables.

After logging the call with the above details, please upload (or get ready to upload) :

nbdb_unload of DB (all files)

nbsu -c -t

khemmerl
Level 5

Thank you so much for all your help.  What I never mentioned was that in addition to moving away from DataDomain, my company is also in the process of moving away from NetBackup.  I thought we had support to the end of this month but just learned that I was mistaken and that our support contract with NetBackup is already expired.  This means I will not be able to open a support ticket for this problem. 

As there are off-site tapes that won't expire for another two years, I'm looking at leaving NetBackup server running but doing nothing until the last of the tapes expires.  The DataDomain device has a sanitize function that securely wipes the disks that I'll use before shutting down the two units, removing them from my primary and disaster recovery sites and recycling.  I would have preferred to have things perfectly neat and orderly but will have to brute-force it a bit instead. 

Lesson learned:  1) Start cleanup long before your license expires.  2) Make sure you know when your support actually expires.

Thanks again for all your help.  Sorry I'm not able to open a ticket to find the resolution - I would have liked to get to the bottom of this.

Ken

mph999
Level 6
Employee Accredited

Sorry to hear you are leaving NBU.

You could just delete the images from the disk directly.  I don't think this will cause any more issues - even on a fully working system, NBU would just delete the images from the catalog even if they aren't on the disk, it might throw an error somewhere but I think that would about be the worst of it.  Seeing you can;t delete the images, it'll just stay in that state, so you might as well reclaim some disk space manually.

Unfortunatley, unless one of the others comes up with some bright ideas I'm a bit stuck - this has got a bit 'complex' for a forum issue, needs to be a bit more hands on to make progress I think.

Mark_Solutions
Level 6
Partner Accredited Certified

Just realised that you are still on NetBackup 7.1 .. so that means there is one last possible solution here if you just want to get rid of everything from that disk pool.

First run:

nbdevquery -listdv -stype PureDisk

This will show all of your disk pools and their mediaid's (like @aaaae)

Once you have the mediaid for that pool use the following - using your mediaid in the command - do be sure of this though as it will get rid of everything in that disk pool!!!!!:

nbstlutil remove_all -mediaid @aaaae –force

See if that sorts it for you .. they dropped this command after 7.1 but if you are still on that version you can give it a try.