List Images On DiskPool Before Phase I Import

Sorry, not had time to look at this today.

What sdo has shown is my understanding. 1 week is standard retention after phase one import, and it can be extended. 

...and with a date change in to the future, and bpimage cleanup:

C:\Users\Administrator>date
The current date is: 06/08/2015
Enter the new date: (dd-mm-yy) 30/08/2015

C:\Users\Administrator>date /t
30/08/2015

C:\Users\Administrator>bpup -f -v
NetBackup 7.7 -- Startup Utility
Starting services
> NetBackup Legacy Network Service
> NetBackup Legacy Network Service -- STARTED
...
> Netbackup Bare Metal Restore Boot Server
> Netbackup Bare Metal Restore Boot Server -- STARTED
Start up completed successfully.

C:\Users\Administrator>bpimagelist -d 01/01/1970 00:00:00 -option ONLY_PRE_IMPORT -idonly
Time:      06/08/2015 19:53:41   ID: hserver_1438887221   FULL (0)
Time:      06/08/2015 18:20:13   ID: hserver_1438881613   FULL (0)

C:\Users\Administrator>bpimage -cleanup -allclients

C:\Users\Administrator>bpimagelist -d 01/01/1970 00:00:00 -option ONLY_PRE_IMPORT -idonly
Time:      06/08/2015 18:20:13   ID: hserver_1438881613   FULL (0)

...which shows that the phase 1 image which had its date set in to the future has survived beyond one week, whereas the other (which did not have a future expiration set) did not survive and was auto-purged from the catalog as an incomplete (un-phased-2) phase 1 image..

apologies - the original retention of 1 day was from the original backup - I thought it best to test with a backup with a retention shorter than the auto 1 week retention that phase 1 applies - just to make sure.

...

...

...and as a slightly different side topic...

...after reverting the VM to original state/date... how about this... expiring the phase 1 (not yet phase 2) images from the catalog did not remove them from the basic disk STU:

C:\Users\Administrator>bpimagelist -d 01/01/1970 00:00:00 -idonly -option ONLY_PRE_IMPORT
Time:      06/08/2015 19:53:41   ID: hserver_1438887221   FULL (0)
Time:      06/08/2015 18:20:13   ID: hserver_1438881613   FULL (0)

C:\Users\Administrator>bpexpdate -backupid hserver_1438887221 -d 0
Are you SURE you want to delete hserver_1438887221 y/n (n)? y

C:\Users\Administrator>bpexpdate -backupid hserver_1438881613 -d 0
Are you SURE you want to delete hserver_1438881613 y/n (n)? y

C:\Users\Administrator>bpimage -cleanup -allclients

C:\Users\Administrator>bpdbjobs -summary
MASTER SERVER QUEUED REQUEUED ACTIVE SUCCESS PARTSUCC FAILED INCOMP SUSP WAITING_RETRY  TOTAL
master01           0   0      0     77       0        5      0      0    0     82

C:\Users\Administrator>bpimagelist -d 01/01/1970 00:00:00 -idonly -option ONLY_PRE_IMPORT
no entity was found

C:\Users\Administrator>dir F:\NBU_DSU2
Volume in drive F is NBU_Logs
Volume Serial Number is 5223-FA98
Directory of F:\NBU_DSU2
06/08/2015  20:09    <DIR>          .
06/08/2015  20:09    <DIR>          ..
06/08/2015  18:20       118,167,552 hserver_1438881613_C1_F1.1438881613.img
06/08/2015  18:20             1,024 hserver_1438881613_C1_F1.1438881613.info
06/08/2015  18:20               512 hserver_1438881613_C1_HDR.1438881613.img
06/08/2015  18:20             1,024 hserver_1438881613_C1_HDR.1438881613.info
06/08/2015  19:54       118,167,552 hserver_1438887221_C1_F1.1438887221.img
06/08/2015  19:54             1,024 hserver_1438887221_C1_F1.1438887221.info
06/08/2015  19:53               512 hserver_1438887221_C1_HDR.1438887221.img
06/08/2015  19:53             1,024 hserver_1438887221_C1_HDR.1438887221.info
               8 File(s)    236,340,224 bytes
               2 Dir(s)  19,932,418,048 bytes free 

...I wonder if OST and/or MSDP STUs would behave in a similar fashion.

I am not sure what the differences would be... all I know is that last time I did this with a different DiskPool and didn't get to the Phase II whithin the week, the images were wiped off disk.  I had a Sev 1 case open with IBM and Symantec on this for about a year while they tried to figure out where the images went and the RCA came down to that Symantec removed the images off disk beause the Phase II did not complete on all the images after the Phase I was complete.... Hence the reason I am leary about running a Phase I without knowing what NBU "can see" on the DiskPool before I run the Phase I.

After phase 1, NBU will know about the fragments, and most of the info about the backup.

What it doesn't know, is info about the individual files in the backup = the contents of the catalog .f file. This is the long part of an import where NBU looks at each file, and rebuilds the .f file (= list of files in the backup).

It's like a chapter in a book.

Phase one reads the chapter titles (the backup headers) - so a quick flick through the images/ fragments of the image

Phase two settles down for a good old read, where each word in the chapter is a file ...
 

Martin, was there ever a bug which caused phase 1 images which reach 7 days age being auto-expired AND also deleted/expunged from disk (i.e. any of BasicDisk, AdvancedDisk, PureDisk, OST) ?

I've had a quick search through the v7.1.0.4 release notes, and the EEB listings for v7.5 and v7.6 and v7.6.1 but I couldn't find anything along those lines... but maybe my searching is not too good :p

Regarding phase 1 imports, those images are only ever held for 7 days before being release again.  Disk based images are more finiky about being overwrite due to the very nature of disk images. As long as your OST devices was told to mark them as garbage, its own garbage cleanup would be responsible for deleting these from disk. If they were already expired, they already are at risk of being overwritten at a sector level (basic disk, adv disk) or being marked as garbage and subject to OST cleanup.

In regards to finding orphaned images

Maybe something by the way of bpstsinfo?

https://support.symantec.com/en_US/article.HOWTO103846.html

-comparedbandstu | -cdas

Compares the image information in the catalog to image information on the storage server physical media. -comparedbandstucompares the specified previous OpenStorage server(s) and volume(s) to the current OpenStorage server and volume.

This will read through that storage (same that phase 1 does) and compares to the EMM records and will list if EMM has something that the storage does not or if the storage has something that EMM does not.

So an example on my local msdp pool on an appliance.

bpstsinfo -cdas -stype PureDisk -ln PureDiskVolume -media_id @aaaao -sn ros5330-01

Results in nothing;

Exits with 

"
MAIN - Verify orphan list
MAIN - Compare DB and Stu complete"

Also some examples to pipe it correctly if you need to output it to a file.

https://support.symantec.com/en_US/article.TECH63720.html

Awesome....

I was under the impression that there was no media id until Phase I.  I ran that -listdv and was able to get the media id and then was able to use the -comparedbandstu

this confirmed to me the information I needed before I run the Phase I import.