12-31-2016 09:47 PM
Happy New Year!!
pardon the subject can't help it ;) i have several year end backup policies that have finished and am looking for how to find all the media it uses. i tried the catalog but i can't change the policy type to SQLServer. it is greyed out at MS-Windows.
how to find all the media used by a particular backup policy?
01-01-2017 09:10 AM
01-01-2017 10:09 PM
i keep getting "no entity was found" so i tried different variations of the command:
[root@master admincmd]# ./bpimagelist -media -U -policy Yearly_DC no entity was found [root@master admincmd]# ./bpimagelist -media -U -policy Yearly_DC -pt MS-Windows no entity was found [root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc01 -policy Yearly_DC -pt MS-Windows no entity was found [root@master admincmd]# ./bpimagelist -media -U -tape -d 12/31/2016 -e 1/1/2017 -client dc01 -policy Yearly_DC -pt MS-Windows no entity was found [root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 16:00:00 -e 1/1/2017 23:59:00 -client dc01 -policy Yearly_DC -pt MS-Windows no entity was found [root@master admincmd]# ./bpimagelist -media -U no entity was found [root@master admincmd]# ./bpimagelist -U -stl_incomplete -policy Yearly_DC no entity was found [root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 16:00:00 -e 01/01/2017 23:59:59 -client dc01 -policy Yearly_DC -pt MS-Windows no entity was found [root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 00:00:00 -e 01/01/2017 23:59:59 -client dc01 -policy Yearly_DC -pt MS-Windows no entity was found
and still same errors. i double checked in the activity monitor regarding that backup policy and there is a backup done on the day of Dec 31st 2016.
01-02-2017 01:51 AM
01-02-2017 04:40 AM
i hope i got it right.
you can see the backup finished successfully.
01-02-2017 05:20 AM
Can you determine which retention level was used? Maybe the backup has already expired?
01-02-2017 05:30 AM
i used SLP and it is set to expire after copy on disk, then infinity on tape.
01-02-2017 05:53 AM
dc02 and dc03 are listed in activity monitor, and dc01 is not. Looks like a backup of dc01 did not run.
What does this show:
# ./bperror -all -d 12/31/2016 00:00:00 -e 01/02/2017 23:59:59 -client dc01 -U
01-02-2017 06:07 AM
i'm sorry that was a typo. should be dc03.
[root@ovmmanager admincmd]# ./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc03 -policy Yearly_DC -pt MS-Windows
no entity was found
the command you provided gave a long list (shortened) below:
TIME SERVER/CLIENT TEXT 12/31/2016 02:00:00 master dc03 started backup job for client dc03, policy Monthly_Windows_Domain_Controller, schedule Monthly on storage unit master-pd-1 12/31/2016 02:00:03 master dc03 started backup job for client dc03, policy Yearly_DC, schedule Yearly on storage unit master-pd-1 12/31/2016 02:00:38 master dc03 started backup job for client dc03, policy Monthly_Windows_Domain_Controller, schedule Monthly on storage unit master-pd-1 12/31/2016 02:00:39 master dc03 started backup job for client dc03, policy Yearly_DC, schedule Yearly on storage unit master-pd-1 12/31/2016 02:01:06 master dc03 begin writing backup id dc03_1483138838, copy 1, fragment 1, destination path PureDiskVolume 12/31/2016 02:01:06 master dc03 begin writing backup id dc03_1483138839, copy 1, fragment 1, destination path PureDiskVolume 12/31/2016 04:19:45 master dc03 from client dc03: WRN - can't open object: System State:\Automated System Recovery\BCD\BCD (WIN32 3758134305: Unknown error) 12/31/2016 04:19:45 master dc03 from client dc03: ERR - Error encountered while attempting to get additional files for System State:\ 12/31/2016 04:33:30 master dc03 successfully wrote backup id dc03_1483138839, copy 1, fragment 1, 17339392 Kbytes at 1896.385 Kbytes/sec 12/31/2016 04:33:30 master dc03 begin writing backup id dc03_1483138839, copy 1, fragment 2, destination path PureDiskVolume 12/31/2016 04:33:30 master dc03 successfully wrote backup id dc03_1483138838, copy 1, fragment 1, 17326080 Kbytes at 1894.924 Kbytes/sec 12/31/2016 04:33:30 master dc03 begin writing backup id dc03_1483138838, copy 1, fragment 2, destination path PureDiskVolume 12/31/2016 04:46:07 master dc03 successfully wrote backup id dc03_1483138839, copy 1, fragment 2, 3332096 Kbytes at 4400.916 Kbytes/sec 12/31/2016 04:46:07 master dc03 begin writing backup id dc03_1483138839, copy 1, fragment 3, destination path ... 01/01/2017 21:17:14 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:17:45 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:18:16 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:18:47 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:19:18 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:19:33 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:20:07 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:20:29 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:21:00 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:21:31 master dc03 image copy is not ready, retry attempt: 176 of 500 object is busy, cannot be closed 01/01/2017 21:21:39 master dc03 StorageServer=PureDisk:master; Report=PDDO Stats (multi-threaded stream used) for (master): scanned: 17343827 KB, CR sent: 75157 KB, CR sent over FC: 0 KB, dedup: 99.6%, cache disabled 01/01/2017 21:21:40 master dc03 Using the media server to write NBU data for backup dc03_1483282848 to master 01/01/2017 21:21:41 master dc03 StorageServer=PureDisk:master; Report=PDDO Stats for (master): scanned: 2 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%, cache disabled 01/01/2017 21:21:51 master dc03 CLIENT dc03 POLICY Daily_DC SCHED Daily EXIT STATUS 0 (the requested operation was successfully completed) 01/01/2017 21:21:57 master dc03 from client dc03: cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.ruh1 dc03_1483282800.1.0 01/01/2017 21:21:58 master dc03 CLIENT dc03 POLICY Daily_DC SCHED Daily EXIT STATUS 0 (the requested operation was successfully completed)
01-02-2017 06:29 AM - edited 01-02-2017 06:31 AM
From the bperror log we see two backups start, one for each policy:
Monthly_Windows_Domain_Controller
Yearly_DC
...and we see two image IDs for:
dc03_1483138838
dc03_1483138839
...but from bperror we cannot determine which image ID is for which policy.
Later on we see this image end ok: dc03_1483282848
...but it looks to be for policy: Daily_DC
Seems like you have (had) client dc03 in at least three policies. We're not going to be able to determine an image name from the above.
.
Let's try this:
# ./bpdbjobs -jobid 515406 -all_columns
01-02-2017 06:34 AM
it gave a long list:
[root@master admincmd]# ./bpdbjobs -jobid 515406 -all_columns 515406,0,3,0,Yearly_DC,Yearly,dc03,master,1483138839,0000014265,1483153104,master-pd-1,1,,48344176,112317,,100,8709,root,1,13,0,0,root,master,-1,-1,0,,,3,ALL_LOCAL_DRIVES,System State:\\,,1,8709,master-pd-1,master,1483138839,0000014266,1483153105,0,the requested operation was successfully completed,29,12/31/2016 02:00:39 - Info nbjm(pid=32742) starting backup job (jobid=515406) for client dc03\, policy Yearly_DC\, schedule Yearly,12/31/2016 02:00:39 - estimated 42741159 kbytes needed,12/31/2016 02:00:39 - Info nbjm(pid=32742) started backup (backupid=dc03_1483138839) job for client dc03\, policy Yearly_DC\, schedule Yearly on storage unit master-pd-1,12/31/2016 02:00:40 - started process bpbrm (8709),12/31/2016 02:00:40 - Info bpbrm(pid=8709) dc03 is the host to backup data from,12/31/2016 02:00:40 - Info bpbrm(pid=8709) reading file list for client,12/31/2016 02:00:41 - Info bpbrm(pid=8709) starting bpbkar on client,12/31/2016 02:00:40 - connecting,12/31/2016 02:00:41 - connected; connect time: 000:00:01,12/31/2016 02:01:04 - Info bpbkar(pid=4752) Backup started,12/31/2016 02:01:04 - Info bpbrm(pid=8709) bptm pid: 8783,12/31/2016 02:01:04 - Info bpbkar(pid=4752) change time comparison:<disabled>,12/31/2016 02:01:04 - Info bpbkar(pid=4752) archive bit processing:<enabled>,12/31/2016 02:01:04 - Info bptm(pid=8783) start,12/31/2016 02:01:04 - Info bptm(pid=8783) using 1048576 data buffer size,12/31/2016 02:01:04 - Info bptm(pid=8783) setting receive network buffer to 262144 bytes,12/31/2016 02:01:04 - Info bptm(pid=8783) using 32 data buffers,12/31/2016 02:01:04 - Info bptm(pid=8783) start backup,12/31/2016 02:01:06 - Info bptm(pid=8783) backup child process is pid 8800,12/31/2016 02:01:06 - begin writing,12/31/2016 02:01:16 - Info bpbkar(pid=4752) not using change journal data for <C:\\>: not enabled,12/31/2016 02:00:39 - started,Status: the requested operation was successfully completed (0),12/31/2016 05:58:20 - Info bptm(pid=8783) waited for full buffer 46602 times\, delayed 925862 times,12/31/2016 05:58:22 - Info bptm(pid=8783) EXITING with status 0 <----------,12/31/2016 05:58:22 - Info master(pid=8783) StorageServer=PureDisk:master; Report=PDDO Stats (multi-threaded stream used) for (master): scanned: 48364935 KB\, CR sent: 5970300 KB\, CR sent over FC: 0 KB\, dedup: 87.7%\, cache disabled,12/31/2016 05:58:22 - Info bpbrm(pid=8709) validating image for client dc03,12/31/2016 05:58:23 - Info bpbkar(pid=4752) done. status: 0: the requested operation was successfully completed,12/31/2016 05:58:23 - end writing; write time: 003:57:17,48344176,112317,515398,3403,,,,,,,,,,,0,,,1,0,0,dc03_1483138839,,,0,0, ,87.699997,
01-02-2017 07:00 AM
Ok - definitely image id: dc03_1483138839 is the one that we need to find.
# ./bpimagelist -backupid dc03_1483138839 -L
01-02-2017 07:06 AM
it gave this:
Client: dc03 Backup ID: dc03_1483138839 Policy: Yearly_DC Policy Type: MS-Windows (13) Proxy Client: (none specified) Creator: root Name1: (none specified) Sched Label: Yearly Schedule Type: FULL (0) Retention Level: infinity (9) Backup Time: Sat 31 Dec 2016 02:00:39 AM AS (1483138839) Elapsed Time: 14263 second(s) Expiration Time: INFINITY (2147483647) Maximum Expiration Time: INFINITY (2147483647) Compressed: no Client Encrypted: no Kilobytes: 48359477 Number of Files: 112317 Number of Copies: 1 Number of Fragments: 8 Histogram: 0 0 0 0 0 0 0 0 0 0 DB Compressed: no Files File Name: Yearly_DC_1483138839_FULL.f Previous Backup Files File Name: (none specified) Parent Backup Image File Name: (none specified) SW Version: (none specified) Options: 0x0 MPX: 0 TIR Info: 2 TIR Expiration: Sun 01 Jan 2017 02:00:39 AM AS (1483225239) Keyword: (none specified) Ext Security Info: no File Restore Raw: no Image Dump Level: 0 File System Only: no Object Descriptor: (none specified) Previous BI Time: Thu 01 Jan 1970 03:00:00 AM AS (0) BI Full Time: Thu 01 Jan 1970 03:00:00 AM AS (0) Request Pid: 0 Backup Status: 0 Stream Number: 0 Backup Copy: Standard (0) Files File size: 15669009 PFI type: 0 IMAGE_ATTRIBUTE: 0 Primary Copy: 1 Image Type: 0 (Regular) Job ID: 515406 Num Resumes: 0 Resume Expiration: Thu 01 Jan 1970 03:00:00 AM AS (0) Data Classification: Gold Data_Classification_ID: 9F5AC974364911E2857CAC40773EEAED Storage Lifecycle Policy: Yearly_Policy Storage Lifecycle Policy Version: 0 STL_Completed: 2 Remote Expiration Time: Thu 01 Jan 1970 03:00:00 AM AS (0) Origin Master Server: (none specified) Origin Master GUID: (none specified) Snap Time: Thu 01 Jan 1970 03:00:00 AM AS (0) IR Enabled: no Client Character Set: 1 Image On Hold: 0 Indexing Status: 0 Copy number: 1 Fragment: TIR (-1) Kilobytes: 15301 Remainder: 785 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x0 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 0 resume num: 0 Key tag: *NULL* STL tag: *NULL* Copy on hold: 0 Copy number: 1 Fragment: 1 Kilobytes: 17339392 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 Expiration Time: INFINITY (2147483647) MPX: 0 retention_lvl: infinity (9) Try to Keep Time: INFINITY (2147483647) Copy Creation Time: Sat 31 Dec 2016 05:58:22 AM AS (1483153102) Data Format: Tar checkpoint: 1 resume num: 0 Copy on hold: 0 Copy number: 1 Fragment: 2 Kilobytes: 3332096 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 1 resume num: 0 Copy on hold: 0 Copy number: 1 Fragment: 3 Kilobytes: 4844544 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 1 resume num: 0 Copy on hold: 0 Copy number: 1 Fragment: 4 Kilobytes: 2595840 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 1 resume num: 0 Copy on hold: 0 Copy number: 1 Fragment: 5 Kilobytes: 5484544 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 1 resume num: 0 Copy on hold: 0 Copy number: 1 Fragment: 6 Kilobytes: 8476672 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 1 resume num: 0 Copy on hold: 0 Copy number: 1 Fragment: 7 Kilobytes: 6271088 Remainder: 0 Media Type: Disk (0) Density: qscsi (0) File Num: 0 ID: @aaaan Host: master Block Size: 1048576 Offset: 0 Media Date: Thu 01 Jan 1970 03:00:00 AM AS (0) Dev Written On: -1 Flags: 0x4 Media Descriptor: 1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0 checkpoint: 0 resume num: 0 Copy on hold: 0
01-02-2017 07:15 AM - edited 01-02-2017 07:16 AM
Ok, the backup image is on a form of disk media ID: @aaaan, with one copy (so far) and is currently in "STL_Completed" state "2" which means "IN PROCESS", see:
DOCUMENTATION: What are the different fields in "bpimagelist -l" output?
http://www.veritas.com/docs/000032604
So, your SLP has not yet finished the duplication to tape.
01-02-2017 07:28 AM
so is that why the command
./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc03 -policy Yearly_DC -pt MS-Windows
is returning "no entity was found"? i see.
i will try and check after a few days then. thank you for your time.
01-02-2017 11:43 AM
Yes - I think so, because you are asking bpimagelist to list images on tape media when using the "-media" switch.
Also, I seem to remember some NetBackup CLI commands defaulting to using the current time when only a date is specified on a date+time option on a command... so... to avoid this, I would normally always specifiy a date+time and never just a date on its own, e.g. your example of:
./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc03 -policy Yearly_DC -pt MS-Windows
...I would actually specify this:
./bpimagelist -media -U -d 12/31/2016 00:00:00 -e 1/1/2017 23:59:59 -client dc03 -policy Yearly_DC -pt MS-Windows
...but this shouldn't affect the listing in this particular instance... and so the only reason I mention the time element of a date+time option is to perhaps steer you to always specifying date+time whenever you use any date(+time) options on NetBackup CLI commands.
.
Plus I seem to remember that plain regular calls to bpimagelist will not list "STL incomplete" images, because the backup images are not really available for administrators to manipulate until all of the SLP stages have ben completed - and so maybe, we only managed to list the backup image because we specified exactly the specific backup image ID using a "-backupid imagename"... but I would have to do some testing to confirm either way.
01-02-2017 06:40 PM
The images are still on your disk, and hence you see the output no entity found.
2 things you can try to identify if the images got duplicated to tape or not.
1) from SLP:
use command nbstlutil list to identify if all the images from your SLP are duplicated.
you can explore more within SLP w.r.t the SLP name etc using below commonds
nbstlutil list
nbstlutil list –lifecycle <name>
nbstlutil list –backupid <id_value>
nbstlutil list -image_state <value>
nbstlutil list -copy_state <value>
nbstlutil list -frag_state <value>
nbstlutil list –mediaid <media_id>_–state 3
more details on SLP can be found here
https://www.veritas.com/support/en_US/article.000109501
2) From catalog
on your activity monitor, go to catalog tab and search for backup images for client (dc03) with the date parameters, by default it list the images for primary copy, check the media id and copy number details from the output. if the media ids are something like @aaaan then these are your disk images and primary.
Now, on top right in the same window you should be able to change copy number and run a search again, seearch for diffrent copy then the one listed above (if copy 1 was primary per previous output then search copy 2).
this is the same serach results as you get in bpimagelist.
if this search result is empty too then you dont have to worry your images still needs to be duplciated to tape... sit tight and have cup of cofee :) .
If above search results shows the media id details your are looking for then you should also be able to see the same from bpimagelist.
and Happy new year to you too!!!!!
Cheers,
Venki
01-02-2017 10:37 PM
The fact that backups have not been duplicated 2 days after the backup, points to inadequate resources or lack of SLP tuning.
You really need to keep a close eye on SLP queues....
01-07-2017 11:52 PM
i only have 2 tape drives being used for both duplication and restores (when needed). last november a botched up Oracle restore took several days and while DBA trying to restore, all my tape drives where being used by him. SLPs were suspended during that time which contributed to the delays.
is there any TN or services we can avail to study and come up with what should be our configuration now?
01-08-2017 05:40 AM
All I can add, is that it is one thing to scope and size an environment for backup, and an almost entirely separate task to scope and size an environment for recovery and restore. They would seen to be the same thing, but in reality they are two fairly separate (yet related) considerations.
A lot of sites focus on backup size/speed/time to complete backups within an SLA run window, but unfortunately some sites completely miss the fact that they may also need to also focus on RPO and in your case RTO. Some sites don't need to focus on RPO/RTO because there is no requirement to.
If you have no business specified RPO/RTO and no business ratified RPO/RTO then you don't have really have a problem because there are no clear hard and fast objectives. What you actually have is a business viability problem because no-one ever scoped in any wide ranging restore recovery requirements.
It would seem to me that your quick solution is to add one or more additional tape drives.
There is no magic solution to scoping and sizing for complete DR/BCP within any meaningful time frame, without first acquiring a budget and then puting in the prior hard work to model / plan / experiment / iterate / estimate / design (probably using lots of progessively more and more complicated modelling in Excel) for full complete site wide scale DR/BCP recovery from backup.