cancel
Showing results for 
Search instead for 
Did you mean: 

Desperately Seeking Susan.. or the yearly media

manatee
Level 6

Happy New Year!!

pardon the subject can't help it ;)  i have several year end backup policies that have finished and am looking for how to find all the media it uses. i tried the catalog but i can't change the policy type to SQLServer. it is greyed out at MS-Windows.

how to find all the media used by a particular backup policy?

20 REPLIES 20

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
bpimagelist is once again your friend.

bpimagelist -media -U [-d date] [-e date] [-client client_name] [-policy policy_name] [-pt policy_type]

i keep getting "no entity was found" so i tried different variations of the command:

[root@master admincmd]# ./bpimagelist -media -U -policy Yearly_DC
no entity was found
[root@master admincmd]# ./bpimagelist -media -U -policy Yearly_DC -pt MS-Windows
no entity was found
[root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc01 -policy Yearly_DC -pt MS-Windows
no entity was found
[root@master admincmd]# ./bpimagelist -media -U -tape -d 12/31/2016 -e 1/1/2017 -client dc01 -policy Yearly_DC -pt MS-Windows
no entity was found
[root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 16:00:00 -e 1/1/2017 23:59:00 -client dc01 -policy Yearly_DC -pt MS-Windows
no entity was found
[root@master admincmd]# ./bpimagelist -media -U
no entity was found
[root@master admincmd]# ./bpimagelist -U -stl_incomplete -policy Yearly_DC
no entity was found
[root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 16:00:00 -e 01/01/2017 23:59:59 -client dc01 -policy Yearly_DC -pt MS-Windows
no entity was found
[root@master admincmd]# ./bpimagelist -media -U -d 12/31/2016 00:00:00 -e 01/01/2017 23:59:59 -client dc01 -policy Yearly_DC -pt MS-Windows
no entity was found

and still same errors. i double checked in the activity monitor regarding that backup policy and there is a backup done on the day of Dec 31st 2016.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Can you show us what you see in Activity Monitor?
Screenshot or .txt output.

i hope i got it right.

yearly_DC.png

 

you can see the backup finished successfully.

 

 

sdo
Moderator
Moderator
Partner    VIP    Certified

Can you determine which retention level was used?   Maybe the backup has already expired?

i used SLP and it is set to expire after copy on disk, then infinity on tape.

sdo
Moderator
Moderator
Partner    VIP    Certified

dc02 and dc03 are listed in activity monitor, and dc01 is not.  Looks like a backup of dc01 did not run.

What does this show:

# ./bperror -all -d 12/31/2016 00:00:00 -e 01/02/2017 23:59:59 -client dc01 -U

i'm sorry that was a typo. should be dc03.

[root@ovmmanager admincmd]# ./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc03 -policy Yearly_DC -pt MS-Windows
no entity was found

the command you provided gave a long list (shortened) below:

    TIME            SERVER/CLIENT                      TEXT
12/31/2016 02:00:00 master dc03  started backup job for client
                    dc03, policy Monthly_Windows_Domain_Controller,
                    schedule Monthly on storage unit master-pd-1
12/31/2016 02:00:03 master dc03  started backup job for client
                    dc03, policy Yearly_DC, schedule Yearly on storage
                    unit master-pd-1
12/31/2016 02:00:38 master dc03  started backup job for client
                    dc03, policy Monthly_Windows_Domain_Controller,
                    schedule Monthly on storage unit master-pd-1
12/31/2016 02:00:39 master dc03  started backup job for client
                    dc03, policy Yearly_DC, schedule Yearly on storage
                    unit master-pd-1
12/31/2016 02:01:06 master dc03  begin writing backup id
                    dc03_1483138838, copy 1, fragment 1, destination path
                    PureDiskVolume
12/31/2016 02:01:06 master dc03  begin writing backup id
                    dc03_1483138839, copy 1, fragment 1, destination path
                    PureDiskVolume
12/31/2016 04:19:45 master dc03  from client dc03: WRN - can't open
                    object: System State:\Automated System Recovery\BCD\BCD
                    (WIN32 3758134305: Unknown error)
12/31/2016 04:19:45 master dc03  from client dc03: ERR - Error
                    encountered while attempting to get additional files for
                    System State:\
12/31/2016 04:33:30 master dc03  successfully wrote backup id
                    dc03_1483138839, copy 1, fragment 1, 17339392 Kbytes
                    at 1896.385 Kbytes/sec
12/31/2016 04:33:30 master dc03  begin writing backup id
                    dc03_1483138839, copy 1, fragment 2, destination path
                    PureDiskVolume
12/31/2016 04:33:30 master dc03  successfully wrote backup id
                    dc03_1483138838, copy 1, fragment 1, 17326080 Kbytes
                    at 1894.924 Kbytes/sec
12/31/2016 04:33:30 master dc03  begin writing backup id
                    dc03_1483138838, copy 1, fragment 2, destination path
                    PureDiskVolume
12/31/2016 04:46:07 master dc03  successfully wrote backup id
                    dc03_1483138839, copy 1, fragment 2, 3332096 Kbytes
                    at 4400.916 Kbytes/sec
12/31/2016 04:46:07 master dc03  begin writing backup id
                    dc03_1483138839, copy 1, fragment 3, destination path

...


01/01/2017 21:17:14 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:17:45 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:18:16 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:18:47 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:19:18 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:19:33 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:20:07 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:20:29 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:21:00 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:21:31 master dc03  image copy is not ready, retry
                    attempt: 176 of 500 object is busy, cannot be closed
01/01/2017 21:21:39 master dc03  StorageServer=PureDisk:master;
                    Report=PDDO Stats (multi-threaded stream used) for
                    (master): scanned: 17343827 KB, CR sent: 75157 KB, CR
                    sent over FC: 0 KB, dedup: 99.6%, cache disabled
01/01/2017 21:21:40 master dc03  Using the media server to write NBU
                    data for backup dc03_1483282848 to master
01/01/2017 21:21:41 master dc03  StorageServer=PureDisk:master;
                    Report=PDDO Stats for (master): scanned: 2 KB, CR
                    sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%, cache
                    disabled
01/01/2017 21:21:51 master dc03  CLIENT dc03  POLICY Daily_DC
                    SCHED Daily  EXIT STATUS 0 (the requested operation was
                    successfully completed)
01/01/2017 21:21:57 master dc03  from client dc03: cannot open
                    C:\Program
                    Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.ruh1
                    dc03_1483282800.1.0
01/01/2017 21:21:58 master dc03  CLIENT dc03  POLICY Daily_DC
                    SCHED Daily  EXIT STATUS 0 (the requested operation was
                    successfully completed)


sdo
Moderator
Moderator
Partner    VIP    Certified

From the bperror log we see two backups start, one for each policy:

Monthly_Windows_Domain_Controller

Yearly_DC

...and we see two image IDs for:

dc03_1483138838

dc03_1483138839

...but from bperror we cannot determine which image ID is for which policy.

Later on we see this image end ok:  dc03_1483282848

...but it looks to be for policy:  Daily_DC

Seems like you have (had) client dc03 in at least three policies.  We're not going to be able to determine an image name from the above.

.

Let's try this:

# ./bpdbjobs -jobid 515406 -all_columns

it gave a long list:

[root@master admincmd]# ./bpdbjobs -jobid 515406 -all_columns
515406,0,3,0,Yearly_DC,Yearly,dc03,master,1483138839,0000014265,1483153104,master-pd-1,1,,48344176,112317,,100,8709,root,1,13,0,0,root,master,-1,-1,0,,,3,ALL_LOCAL_DRIVES,System State:\\,,1,8709,master-pd-1,master,1483138839,0000014266,1483153105,0,the requested operation was successfully completed,29,12/31/2016 02:00:39 - Info nbjm(pid=32742) starting backup job (jobid=515406) for client dc03\, policy Yearly_DC\, schedule Yearly,12/31/2016 02:00:39 - estimated 42741159 kbytes needed,12/31/2016 02:00:39 - Info nbjm(pid=32742) started backup (backupid=dc03_1483138839) job for client dc03\, policy Yearly_DC\, schedule Yearly on storage unit master-pd-1,12/31/2016 02:00:40 - started process bpbrm (8709),12/31/2016 02:00:40 - Info bpbrm(pid=8709) dc03 is the host to backup data from,12/31/2016 02:00:40 - Info bpbrm(pid=8709) reading file list for client,12/31/2016 02:00:41 - Info bpbrm(pid=8709) starting bpbkar on client,12/31/2016 02:00:40 - connecting,12/31/2016 02:00:41 - connected; connect time: 000:00:01,12/31/2016 02:01:04 - Info bpbkar(pid=4752) Backup started,12/31/2016 02:01:04 - Info bpbrm(pid=8709) bptm pid: 8783,12/31/2016 02:01:04 - Info bpbkar(pid=4752) change time comparison:<disabled>,12/31/2016 02:01:04 - Info bpbkar(pid=4752) archive bit processing:<enabled>,12/31/2016 02:01:04 - Info bptm(pid=8783) start,12/31/2016 02:01:04 - Info bptm(pid=8783) using 1048576 data buffer size,12/31/2016 02:01:04 - Info bptm(pid=8783) setting receive network buffer to 262144 bytes,12/31/2016 02:01:04 - Info bptm(pid=8783) using 32 data buffers,12/31/2016 02:01:04 - Info bptm(pid=8783) start backup,12/31/2016 02:01:06 - Info bptm(pid=8783) backup child process is pid 8800,12/31/2016 02:01:06 - begin writing,12/31/2016 02:01:16 - Info bpbkar(pid=4752) not using change journal data for <C:\\>: not enabled,12/31/2016 02:00:39 - started,Status: the requested operation was successfully completed (0),12/31/2016 05:58:20 - Info bptm(pid=8783) waited for full buffer 46602 times\, delayed 925862 times,12/31/2016 05:58:22 - Info bptm(pid=8783) EXITING with status 0 <----------,12/31/2016 05:58:22 - Info master(pid=8783) StorageServer=PureDisk:master; Report=PDDO Stats (multi-threaded stream used) for (master): scanned: 48364935 KB\, CR sent: 5970300 KB\, CR sent over FC: 0 KB\, dedup: 87.7%\, cache disabled,12/31/2016 05:58:22 - Info bpbrm(pid=8709) validating image for client dc03,12/31/2016 05:58:23 - Info bpbkar(pid=4752) done. status: 0: the requested operation was successfully completed,12/31/2016 05:58:23 - end writing; write time: 003:57:17,48344176,112317,515398,3403,,,,,,,,,,,0,,,1,0,0,dc03_1483138839,,,0,0, ,87.699997,

sdo
Moderator
Moderator
Partner    VIP    Certified

Ok - definitely image id:   dc03_1483138839   is the one that we need to find.

# ./bpimagelist -backupid dc03_1483138839 -L

 

it gave this:

Client:            dc03
Backup ID:         dc03_1483138839
Policy:            Yearly_DC
Policy Type:       MS-Windows (13)
Proxy Client:      (none specified)
Creator:           root
Name1:             (none specified)
Sched Label:       Yearly
Schedule Type:     FULL (0)
Retention Level:   infinity (9)
Backup Time:       Sat 31 Dec 2016 02:00:39 AM AS (1483138839)
Elapsed Time:      14263 second(s)
Expiration Time:             INFINITY       (2147483647)
Maximum Expiration Time:             INFINITY       (2147483647)
Compressed:        no
Client Encrypted:  no
Kilobytes:         48359477
Number of Files:   112317
Number of Copies:  1
Number of Fragments:   8
Histogram:         0 0 0 0 0 0 0 0 0 0
DB Compressed:     no
Files File Name:   Yearly_DC_1483138839_FULL.f
Previous Backup Files File Name:   (none specified)
Parent Backup Image File Name:   (none specified)
SW Version:        (none specified)
Options:           0x0
MPX:               0
TIR Info:          2
TIR Expiration:    Sun 01 Jan 2017 02:00:39 AM AS (1483225239)
Keyword:           (none specified)
Ext Security Info: no
File Restore Raw:  no
Image Dump Level:  0
File System Only:  no
Object Descriptor: (none specified)
Previous BI Time:  Thu 01 Jan 1970 03:00:00 AM AS (0)
BI Full Time:      Thu 01 Jan 1970 03:00:00 AM AS (0)
Request Pid:       0
Backup Status:     0
Stream Number:     0
Backup Copy:       Standard (0)
Files File size:     15669009
PFI type:     0
IMAGE_ATTRIBUTE:     0
Primary Copy:      1
Image Type:        0  (Regular)
Job ID:            515406
Num Resumes:       0
Resume Expiration: Thu 01 Jan 1970 03:00:00 AM AS (0)
Data Classification:    Gold
Data_Classification_ID: 9F5AC974364911E2857CAC40773EEAED
Storage Lifecycle Policy:    Yearly_Policy
Storage Lifecycle Policy Version:    0
STL_Completed:      2
Remote Expiration Time: Thu 01 Jan 1970 03:00:00 AM AS (0)
Origin Master Server:  (none specified)
Origin Master GUID:    (none specified)
Snap Time:      Thu 01 Jan 1970 03:00:00 AM AS (0)
IR Enabled:      no
Client Character Set:     1
Image On Hold:     0
Indexing Status:   0
Copy number:       1
 Fragment:         TIR (-1)
 Kilobytes:        15301
 Remainder:        785
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x0
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       0
 resume num:       0
 Key tag:          *NULL*
 STL tag:          *NULL*
 Copy on hold:     0
Copy number:       1
 Fragment:         1
 Kilobytes:        17339392
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 Expiration Time:            INFINITY       (2147483647)
 MPX:              0
 retention_lvl:    infinity (9)
 Try to Keep Time:            INFINITY       (2147483647)
 Copy Creation Time:  Sat 31 Dec 2016 05:58:22 AM AS (1483153102)
 Data Format:      Tar
 checkpoint:       1
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         2
 Kilobytes:        3332096
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       1
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         3
 Kilobytes:        4844544
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       1
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         4
 Kilobytes:        2595840
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       1
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         5
 Kilobytes:        5484544
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       1
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         6
 Kilobytes:        8476672
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       1
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         7
 Kilobytes:        6271088
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaan
 Host:             master
 Block Size:       1048576
 Offset:           0
 Media Date:       Thu 01 Jan 1970 03:00:00 AM AS (0)
 Dev Written On:   -1
 Flags:            0x4
 Media Descriptor:        1;PureDisk;master;NBU-PureDisk-1;PureDiskVolume;0
 checkpoint:       0
 resume num:       0
 Copy on hold:     0

sdo
Moderator
Moderator
Partner    VIP    Certified

Ok, the backup image is on a form of disk media ID: @aaaan, with one copy (so far) and is currently in "STL_Completed" state "2" which means "IN PROCESS", see:

DOCUMENTATION: What are the different fields in "bpimagelist -l" output?
http://www.veritas.com/docs/000032604

So, your SLP has not yet finished the duplication to tape.

so is that why the command

./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc03 -policy Yearly_DC -pt MS-Windows

is returning "no entity was found"? i see.

i will try and check after a few days then. thank you for your time.

sdo
Moderator
Moderator
Partner    VIP    Certified

Yes - I think so, because you are asking bpimagelist to list images on tape media when using the "-media" switch.

Also, I seem to remember some NetBackup CLI commands defaulting to using the current time when only a date is specified on a date+time option on a command... so... to avoid this, I would normally always specifiy a date+time and never just a date on its own, e.g. your example of:

./bpimagelist -media -U -d 12/31/2016 -e 1/1/2017 -client dc03 -policy Yearly_DC -pt MS-Windows

...I would actually specify this:

./bpimagelist -media -U -d 12/31/2016 00:00:00 -e 1/1/2017 23:59:59 -client dc03 -policy Yearly_DC -pt MS-Windows

...but this shouldn't affect the listing in this particular instance... and so the only reason I mention the time element of a date+time option is to perhaps steer you to always specifying date+time whenever you use any date(+time) options on NetBackup CLI commands.

.

Plus I seem to remember that plain regular calls to bpimagelist will not list "STL incomplete" images, because the backup images are not really available for administrators to manipulate until all of the SLP stages have ben completed - and so maybe, we only managed to list the backup image because we specified exactly the specific backup image ID using a "-backupid imagename"... but I would have to do some testing to confirm either way.

The images are still on your disk, and hence you see the output no entity found.

2 things you can try to identify if the images got duplicated to tape or not.

1) from SLP:

use command nbstlutil list  to identify if all the images from your SLP are duplicated.

you can explore more within SLP w.r.t the SLP name etc using below commonds

nbstlutil list
nbstlutil list –lifecycle <name>
nbstlutil list –backupid <id_value>
nbstlutil list -image_state <value>
nbstlutil list -copy_state <value>
nbstlutil list -frag_state <value>
nbstlutil list –mediaid <media_id>_–state 3

more details on SLP can be found here

https://www.veritas.com/support/en_US/article.000109501

2) From catalog

on your activity monitor, go to catalog tab and search for backup images for client (dc03) with the date parameters, by default it list the images for primary copy, check the media id and copy number details from the output. if the media ids are something like @aaaan then these are your disk images and primary.

Now, on top right in the same window you should be able to change copy number and run a search again, seearch for diffrent copy then the one listed above (if copy 1 was primary per previous output then search copy 2).

this is the same serach results as you get in bpimagelist.

if this search result is empty too then you dont have to worry your images still needs to be duplciated to tape... sit tight and have cup of cofee :) .

If above search results shows the media id details your are looking for then you should also be able to see the same from bpimagelist.

and Happy new year to you too!!!!!

Cheers,

Venki

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The fact that backups have not been duplicated 2 days after the backup, points to inadequate resources or lack of SLP tuning.

You really need to keep a close eye on SLP queues....

 

i only have 2 tape drives being used for both duplication and restores (when needed). last november a botched up Oracle restore took several days and while DBA trying to restore, all my tape drives where being used by him. SLPs were suspended during that time which contributed to the delays.

is there any TN or services we can avail to study and come up with what should be our configuration now?

sdo
Moderator
Moderator
Partner    VIP    Certified

All I can add, is that it is one thing to scope and size an environment for backup, and an almost entirely separate task to scope and size an environment for recovery and restore.  They would seen to be the same thing, but in reality they are two fairly separate (yet related) considerations.

A lot of sites focus on backup size/speed/time to complete backups within an SLA run window, but unfortunately some sites completely miss the fact that they may also need to also focus on RPO and in your case RTO.  Some sites don't need to focus on RPO/RTO because there is no requirement to.

If you have no business specified RPO/RTO and no business ratified RPO/RTO then you don't have really have a problem because there are no clear hard and fast objectives.  What you actually have is a business viability problem because no-one ever scoped in any wide ranging restore recovery requirements.

It would seem to me that your quick solution is to add one or more additional tape drives.

There is no magic solution to scoping and sizing for complete DR/BCP within any meaningful time frame, without first acquiring a budget and then puting in the prior hard work to model / plan / experiment / iterate / estimate / design (probably using lots of progessively more and more complicated modelling in Excel) for full complete site wide scale DR/BCP recovery from backup.