08-14-2014 05:26 AM
All,
Our Vault Duplication Job completed successfully to tape. We run month end test restores for several servers, using a touch file to force NB to use copy 3 and use tapes to run the restores.
So far every restore has been successful except for 1 server. (LNCVIM)
error VMware policy restore error (2820)
with an error in policy of NBU status: 95, EMM status: Media does not exist
i checked the Vault logs and the LNCVIM duplicated successfully.
If you run a restore of of the primary copy - ie our datadomain - the job restores successfully - selecting 25th July as our month end date.
If you do the same from tape - copy 3 - it failes. Some exctra note is it is a VmWare Policy type we are using.
I have tested selecting one files, folders and the whole D drive and it still fails.
I kicked off another duplication and verifyed the duplication in the catalog as successful - it obviously then wrote the data to a different tape and this again is failing to restore with the above same error message.
Can anybody advise if they have seen this before, or if they require any more information to assist in resolving this?
Many thanks
Solved! Go to Solution.
08-14-2014 10:26 AM
Restore failed due to failure to read the header from the backup image(2841)
The tape is encrypted, is the server (and drive) doing the restore abel to read the encrypted media?
08-14-2014 05:49 AM
To test, make the tape primary copy and see if it works. Can then work back from there to figure out what is wrong.
08-14-2014 05:53 AM
NBU status: 95, EMM status: Media does not exist
please post detail status of the failed restore job
also below command output for the media that is trying to use for restore..
nbemmcmd -listmedia -mediaid <string>
08-14-2014 07:09 AM
I would like to see if NBU sees 3 valid copies for this image.
Please show image output for this client:
bpimagelist -client LNCVIM -d 07/25/2014 -e 07/26/2014 -L
08-14-2014 08:02 AM
Nagalla, please find the details you requested below:
Command output:
C:\Windows\system32>nbemmcmd -listmedia -mediaid 000034
NBEMMCMD, Version: 7.5.0.5
====================================================================
Media GUID: d002e732-1ab8-4eac-acd2-3d56903a2ce2
Media ID: 000034
Partner: -
Media Type: HCART2
Volume Group: 000_00001_TLD
Application: Netbackup
Media Flags: 1
Description: Scratch Media Label
Barcode: 000034
Partner Barcode: --------
Last Write Host: ln1nb01.domain.com
Created: 12/14/2012 11:02
Time Assigned: 08/14/2014 11:09
First Mount: 12/31/2012 03:50
Last Mount: 08/14/2014 11:10
Volume Expiration: -
Data Expiration: 01/27/2015 22:36
Last Written: 08/14/2014 11:09
Last Read: -
Robot Type: TLD
Robot Control Host: ln1nb01.domain.com
Robot Number: 1
Slot: 4
Side/Face: -
Cleanings Remaining: -
Number of Mounts: 2
Maximum Mounts Allowed: 0
Media Status:
Kilobytes: 14164174
Images: 3
Valid Images: 3
Retention Period: 6
Number of Restores: 0
Optical Header Size Bytes: 1024
Optical Sector Size Bytes: 0
Optical Partition Size Bytes: 0
Last Header Offset: 55337
Adamm Guid: 00000000-0000-0000-0000-000000000000
Rsm Guid: 00000000-0000-0000-0000-000000000000
Origin Host: NONE
Master Host: lncvnbmgt01.domain.com
Server Group: NO_SHARING_GROUP
Upgrade Conflicts Flag:
Pool Number: 7
Volume Pool: ENCR_Monthly_Vault
Previous Pool Name: Scratch
Vault Flags: -
Vault Container: -
Vault Name: -
Vault Slot: -
Session ID: -
Date Vaulted: -
Return Date: -
Media on Hold: 0
====================================================================
Command completed successfully.
Error code details
(Attempt 1)
14/08/2014 13:07:06 - begin Restore
14/08/2014 13:07:08 - media 000034 required
14/08/2014 13:07:08 - restoring image lncvimpr01_1406327802
14/08/2014 13:07:08 - Info bprd(pid=1480) Restoring from copy 4 of image created 07/25/14 23:36:42
14/08/2014 13:07:12 - Info bptm(pid=6576) start
14/08/2014 13:07:12 - started process bptm (6576)
14/08/2014 13:07:12 - Info bpdm(pid=6576) reading backup image
14/08/2014 13:07:12 - Info bptm(pid=6576) using 64 data buffers
14/08/2014 13:07:12 - Info bptm(pid=6576) spawning a child process
14/08/2014 13:07:12 - Info bptm(pid=6576) child pid: 5264
14/08/2014 13:07:12 - Info bptm(pid=5264) start
14/08/2014 13:07:12 - started process bptm (5264)
14/08/2014 13:07:14 - end Restore; elapsed time: 00:00:08
Restore failed due to failure to read the header from the backup image(2841)
(Attempt 2)
14/08/2014 13:07:14 - begin Restore
14/08/2014 13:07:14 - requesting resource @AAAAD
14/08/2014 13:07:14 - Error nbjm(pid=8696) NBU status: 95, EMM status: Media does not exist
14/08/2014 13:07:14 - Error nbjm(pid=8696) NBU status: 95, EMM status: Media does not exist
14/08/2014 13:07:15 - Info bprd(pid=1480) Retrying restore from copy 4 of image created 07/25/14 23:36:42
14/08/2014 13:07:15 - Info bptm(pid=6576) EXITING with status 95 <----------
14/08/2014 13:07:18 - Info bptm(pid=8608) start
14/08/2014 13:07:18 - started process bptm (8608)
14/08/2014 13:07:18 - Info bpdm(pid=8608) reading backup image
14/08/2014 13:07:18 - Info bptm(pid=8608) using 64 data buffers
14/08/2014 13:07:18 - Info bptm(pid=8608) spawning a child process
14/08/2014 13:07:18 - Info bptm(pid=8608) child pid: 9212
14/08/2014 13:07:19 - Info bptm(pid=9212) start
14/08/2014 13:07:19 - started process bptm (9212)
14/08/2014 13:07:21 - Info bptm(pid=8608) EXITING with status 95 <----------
14/08/2014 13:07:21 - end Restore; elapsed time: 00:00:07
VMware policy restore error(2820)
14/08/2014 13:07:21 - requesting resource @AAAAD
14/08/2014 13:07:21 - Error nbjm(pid=8696) NBU status: 95, EMM status: Media does not exist
14/08/2014 13:07:21 - Error nbjm(pid=8696) NBU status: 95, EMM status: Media does not exist
08-14-2014 08:04 AM
C:\Windows\system32>bpimagelist -client LNCVIMPR01 -d 07/25/2014 -e 07/26/2014 -L
Client: lncvimpr01
Backup ID: lncvimpr01_1406327802
Policy: LNC_VMWare
Policy Type: VMware (40)
Proxy Client: (none specified)
Creator: root
Name1: (none specified)
Sched Label: Monthly_Backup
Schedule Type: FULL (0)
Retention Level: 6 months (6)
Backup Time: 25/07/2014 23:36:42 (1406327802)
Elapsed Time: 590 second(s)
Expiration Time: 27/01/2015 22:36:42 (1422398202)
Compressed: no
Client Encrypted: no
Kilobytes: 14164173
Number of Files: 62994
Number of Copies: 4
Number of Fragments: 10
Histogram: 0 0 0 0 0 0 0 0 0 0
DB Compressed: no
Files File Name: LNC_VMWare_1406327802_FULL.f
Previous Backup Files File Name: (none specified)
Parent Backup Image File Name: (none specified)
SW Version: (none specified)
Options: 0x0
MPX: 0
TIR Info: 0
TIR Expiration: 01/01/1970 00:00:00 (0)
Keyword: (none specified)
Ext Security Info: no
File Restore Raw: yes
Image Dump Level: 0
File System Only: no
Object Descriptor: (none specified)
Previous BI Time: 25/07/2014 23:36:42 (1406327802)
BI Full Time: 25/07/2014 23:36:42 (1406327802)
Request Pid: 0
Backup Status: 0
Stream Number: 105
Backup Copy: Standard (0)
Files File size: 3036026
PFI type: 0
IMAGE_ATTRIBUTE: 0
Primary Copy: 1
Image Type: 0 (Regular)
Job ID: 1552374
Num Resumes: 3
Resume Expiration: 01/01/1970 00:00:00 (0)
Data Classification: (none specified)
Data_Classification_ID: (none specified)
Storage Lifecycle Policy: Monthly_Backup_LNC_LN1
Storage Lifecycle Policy Version: 6
STL_Completed: 3
Remote Expiration Time: 01/01/1970 00:00:00 (0)
Origin Master Server: (none specified)
Origin Master GUID: (none specified)
Snap Time: 25/07/2014 23:35:40 (1406327740)
IR Enabled: no
Client Character Set: 0
Image On Hold: 0
Indexing Status: 0
Copy number: 1
Fragment: TIR (-1)
Kilobytes: 7637
Remainder: 455
Media Type: Disk (0)
Density: qscsi (0)
File Num: 0
ID: @aaaad
Host: lncnb01.domain.com
Block Size: 262144
Offset: 0
Media Date: 01/01/1970 00:00:00 (0)
Dev Written On: -1
Flags: 0x0
Media Descriptor: 1;DataDomain;lncemcdd01.domain.com;lncemcdd01-dp;lncemcdd01-lsu;0
checkpoint: 0
resume num: 0
Key tag: *NULL*
STL tag: *NULL*
Copy on hold: 0
Copy number: 1
Fragment: 1
Kilobytes: 14156536
Remainder: 0
Media Type: Disk (0)
Density: qscsi (0)
File Num: 0
ID: @aaaad
Host: lncnb01.domain.com
Block Size: 262144
Offset: 0
Media Date: 01/01/1970 00:00:00 (0)
Dev Written On: -1
Flags: 0x0
Media Descriptor: 1;DataDomain;lncemcdd01.domain.com;lncemcdd01-dp;lncemcdd01-lsu;0
Expiration Time: 27/01/2015 22:36:42 (1422398202)
MPX: 0
retention_lvl: 6 months (6)
Try to Keep Time: 27/01/2015 22:36:42 (1422398202)
Copy Creation Time: 25/07/2014 23:46:32 (1406328392)
Data Format: Tar
checkpoint: 0
resume num: 0
Copy on hold: 0
Copy number: 2
Fragment: TIR (-1)
Kilobytes: 7637
Remainder: 455
Media Type: Disk (0)
Density: qscsi (0)
File Num: 0
ID: @aaaab
Host: ln1nb02.domain.com
Block Size: 262144
Offset: 0
Media Date: 01/01/1970 00:00:00 (0)
Dev Written On: -1
Flags: 0x0
Media Descriptor: 1;DataDomain;ln1emcdd01.domain.com;ln1emcdd01-dp;ln1emcdd01-lsu;0
checkpoint: 0
resume num: 1
Key tag: *NULL*
STL tag: *NULL*
Copy on hold: 0
Copy number: 2
Fragment: 1
Kilobytes: 14156536
Remainder: 0
Media Type: Disk (0)
Density: qscsi (0)
File Num: 0
ID: @aaaab
Host: ln1nb02.domain.com
Block Size: 262144
Offset: 0
Media Date: 01/01/1970 00:00:00 (0)
Dev Written On: -1
Flags: 0x0
Media Descriptor: 1;DataDomain;ln1emcdd01.domain.com;ln1emcdd01-dp;ln1emcdd01-lsu;0
Expiration Time: 27/01/2015 22:36:42 (1422398202)
MPX: 0
retention_lvl: 6 months (6)
Try to Keep Time: 27/01/2015 22:36:42 (1422398202)
Copy Creation Time: 25/07/2014 23:53:57 (1406328837)
Data Format: Tar
checkpoint: 0
resume num: 1
Copy on hold: 0
Copy number: 3
Fragment: TIR (-1)
Kilobytes: 7638
Remainder: 0
Media Type: Media Manager (2)
Density: hcart2 (14)
File Num: 292
ID: 000450
Host: ln1nb01.domain.com
Block Size: 262144
Offset: 1446184
Media Date: 27/07/2014 04:01:14 (1406430074)
Dev Written On: 2
Flags: 0x40 (Tape Encrypted)
Media Descriptor: ?
checkpoint: 0
resume num: 0
Key tag: 2a2e3da16e31eaa8231b2f52d4e905ed8e6279252f57b029d27802e07edb8cf8
STL tag: *NULL*
Copy on hold: 0
Copy number: 3
Fragment: 1
Kilobytes: 10240000
Remainder: 0
Media Type: Media Manager (2)
Density: hcart2 (14)
File Num: 290
ID: 000450
Host: ln1nb01.domain.com
Block Size: 262144
Offset: 1390881
Media Date: 27/07/2014 04:01:14 (1406430074)
Dev Written On: 2
Flags: 0x40 (Tape Encrypted)
Media Descriptor: ?
Expiration Time: 22/07/2024 23:36:42 (1721687802)
MPX: 0
retention_lvl: 10 years (10)
Try to Keep Time: 01/01/1970 00:00:00 (0)
Copy Creation Time: 27/07/2014 06:52:16 (1406440336)
Data Format: Tar
checkpoint: 0
resume num: 0
Copy on hold: 0
Copy number: 3
Fragment: 2
Kilobytes: 3916536
Remainder: 0
Media Type: Media Manager (2)
Density: hcart2 (14)
File Num: 291
ID: 000450
Host: ln1nb01.domain.com
Block Size: 262144
Offset: 1430883
Media Date: 27/07/2014 04:01:14 (1406430074)
Dev Written On: 2
Flags: 0x40 (Tape Encrypted)
Media Descriptor: ?
checkpoint: 0
resume num: 0
Copy on hold: 0
Copy number: 4
Fragment: TIR (-1)
Kilobytes: 7638
Remainder: 0
Media Type: Media Manager (2)
Density: hcart2 (14)
File Num: 3
ID: 000034
Host: ln1nb01.domain.com
Block Size: 262144
Offset: 55305
Media Date: 14/08/2014 11:09:31 (1408010971)
Dev Written On: 0
Flags: 0x40 (Tape Encrypted)
Media Descriptor: ?
checkpoint: 0
resume num: 0
Key tag: 2a2e3da16e31eaa8231b2f52d4e905ed8e6279252f57b029d27802e07edb8cf8
STL tag: *NULL*
Copy on hold: 0
Copy number: 4
Fragment: 1
Kilobytes: 10240000
Remainder: 0
Media Type: Media Manager (2)
Density: hcart2 (14)
File Num: 1
ID: 000034
Host: ln1nb01.domain.com
Block Size: 262144
Offset: 2
Media Date: 14/08/2014 11:09:31 (1408010971)
Dev Written On: 0
Flags: 0x40 (Tape Encrypted)
Media Descriptor: ?
Expiration Time: 27/01/2015 22:36:42 (1422398202)
MPX: 0
retention_lvl: 6 months (6)
Try to Keep Time: 01/01/1970 00:00:00 (0)
Copy Creation Time: 14/08/2014 11:14:16 (1408011256)
Data Format: Tar
checkpoint: 0
resume num: 0
Copy on hold: 0
Copy number: 4
Fragment: 2
Kilobytes: 3916536
Remainder: 0
Media Type: Media Manager (2)
Density: hcart2 (14)
File Num: 2
ID: 000034
Host: ln1nb01.domain.com
Block Size: 262144
Offset: 40004
Media Date: 14/08/2014 11:09:31 (1408010971)
Dev Written On: 0
Flags: 0x40 (Tape Encrypted)
Media Descriptor: ?
checkpoint: 0
resume num: 0
Copy on hold: 0
08-14-2014 09:55 AM
Marianne,
Just to clarify on the above output. There are 4 copies, because when we could not restore from tape 000450, we then ran another duplication job that ran successfully.
We then changed the touch file to use Copy 4 to try the restore and even though you can see it runs from copy 4 and tries to use the new tape 000034, it fails with the exact same error.
08-14-2014 10:26 AM
Restore failed due to failure to read the header from the backup image(2841)
The tape is encrypted, is the server (and drive) doing the restore abel to read the encrypted media?
08-14-2014 01:31 PM
I don't understand...
Attempt 1
identifies copy 4 location as media 000034 and fails with this error:
Restore failed due to failure to read the header from the backup image(2841)
Attempt 2
identifies copy 4 as Disk (@AAAAD) that apparently no longer exists?
14/08/2014 13:07:14 - requesting resource @AAAAD
14/08/2014 13:07:14 - Error nbjm(pid=8696) NBU status: 95, EMM status: Media does not exist
I also battle to understand how 2 different attempts have exactly the same bprd and bptm PIDs?
Attempt 1
14/08/2014 13:07:08 - Info bprd(pid=1480) Restoring from copy 4 of image created 07/25/14 23:36:42
14/08/2014 13:07:12 - Info bptm(pid=6576) start
Attempt 2
4/08/2014 13:07:15 - Info bprd(pid=1480) Retrying restore from copy 4 of image created 07/25/14 23:36:42
14/08/2014 13:07:15 - Info bptm(pid=6576) EXITING with status 95 <----------
08-14-2014 05:01 PM
The 1st attempt seems to read from copy 4 (mediaID 000034), but it fails with error 2841.
Probably 2nd attempt kicks off but this time it didn't try from tape 000034 but the disk @aaaad (copy 1).
I would suggest you assign copy4 to be primary copy and retry, rather than creating the touch file (was it ALT_RESTORE_COPY_NUMBER ?) to point it to copy4, as I am not sure how reliable that touch file can work.
Also see the error 2841 explanation: http://www.symantec.com/docs/HOWTO91934