Restore status 83

[Netbackup_Adm4]

Reference: https://www-secure.symantec.com/connect/forums/client-restore-exit-status-83-media-open-error-disk

I am having this same error on my restores that are from basic disk on our media servers.  I am able to backup without issue but cannot restore.  I get the following errors.  This has been happening since migrating to 7.5.03 on new hardware.   Any help would be appreciated.  When this data is duplicated to tape we can restore successfully.

2012/08/13    9:39:07 AM    Critical    58873    Media Device    Invalid storage device: E:\DiskαÇáStaging no more entries
2012/08/13    9:39:07 AM    Critical    58873    Media Device    sts_get_lsu_prop_byname on LSU E:\Disk Staging failed: 2060013 no more entries
2012/08/13    9:39:07 AM    Error    58873    Retrieve    client restore EXIT STATUS 83: media open error
2012/08/13    9:39:07 AM    Warning    58873    Retrieve    Restore must be resumed prior to first image expiration on 8/20/2012 7:00:00 PM
2012/08/13    9:39:07 AM    Error    58873    Retrieve    The following files/folders were not restored:

[Marianne]

Please tell us how migration to new hardware was done?

We see the following reason for failure:

Invalid storage device: E:\DiskαÇáStaging

[Netbackup_Adm4]

1. We have 1 master server and 2 media servers
2. Old hardware was on W2K3 SP2 x86 with Netbackup 7.1.04.  New hardware on W2K8R2 SP1 (x64).
3. Performed catalog backup on old master server
4. Installed Netbackup 7.1.04 onto new master server
5. Recoverd catalog from old master server onto new master server
6. Deactivated old media servers and created new media servers on new hardware with same name and ip address. Activated media servers and configured storage etc.
7. Upgraded new master server from 7.1.04 to 7.5 and then to 7.5.03.
8. Upgraded media servers to 7.5 then to 7.5.03.

The name in the logs "E:\DiskαÇáStaging" is normal as netbackup seems to put those extra characters in the log to represent spaces for the basic disk location.

Any idea's why I would be getting status 83's?

[mph999]

Hmm, no idea if this is normal E:\DiskαÇáStaging - but I accept that is what you see on your system.  Personally, it doesn't look right to me.

How about this :

Create a new storage area say E:\Disk_Staging and see if that works, if only as a test.

I'll be honest, I don;t like spaces in things, good to put an _

The upgrade plan looks good at first glance, I wonder if there is something in 7.5.0.3 that doesn't like spaces.

If my idea works, how about just leave it like that ?

Martin

[Netbackup_Adm4]

Thanks for the help.  I could try that but the backups work fine.  My problem only occurs when trying to restore from disk.  We have had spaces in this basic disk folder location since we started using basic disk a few years ago.  These are synthetic fulls so I think I would have to perform a real full backup if I move to a new folder.  There are huge backups ( they will take 48 hours each) in this location so am a little relecutant to do this.  Anyone else have ideas?

[mph999]

Backups work, yes, but the restores don't so at the moment your backups might be useless.

How about running it as a test, create a new area and run a couple of test backups then try and restore.

In fact create two new areas, one with spaces, one without - what happens.

Let's just try and narrow things down.

Martin

[Netbackup_Adm4]

Thanks Martin.  I will give it a shot, good idea, better to narrow things down :)

[mph999]

Yes, not very technical, but a damm sight easier sometimes.

Have to go out, will check back later.

M 

[Marianne]

Are you trying to restore from pre-migration backups?

Is DSU path name EXACTLY as before migration?

Seems to me restore is looking for original path and cannot find it.

I assume that you have only copied last section of restore failure for your post. Can you see image number in the job details? If so, please post output of :

 bpimagelist -backupid client_10-digit-number -L

as well as

bpstulist -U

[Netbackup_Adm4]

I created a new storage unit that points to a folder without spaces (E:\StagingTest) and I receive the same error.

Critical    61123    Media Device    sts_get_lsu_prop_byname on LSU E:\StagingTest failed: 2060013 no more entries
Critical    61123    Media Device    Invalid storage device: E:\StagingTest no more entries
Critical    61123    Media Device    failure to open disk at path E:\StagingTest: plug-in reports error 2060013
Error        61123    Retrieve    client restore EXIT STATUS 83: media open error
Warning        61123    Retrieve    Restore must be resumed prior to first image expiration on 8/23/2012 11:24:04 AM

[Netbackup_Adm4]

Hi Marianne,

No, I am trying to restore from post migration backups.  Actually for the last test I just created the backup this morning. 

Here is the full log from the activity monitor.  I will get the other information to you shortly.

In the log below I replaced the server name with <servername>.

Thanks for your assistance.

2012/08/14 11:27:54 AM - begin Restore
2012/08/14 11:27:55 AM - restoring image <servername>_1344968644
2012/08/14 11:27:57 AM - Info bprd(pid=6672) Restoring from copy 1 of image created 08/14/12 11:24:04    
2012/08/14 11:27:58 AM - Info bpbrm(pid=9104) <servername> is the host to restore to      
2012/08/14 11:27:58 AM - Info bpbrm(pid=9104) reading file list from client        
2012/08/14 11:28:01 AM - connecting
2012/08/14 11:28:01 AM - Info bpbrm(pid=9104) starting bptm           
2012/08/14 11:28:04 AM - Info tar32(pid=7584) Restore started           
2012/08/14 11:28:04 AM - connected; connect time: 00:00:03
2012/08/14 11:28:04 AM - Info bptm(pid=6900) start            
2012/08/14 11:28:04 AM - started process bptm (6900)
2012/08/14 11:28:04 AM - Info bpdm(pid=6900) reading backup image          
2012/08/14 11:28:05 AM - Info bptm(pid=6900) using 32 data buffers         
2012/08/14 11:28:05 AM - Info bptm(pid=6900) spawning a child process         
2012/08/14 11:28:05 AM - Info bptm(pid=6900) child pid: 6524          
2012/08/14 11:28:05 AM - Info bptm(pid=6524) start            
2012/08/14 11:28:05 AM - started process bptm (6524)
2012/08/14 11:28:05 AM - Critical bptm(pid=6900) sts_get_lsu_prop_byname on LSU E:\StagingTest failed: 2060013 no more entries    
2012/08/14 11:28:05 AM - Critical bptm(pid=6900) Invalid storage device: E:\StagingTest no more entries      
2012/08/14 11:28:05 AM - Critical bptm(pid=6900) failure to open disk at path E:\StagingTest: plug-in reports error 2060013  
2012/08/14 11:28:05 AM - Info bptm(pid=6900) EXITING with status 83 <----------        
2012/08/14 11:28:05 AM - Error bptm(pid=6524) The following files/folders were not restored:       
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/AdobeFlash10.2.202.228.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/AdobeFlash11.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/CMS1.4.2.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/CheckPointFDER73_7.6.119.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/ConnectionManager5.0.0.22Script.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/ConnectionMgr5.0.0.22.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/DB2_9.7FP4_RT.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) UTF - /C/temp/Logs/EV_V9.0.2_HTTP.log          
2012/08/14 11:28:05 AM - Error bptm(pid=6524) more than 10 files were not restored, remaining ones are shown in the progress log.
2012/08/14 11:28:07 AM - Info tar32(pid=7584) done. status: 83: media open error       
2012/08/14 11:28:07 AM - Error bpbrm(pid=9104) client restore EXIT STATUS 83: media open error     
2012/08/14 11:28:07 AM - restored image <servername>_1344968644 - (media open error(83)); restore time 00:00:12
2012/08/14 11:28:07 AM - Warning bprd(pid=6672) Restore must be resumed prior to first image expiration on 8/23/2012 11:24:04 AM
2012/08/14 11:28:07 AM - end Restore; elapsed time: 00:00:13
media open error(83)

[Netbackup_Adm4]

Here is the result of the bpimagelist:

C:\Windows\System32>bpimagelist -backupid client1_1344968644 -

Client:            client1
Backup ID:         client1_1344968644
Policy:            synthetic_test
Policy Type:       MS-Windows (13)
Proxy Client:      (none specified)
Creator:           root
Name1:             (none specified)
Sched Label:       full_synthetic_backup
Schedule Type:     FULL (0)
Retention Level:   9 days (24)
Backup Time:          8/14/2012 11:24:04 AM (1344968644)
Elapsed Time:      86 second(s)
Expiration Time:      8/23/2012 11:24:04 AM (1345746244)
Compressed:        no
Client Encrypted:  no
Kilobytes:         497658
Number of Files:   962
Number of Copies:  1
Number of Fragments:   2
Histogram:         0 0 0 0 0 0 0 0 0 0
DB Compressed:     no
Files File Name:   synthetic_test_1344968644_FULL.f
Previous Backup Files File Name:   (none specified)
Parent Backup Image File Name:   (none specified)
SW Version:        (none specified)
Options:           0x0
MPX:               0
TIR Info:          2
TIR Expiration:       8/15/2012 11:24:04 AM (1345055044)
Keyword:           area offices
Ext Security Info: no
File Restore Raw:  no
Image Dump Level:  0
File System Only:  no
Object Descriptor: (none specified)
Previous BI Time:     12/31/1969 4:00:00 PM (0)
BI Full Time:         12/31/1969 4:00:00 PM (0)
Request Pid:       0
Backup Status:     0
Stream Number:     0
Backup Copy:       Standard (0)
Files File size:     125007
PFI type:     0
IMAGE_ATTRIBUTE:     0
Primary Copy:      1
Image Type:        0  (Regular)
Job ID:            61122
Num Resumes:       0
Resume Expiration:    12/31/1969 4:00:00 PM (0)
Data Classification:    (none specified)
Data_Classification_ID: (none specified)
Storage Lifecycle Policy:    (none specified)
Storage Lifecycle Policy Version:    0
STL_Completed:      0
Remote Expiration Time:    12/31/1969 4:00:00 PM (0)
Origin Master Server:  (none specified)
Origin Master GUID:    (none specified)
Snap Time:         12/31/1969 4:00:00 PM (0)
IR Enabled:      no
Client Character Set:     0
Image On Hold:     0
Indexing Status:   0
Copy number:       1
 Fragment:         TIR (-1)
 Kilobytes:        122
 Remainder:        79
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               E:\StagingTest\client1_1344968644_C1_TIR
 Host:             mediaserver2
 Block Size:       65536
 Offset:           0
 Media Date:          12/31/1969 4:00:00 PM (0)
 Dev Written On:   -1
 Flags:            0x0
 Media Descriptor:        mediaserver2
 checkpoint:       0
 resume num:       0
 Key tag:          *NULL*
 STL tag:          *NULL*
 Copy on hold:     0
Copy number:       1
 Fragment:         1
 Kilobytes:        497536
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               E:\StagingTest\client1_1344968644_C1_F1
 Host:             mediaserver2
 Block Size:       65536
 Offset:           0
 Media Date:          12/31/1969 4:00:00 PM (0)
 Dev Written On:   -1
 Flags:            0x0
 Media Descriptor:        mediaserver2
 Expiration Time:     8/23/2012 11:24:04 AM (1345746244)
 MPX:              0
 retention_lvl:    9 days (24)
 Try to Keep Time:     12/31/1969 4:00:00 PM (0)
 Copy Creation Time:     8/14/2012 11:25:30 AM (1344968730)
 Data Format:      Tar
 checkpoint:       0
 resume num:       0
 Copy on hold:     0
 

[Netbackup_Adm4]

here is the bpstulist -U results:

Label:                mediaserver2TEST
Storage Unit Type:    Disk
Storage Unit Subtype: Basic (1)
Host Connection:      mediaserver2
Concurrent Jobs:      5
On Demand Only:       yes
Max MPX:              1
Path:                 "E:\StagingTest"
Max Fragment Size:    2000 MB
Stage data:           no
Block Sharing:        no
High Water Mark:      95
Low Water Mark:       80
Ok On Root:           no

[Marianne]

Please show us contents of E:\StagingTest\  on mediaserver2?

Post screenshot or else 'dir' listing in cmd window.

[Netbackup_Adm4]

E:\StagingTest>dir
 Volume in drive E
 Volume Serial Number is

 Directory of E:\StagingTest

08/14/2012  11:25 AM    <DIR>          .
08/14/2012  11:25 AM    <DIR>          ..
08/14/2012  11:21 AM       506,914,816 client1_1344968433_C1_F1.1344968433.img
08/14/2012  11:21 AM             1,024 client1_1344968433_C1_F1.1344968433.info

08/14/2012  11:20 AM               512 client1_1344968433_C1_HDR.1344968433.img

08/14/2012  11:20 AM             1,024 client1_1344968433_C1_HDR.1344968433.inf
o
08/14/2012  11:21 AM           121,856 client1_1344968433_C1_TIR.1344968433.img

08/14/2012  11:21 AM             1,024 client1_1344968433_C1_TIR.1344968433.inf
o
08/14/2012  11:24 AM         2,829,312 client1_1344968643_C1_F1.1344968643.img
08/14/2012  11:24 AM             1,024 client1_1344968643_C1_F1.1344968643.info

08/14/2012  11:24 AM               512 client1_1344968643_C1_HDR.1344968643.img

08/14/2012  11:24 AM             1,024 client1_1344968643_C1_HDR.1344968643.inf
o
08/14/2012  11:24 AM           125,952 client1_1344968643_C1_TIR.1344968643.img

08/14/2012  11:24 AM             1,024 client1_1344968643_C1_TIR.1344968643.inf
o
08/14/2012  11:25 AM       509,476,864 client1_1344968644_C1_F1.1344968644.img
08/14/2012  11:25 AM             1,024 client1_1344968644_C1_F1.1344968644.info

08/14/2012  11:25 AM               512 client1_1344968644_C1_HDR.1344968644.img

08/14/2012  11:25 AM             1,024 client1_1344968644_C1_HDR.1344968644.inf
o
08/14/2012  11:25 AM           125,952 client1_1344968644_C1_TIR.1344968644.img

08/14/2012  11:25 AM             1,024 client1_1344968644_C1_TIR.1344968644.inf
o
              18 File(s)  1,019,605,504 bytes
               2 Dir(s)  1,450,296,897,536 bytes free

[Marianne]

I am officially out of ideas...

Hopefully Martin will be back  s-o-o-n !

[mph999]

The good news - here I am

The bad news - I am out of ideas ...

Ok, on this new STU did you make a new backup and then try and restore from the new backup.

I think you did, but please just confirm... Then, run (yes another) brand new backup to your new STU (reason being I don't want to play with real images).

Then confirm the restore of this new image fails

Then run bpimagelist -backupid <backup id of this new image> and post the output here .. Lets look at the 'raw catalog data', just maybe we can spot something.

Hopefully this will work ...

nbdevquery -listdp -stype BasicDisk -U (I presume this is basic disk, if not put AdvancedDisk)

Think this should list the 'flags' on the diskpool, perhaps something is amiss.

Very odd, how on earth it can write to it and not read.

M

[mph999]

Got to go and pick up the other half, won't be long, I'll have a quick look in the interal DB / etracks when I get back.

M

[Netbackup_Adm4]

thanks for your assistance on this:)  I am heading out for the day.  I'll catch up with you tomorrow.

[mph999]

Before I forget, can we get bptm and bpdm logs at verbose 5 / general 2

thx.

M