cancel
Showing results for 
Search instead for 
Did you mean: 

Netbackup restore failed - Restore error(2850)

Henning_General
Level 4

Hello,

I need some help. I want to restore LTO3 tapes with a robot libary. I want to restore via ndmp to an cellera ns480 (nas).

I´m getting the following error message:

12/4/2015 10:29:06 AM - begin Restore
12/4/2015 10:29:07 AM - 1 images required
12/4/2015 10:29:07 AM - media 000399 required
12/4/2015 10:29:08 AM - restoring image defmoa_1230568092
12/4/2015 10:29:08 AM - Info bpbrm(pid=5912) defmoa is the host to restore to      
12/4/2015 10:29:08 AM - Info bpbrm(pid=5912) telling media manager to start restore on client     
12/4/2015 10:29:09 AM - Info bpbrm(pid=5924) defmoa07 is the host to restore to      
12/4/2015 10:29:10 AM - Info bpbrm(pid=5924) start tar32 on client         
12/4/2015 10:29:10 AM - Info tar32(pid=4624) Restore started.           
12/4/2015 10:29:10 AM - connected
12/4/2015 10:29:10 AM - requesting resource 000399
12/4/2015 10:29:11 AM - Info bptm(pid=6960) Waiting for mount of media id 000399 (copy 1) on server ge-2008. 
12/4/2015 10:29:11 AM - started process bptm (6960)
12/4/2015 10:29:11 AM - mounting 000399
12/4/2015 10:29:11 AM - Info bptm(pid=6960) INF - Waiting for mount of media id 000399 on server ge-2008 for reading.
12/4/2015 10:29:11 AM - granted resource 000399
12/4/2015 10:29:11 AM - granted resource HP.ULTRIUM3-SCSI.000
12/4/2015 10:44:14 AM - Error bptm(pid=6960) error requesting media, TpErrno = Robot operation failed     
12/4/2015 10:44:55 AM - current media 000399 complete, requesting next resource HP.ULTRIUM3-SCSI.000:NetBackup:000399
12/4/2015 11:15:23 AM - Error ndmpagent(pid=4624) NDMP restore failed from path UNKNOWN       
12/4/2015 11:15:23 AM - Info ndmpagent(pid=4624) done. status: 150: termination requested by administrator      
12/4/2015 11:15:23 AM - Info bpbrm(pid=5912) got ERROR 98 from media manager       
12/4/2015 11:15:23 AM - Info bpbrm(pid=5912) terminating bpbrm child 5924 jobid=11        
12/4/2015 11:15:23 AM - restored image defmoa_1230568092 - (error requesting media (tpreq)(98)); restore time 00:46:15
12/4/2015 11:15:23 AM - Error nbjm(pid=3544) NBU status: 98, EMM status: The media server reported a system error 
12/4/2015 11:15:24 AM - end Restore; elapsed time: 00:46:18
Restore error(2850)

 

I´m able to move the tapes with robotest:

Robot Selection
---------------
  1)  TLD 0
  2)  none/quit
Enter choice: 1
1

Robot selected: TLD(0)   robotic path = {3,0,6,0}

Invoking robotic test utility:
C:\Program Files\Veritas\Volmgr\bin\tldtest.exe -rn 0 -r {3,0,6,0}

Opening {3,0,6,0}
MODE_SENSE complete
Enter tld commands (? returns help information)
s d
drive 1 (addr 500) access = 0 Contains Cartridge = yes
Source address = 1000 (slot 1)
Barcode = 000401
drive 2 (addr 501) access = 1 Contains Cartridge = no
drive 3 (addr 502) access = 0 Contains Cartridge = yes
Source address = 1002 (slot 3)
Barcode = 000399
drive 4 (addr 503) access = 1 Contains Cartridge = no
drive 5 (addr 504) access = 0 Contains Cartridge = yes
Source address = 1001 (slot 2)
Barcode = 000400
drive 6 (addr 505) access = 1 Contains Cartridge = no
READ_ELEMENT_STATUS complete

Invalid command
m d1 s100
Initiating MOVE_MEDIUM from address 500 to 1099
MOVE_MEDIUM complete
m d2 s101
Initiating MOVE_MEDIUM from address 501 to 1100
move_medium failed
sense key = 0x5, asc = 0x3b, ascq = 0xe, MEDIUM SOURCE ELEMENT EMPTY
m d3 s101
Initiating MOVE_MEDIUM from address 502 to 1100
MOVE_MEDIUM complete
m d5 s102
Initiating MOVE_MEDIUM from address 504 to 1101
MOVE_MEDIUM complete

 

Has anyone an idea ? 

 

Thnaks for your help,

Henning

 

1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Have a look at this TN:

3rd PARTY ISSUE: An "NDMP_HALT_INTERNAL_ERROR" occurs during a restore to an EMC Celerra NAS appliance. 
http://www.veritas.com/docs/000029418 

Extract:

Although the EMC Data Mover's root partition (/) can be backed up, the EMC Celerra does not allow a restore of the root partition to its original location. It is necessary to restore data to a different path, or create a folder to direct the restore to.

View solution in original post

20 REPLIES 20

mnolan
Level 6
Employee Accredited Certified

Your robot control host system logs should contain the error for tld from 12/4/2015 10:44:14 AM.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
You will probably need to enable Media Manager logging. On the robot control host, add VERBOSE entry to vm.conf. Create debug directory under volmgr. Create these folders in debug: robot reqlib daemon. Restart NBU Device Manager service. Also ensure that bptm log folder exists under netbackup\logs and increase logging for bptm to minimum of 3. After next failed restore attempt, copy logs to .txt files (e.g. robot.txt) and upload as File attachments. Check Windows Event Viewer Application log for errors as well.

cruisen
Level 6
Partner Accredited

Hello Henning_General,

This remembers me an issue that was related to buffer size parameter on the master or mediaserver that were incompatible with the setting of the NDMP host.

Take into consideration that if the backup happened at 64 kb size and now you have set this up on the server to 256, this will not work.

The first thing would be to check how buffer size is set on the ndmp device, which are supported. Even easier is to set up buffersize manually on Netbackup  so that it is the same as on NDMP device. Take also in consideration when the backup happened, to gather which setting probably exist on these days, to find the exact valeu, that matches.

https://www.veritas.com/support/en_US/article.TECH1724

best regards,

Cruisen

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Buffer is not applicable here. The mount is not even taking place and fails with robot error. We will need logs to see why NBU says the mount has failed... Something that we have not determined is if the mount is acknowledged by the OS. Even if the mount is successful, the only comms at this point is on the control path. After this, the mount must be acknowledged at OS level via the data path. Curious to see what vmoprcmd says after the robtest tape mount. The logs will also tell if the OS reports successful mount to NBU.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Curious to know if normal backups and restores (non-ndmp) are working using this tape library and drives on this master/media server.

cruisen
Level 6
Partner Accredited

Hi Marianne,

I looked this up, i was not aware of that but, Nicolai states in ; https://www-secure.symantec.com/connect/forums/how-find-maximum-block-size-tape-device

==> Direct NDMP where the device is writing its own data to tape, Netbackup is merely a control station and out of the data loop. Depending on the vendor (Neapp, EMC) and firmware revision, they may have a issue with other settings than the default. Best way to verify is look in the vendors documentation.

Does this not mean that control of same bufferzize mostly should happen befor the data is send from the ndmp device to robot.

If this check fails, why should there be a mount request, but I have to say that i do not know exactly the processes behind. I just experienced issues that where related to buffersize incompatible to celerra devices.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
We need to see logs to see what happened in the period between assigning resources and the error: 12/4/2015 10:29:11 AM - granted resource HP.ULTRIUM3-SCSI.000 .... NOTE the 15 minutes gap here.... 12/4/2015 10:44:14 AM - Error bptm(pid=6960) error requesting media, TpErrno = Robot operation failed

Henning_General
Level 4

Hello,

I created the log/debug folder.

After a start from the nbu services the drives are not avaible. (Drive Path: Missing Drive Path)

The device manager don´t find the drives as well.

When I try to "up Drive" this errror appear : Unable to up the selected drive : HP.Ultrium3-scsi.000. Unkown Drive 297.

What can I do ?

You need any logs ?

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Oh dear... one step forward, 3 steps back.

It seems the OS has lost connectivity to the devices.

Check physical connection.
Try to rebbot the tape library.
Reboot the master/media server.

Delete existing tape drive(s) and Medium Changer in Windows Device Manager and then rescan.

If devices are now seen in Device Manager, use 'scan' command (in ...volmgr\bin) to see if devices are responding to SCSI-commands.

Only if scan can see the devices correctly, see if NBU can now see and use devices.

Device paths may have changed - if so, delete and re-add devices using the Device Wizard.

On another note:

Always remember to exit robtest after testing hardware.
If left open, this utility will remove robot control from NetBackup and leave drives in AVR (non-robotic) state.

Henning_General
Level 4

Hello cruisen  Marianne ,

 I´m able to connected to the drives.

Have a look in the appendix, I will Upload the log files.

 

The Restore Job failed with the following Error:

12/8/2015 3:59:54 PM - begin Restore
12/8/2015 3:59:55 PM - 1 images required
12/8/2015 3:59:55 PM - media 000399 required
12/8/2015 3:59:56 PM - restoring image defmoa07psge_1230568092
12/8/2015 3:59:56 PM - Info bpbrm(pid=1812) defmoa07psge is the host to restore to     
12/8/2015 3:59:56 PM - Info bpbrm(pid=1812) telling media manager to start restore on client    
12/8/2015 3:59:57 PM - Info bpbrm(pid=3864) defmoa07psge is the host to restore to     
12/8/2015 3:59:57 PM - Info bpbrm(pid=3864) start tar32 on client        
12/8/2015 3:59:58 PM - Info tar32(pid=3156) Restore started.          
12/8/2015 3:59:58 PM - connected
12/8/2015 3:59:58 PM - Info bptm(pid=3740) Waiting for mount of media id 000399 (copy 1) on server ge-2008.
12/8/2015 3:59:58 PM - started process bptm (3740)
12/8/2015 3:59:58 PM - mounting 000399
12/8/2015 3:59:58 PM - Info bptm(pid=3740) INF - Waiting for mount of media id 000399 on server ge-2008 for reading.
12/8/2015 3:59:58 PM - requesting resource 000399
12/8/2015 3:59:58 PM - granted resource 000399
12/8/2015 3:59:58 PM - granted resource HP.ULTRIUM3-SCSI.001
12/8/2015 4:00:38 PM - mounted; mount time: 00:00:40
12/8/2015 4:00:38 PM - Info bptm(pid=3740) 000399           
12/8/2015 4:00:38 PM - Info bptm(pid=3740) INF - Waiting for positioning of media id 000399 on server ge-2008 for reading.
12/8/2015 4:00:38 PM - positioning 000399 to file 7
12/8/2015 4:02:08 PM - positioned 000399; position time: 00:01:30
12/8/2015 4:02:08 PM - begin reading
12/8/2015 4:02:09 PM - Info ndmpagent(pid=3156) INF - Restoring NDMP files from /root_vdm_4/DEP5/Blizzard/Electronics/1995/BUE/BERICHTE/ to [See line below] 
12/8/2015 4:02:09 PM - Info ndmpagent(pid=3156) INF - Restoring NDMP files from [See line above] to //192.168.100.21/enovia 
12/8/2015 4:02:10 PM - Error ndmpagent(pid=3156) ndmp_authenticate_connection: ndmp_attr_lookup_host for host defmoa07psge failed      
12/8/2015 4:02:10 PM - Error ndmpagent(pid=3156) failed to connect to NDMP server defmoa07psge, connect error = -1001 (NDMP attribute lookup failed - verify attributes with tpautoconf -verify)
12/8/2015 4:02:10 PM - Error ndmpagent(pid=3156) NDMP restore failed from path /root_vdm_4/DEP5/Blizzard      
12/8/2015 4:02:10 PM - Info ndmpagent(pid=3156) done. status: 25: cannot connect on socket     
12/8/2015 4:02:10 PM - Info bpbrm(pid=1812) child done, status 25        
12/8/2015 4:02:10 PM - Info bpbrm(pid=1812) sending message to media manager: STOP RESTORE defmoa07psge_1230568092    
12/8/2015 4:02:11 PM - Info bpbrm(pid=1812) got ERROR 5 from media manager      
12/8/2015 4:02:11 PM - restored image defmoa07psge_1230568092 - (the restore failed to recover the requested files(5)); restore time 00:02:15
12/8/2015 4:02:11 PM - end Restore; elapsed time: 00:02:17
Restore error(2850)

Henning_General
Level 4

The log files bptm & daemon

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
No idea what you've done since this morning, but the device errors are now gone and we are back to ndmp protocol error: Error ndmpagent(pid=3156)  ndmp_authenticate_connection: ndmp_attr_lookup_host for host defmoa07psge  failed       12/8/2015 4:02:10 PM - Error ndmpagent(pid=3156) failed to connect to NDMP  server defmoa07psge, connect error = -1001 (NDMP attribute lookup failed -  verify attributes with tpautoconf -verify) Now that you have connected the filer to the backup network, start again with ndmp connect and verify as per your original post when filer was behind firewall. Have you added hosts entry on the master/media server to lookup filer hostname using backup network IP?

Henning_General
Level 4

s

Henning_General
Level 4

Ignore the comment above.

I found the mistake ..

Have a look at this error:

12/9/2015 8:33:31 AM - begin Restore
12/9/2015 8:33:31 AM - 1 images required
12/9/2015 8:33:31 AM - media 000399 required
12/9/2015 8:33:32 AM - restoring image defmoa07psge_1230568092
12/9/2015 8:33:33 AM - Info bpbrm(pid=6244) defmoa07psge is the host to restore to     
12/9/2015 8:33:33 AM - Info bpbrm(pid=6244) telling media manager to start restore on client    
12/9/2015 8:33:34 AM - Info bpbrm(pid=3872) defmoa07psge is the host to restore to     
12/9/2015 8:33:34 AM - Info bpbrm(pid=3872) start tar32 on client        
12/9/2015 8:33:34 AM - Info tar32(pid=6568) Restore started.          
12/9/2015 8:33:34 AM - connected
12/9/2015 8:33:35 AM - requesting resource 000399
12/9/2015 8:33:35 AM - awaiting resource 000399 Reason: Media is in use, Media Server: N/A,
  Robot Number: NONE, Robot Type: NONE, Media ID: 000399, Drive Name: N/A,
  Volume Pool: N/A, Storage Unit: N/A, Drive Scan Host: N/A
 
12/9/2015 8:34:33 AM - Info bptm(pid=2000) Waiting for mount of media id 000399 (copy 1) on server ge-2008.
12/9/2015 8:34:33 AM - started process bptm (2000)
12/9/2015 8:34:33 AM - mounting 000399
12/9/2015 8:34:33 AM - Info bptm(pid=2000) INF - Waiting for mount of media id 000399 on server ge-2008 for reading.
12/9/2015 8:34:33 AM - granted resource 000399
12/9/2015 8:34:33 AM - granted resource HP.ULTRIUM3-SCSI.000
12/9/2015 8:35:14 AM - mounted; mount time: 00:00:41
12/9/2015 8:35:14 AM - Info bptm(pid=2000) 000399           
12/9/2015 8:35:14 AM - Info bptm(pid=2000) INF - Waiting for positioning of media id 000399 on server ge-2008 for reading.
12/9/2015 8:35:14 AM - positioning 000399 to file 7
12/9/2015 8:36:44 AM - positioned 000399; position time: 00:01:30
12/9/2015 8:36:44 AM - begin reading
12/9/2015 8:36:46 AM - Info ndmpagent(pid=6568) INF - Restoring NDMP files from /root_vdm_4/DEP5/Blizzard/Electronics/1995/BUE/BERICHTE/ to [See line below] 
12/9/2015 8:36:46 AM - Info ndmpagent(pid=6568) INF - Restoring NDMP files from [See line above] to /\\defmoa07psge/java 
12/9/2015 8:36:47 AM - Info ndmpagent(pid=6568) DAR enabled          
12/9/2015 8:36:48 AM - Error ndmpagent(pid=6568) defmoa07psge: Permission denied to restore on Root fs    
12/9/2015 8:36:49 AM - Info ndmpagent(pid=6568) defmoa07psge: server_archive: emctar vol 1, 1 files, 131072 bytes read, 0 bytes written
12/9/2015 8:36:49 AM - Error ndmpagent(pid=6568) NDMP restore failed from path /root_vdm_4/DEP5/Blizzard      
12/9/2015 8:36:49 AM - Info ndmpagent(pid=6568) done. status: 5: the restore failed to recover the requested files 
12/9/2015 8:36:49 AM - end reading; read time: 00:00:05
12/9/2015 8:36:49 AM - Info bpbrm(pid=6244) media manager for backup id defmoa07psge_1230568092 exited with status 0: the requested operation was successfully completed
12/9/2015 8:36:49 AM - Info bpbrm(pid=6244) child done, status 5        
12/9/2015 8:36:49 AM - restored image defmoa07psge_1230568092 - (the restore failed to recover the requested files(5)); restore time 00:03:17
12/9/2015 8:36:49 AM - end Restore; elapsed time: 00:03:18
Restore error(2850)


C:\Program Files\Veritas\Volmgr\bin>tpautoconf -verify defmoa07psge
Connecting to host "defmoa07psge" as user "ndmp_new"...
Waiting for connect notification message...
Opening session--attempting with NDMP protocol version 4...
Opening session--successful with NDMP protocol version 4
  host supports TEXT authentication
  host supports MD5 authentication
Getting MD5 challenge from host...
Logging in using MD5 method...
Host info is:
  host name "server_2"
  os type "DartOS"
  os version "EMC Celerra File Server.T.5.6.52.2"
  host id "abc1997"
Login was successful
Host supports LOCAL backup/restore
Host supports 3-way backup/restore

 

regards, Henning

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Permission denied to restore on Root fs    

 

So - successful when you restore somewhere else?

Henning_General
Level 4

Not really, it doesn´t matter which path is selected. I always get the same error :

12/9/2015 8:53:42 AM - Error ndmpagent(pid=3772) defmoa07psge: Permission denied to restore on Root fs   

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Do you have a Filer Admin?

You need to select/specify a folder on a volume that is not part of root filesystem.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Have a look at this TN:

3rd PARTY ISSUE: An "NDMP_HALT_INTERNAL_ERROR" occurs during a restore to an EMC Celerra NAS appliance. 
http://www.veritas.com/docs/000029418 

Extract:

Although the EMC Data Mover's root partition (/) can be backed up, the EMC Celerra does not allow a restore of the root partition to its original location. It is necessary to restore data to a different path, or create a folder to direct the restore to.

Henning_General
Level 4

Thanks a lot, it works.

I created a new share. Now Im able to restore to the new created folder.