Restore from tape "pending" assignment of tape.

[smckelvey]

Netbackup 7.1.0.4 running on Windows Server 2008R2. Couple of Windows Media Servers (2003 and 2008). Storagetek L40 Library with SN3300 Fibre/SCSI router to each server can communicate direclty with the HCART LT03 tape drives in the library.

All of the equipment has been in place and working for a number of years, EXCEPT two weeks ago:

• I upgraded the Master Server from a 2003 machine to a 2008 machine. I did everything per the book and things have been fine.
• I also changed the Robot Control Host to one of the Media servers. This was necessary because of available SCSI connections.

Also in August, the SN3300 failed, so the drives and Storage Units were reconfigured temporarily until we got a replacement.

Anyways, I'm trying to do a restore and it sits indefinitely at "Pending" b/c it's waiting for one of the tapes see text below. All the tapes for the job are in the library, it has been inventoried, and the tapes show up. But the job will not run.

11/27/2012 4:03:11 PM - awaiting resource 000226 A pending request has been generated for this resource request.
  Operator action may be required. Pending Action: No action.,
  Media ID: 000226, Barcode: 000226, Density: hcart, Access Mode: Read,
  Action Drive Name: N/A, Action Media Server: N/A, Robot Number: N/A, Robot Type: NONE,
  Volume Group: 000_00000_TLD, Action Acs: N/A, Action Lsm: N/A
 

As I said, I haven't really ever had a problem before. Today I also did the following in the course of troubleshooting:

• I did a restore from a backup after the Master Server swap (from disk). That worked perfectly.
• I did a restore from before the Master Server swap (from tape, 2 mo old). That worked fine.
• I did a restore from before the SN3300 problem in August (from Tape). That worked fine.

I've also tried the following with no success:

• I've tried this particular restore several times.
• I've reinventoried more than once.
• Turned UAC off on the server. Restarted the server.
• Tried initiating the restore from another Media Server (including the Robot Control Host).

So, I'm at a loss and input is appreciated. Below is the nbemmcmd -listmedia -mediaid 000226 output. As well as another output for a tape from which I did restore today.

C:\Program Files\Veritas\NetBackup\bin\admincmd>nbemmcmd -listmedia -mediaid 000
226
NBEMMCMD, Version:7.1.0.4
====================================================================
Media GUID:                     d3915c0d-a246-4224-85e2-d7106b9132ec
Media ID:                       000226
Partner:                        -
Media Type:                     HCART
Volume Group:                   000_00000_TLD
Application:                    Netbackup
Media Flags:                    1
Description:                    Added by Media Manager
Barcode:                        000226
Partner Barcode:                --------
Last Write Host:                acct-srvr-db2
Created:                        07/25/2008 19:06
Time Assigned:                  11/15/2011 09:06
First Mount:                    07/26/2008 09:42
Last Mount:                     11/20/2011 11:57
Volume Expiration:              -
Data Expiration:                INFINITY
Last Written:                   11/20/2011 02:12
Last Read:                      -
Robot Type:                     TLD
Robot Control Host:             wtp-nbumedia
Robot Number:                   0
Slot:                           16
Side/Face:                      -
Cleanings Remaining:            -
Number of Mounts:               96
Maximum Mounts Allowed:         0
Media Status:                   FULL MPX
Kilobytes:                      406138272
Images:                         69
Valid Images:                   69
Retention Period:               9
Number of Restores:             0
Optical Header Size Bytes:      1024
Optical Sector Size Bytes:      0
Optical Partition Size Bytes:   0
Last Header Offset:             2504148
Adamm Guid:                     00000000-0000-0000-0000-000000000000
Rsm Guid:                       00000000-0000-0000-0000-000000000000
Origin Host:                    balt-srvr-nbu
Master Host:                    balt-srvr-nbu
Server Group:                   UNRESTRICTED_SHARING_GROUP
Upgrade Conflicts Flag:
Pool Number:                    8
Volume Pool:                    offsite_fulls
Previous Pool Name:             SCRATCH
Vault Flags:                    -
Vault Container:                -
Vault Name:                     -
Vault Slot:                     -
Session ID:                     -
Date Vaulted:                   -
Return Date:                    -
====================================================================
Command completed successfully.
 

C:\Program Files\Veritas\NetBackup\bin\admincmd>nbemmcmd -listmedia -mediaid 000
204
NBEMMCMD, Version:7.1.0.4
====================================================================
Media GUID:                     5b2ff07f-8b53-453a-91a9-d7638dff230f
Media ID:                       000204
Partner:                        -
Media Type:                     HCART
Volume Group:                   000_00000_TLD
Application:                    Netbackup
Media Flags:                    1
Description:                    Added by Media Manager
Barcode:                        000204
Partner Barcode:                --------
Last Write Host:                balt-srvr-nbu
Created:                        11/02/2007 17:24
Time Assigned:                  11/03/2007 21:02
First Mount:                    11/04/2007 01:37
Last Mount:                     11/27/2012 17:01
Volume Expiration:              -
Data Expiration:                INFINITY
Last Written:                   05/20/2012 17:05
Last Read:                      11/27/2012 17:01
Robot Type:                     TLD
Robot Control Host:             wtp-nbumedia
Robot Number:                   0
Slot:                           25
Side/Face:                      -
Cleanings Remaining:            -
Number of Mounts:               24
Maximum Mounts Allowed:         0
Media Status:                   FULL MPX
Kilobytes:                      414388429
Images:                         121
Valid Images:                   121
Retention Period:               9
Number of Restores:             1
Optical Header Size Bytes:      1024
Optical Sector Size Bytes:      0
Optical Partition Size Bytes:   0
Last Header Offset:             4717689
Adamm Guid:                     00000000-0000-0000-0000-000000000000
Rsm Guid:                       00000000-0000-0000-0000-000000000000
Origin Host:                    balt-srvr-nbu
Master Host:                    balt-srvr-nbu
Server Group:                   NO_SHARING_GROUP
Upgrade Conflicts Flag:
Pool Number:                    5
Volume Pool:                    FULL
Previous Pool Name:             SCRATCH
Vault Flags:                    -
Vault Container:                -
Vault Name:                     -
Vault Slot:                     -
Session ID:                     -
Date Vaulted:                   -
Return Date:                    -
====================================================================
Command completed successfully.

[Khajan]

Is this restore trying to mount tape on the master or media? You might have to try adding FORCE_RESTORE_MEDIA_SERVER = masterservername mediaservernamr into master's bp.conf.

[Marianne]

We see that media id 000226 was last written by media server acct-srvr-db2 and that tape is in slot 16 of robot 0, controlled by wtp-nbumedia. It was 'assigned' on 11/15/2011 09:06 and last written on 11/20/2011 02:12. So, changes made in August do not apply in this instance.

Question: does media server acct-srvr-db2  have access to tape/robot?
If not, assign ownership of tape to a media server that does:

bpmedia  -movedb -m <media_id> -newserver <hostname>

Else, config 'Failover Media Server' in Host Properties > Master > General Server > Media host override.

[smckelvey]

Marianne, I noticed that, and did check some things, but it looked fine to me (but I'm by no means an expert).

• It was showing up in the Robots node in the console.
• It's been running backups w/o issue (I assume that it would need robot control in order to do so).
• I could run an Inventory from said Media server.
• I've done successful restores w/ 1 media (happens to be the Robot Control Host) and the Master Server listed as the "Media Server" in the console. 

Previously, thinking that might be part of the problem, I edited the Restore Failover properties of the Master Server. I had no idea if that would help, but I tried it on a whim, to no avail.

I then checked the Media server (Acct-srvr-db2) and decided to try to reconfigure the the storage devices through the console. Eventhough i see the tape drives in Windows Device Manager, Netbackup isn't seeing them, so I guess that explains it. Restarting the services didn't help. The server is in production and I can't reboot it in the forseeable future. So, I tried your command on the Master Server to move the tape, and got the following:

C:\Program Files\Veritas\NetBackup\bin\admincmd>bpmedia -movedb -m 000226 -newse
rver balt-srvr-nbu
requested media id is not assigned to this host in the EMM database

CONCLUSION:

Finally, I added said media server to the "Media Host Override" list in the Master Server properties, per the Guide. That worked for the time being, until I can reboot the server.

I have no idea how said Media server is working for backups and how I can inventory from it, but obviously the problem is appears that the problem is that Netbackup doesn't know it has drives at this time

[Marianne · Accepted solution]

So, problem was solved by following the last point in my advice above to configure 'Media Host Override'?

Glad I could help to point you in the right direction!

[smckelvey]

I actually didn't notice that last sentence in your post: I ended up looking at the Admin Guide when reading online more about the Failover Restore Server setting (wondering if I'd missed something.

**I am confused what the difference is b/t these two things, but that needn't be addressed here**

Your post prompted me to delve further into the possible problem of it being the Media Server, which is how I found that. And had I not gotten sidetracked on running your command, the next step would have been your Media Host Override suggestion.

I wasn't sure if you were just asking, or passively implying that I should have marked your reply as the "Solution" (for points or whatever). You are welcome to it, if that's what you are asking for: I just figured for someone searching for a solution to a problem, it would be better to direct them to the reply with all of the details of the problem and work around.................

Regardless, as always, your posts are right on the money, very helpful, and much appreciated (whether it's my thread or someone else's that I'm reading). So, thanks for taking the time.