cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Veritas.com
Support

Media server -- Drive not visible when i run vmoprcmd command

Krishma
Level 4

‎07-24-2017 12:23 PM

Hi,

Need your help . We have a master/ media server on LINUX with NBU version 6.5 . I am not able to bring UP a drive . It says :

oprd returned abnormal status (96)
IPC Error: Daemon may not be running

Here are the logs :


#############################

[root@veritas log]# grep "DOWN" /var/log/messages-20170716
Jul 10 11:24:53 veritas ltid[18095]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 0)

Jul 19 09:52:21 veritas ltid[21804]: ltid can not be started while resources are assigned to the host.

/usr/openv/volmgr/debug - Logs

09:56:22.491 [22231] <2> wait_oprd_ready: oprd response: EXIT_STATUS 278
09:56:22.491 [22231] <2> put_string: cannot write data to network: Broken pipe (32)
09:56:22.491 [22231] <16> send_string: unable to send data to socket: Broken pipe (32), stat=-5

09:56:22.491 [22233] <16> oprd: device management error: IPC Error: Daemon may not be running

[root@veritas bin]# /usr/openv/volmgr/bin/vmoprcmd

HOST STATUS
Host Name Version Host Status
========================================= ======= ===========
veritas 656000 ACTIVE-DISK

PENDING REQUESTS


<NONE>

DRIVE STATUS

Drive Name Label Ready RecMID ExtMID Wr.Enbl. Type
Host DrivePath Status
=============================================================================

[root@veritas ltid]# /usr/openv/volmgr/bin/tpconfig -l
Device Robot Drive Robot Drive Device Second
Type Num Index Type DrNum Status Comment Name Path Device Path
robot 0 - TLD - - - - /dev/sg6
drive - 0 dlt3 1 DOWN - IBM.ULTRIUM-HH6.000 /dev/nst2
drive - 1 dlt3 2 UP - IBM.ULTRIUM-HH6.001 /dev/nst5
drive - 2 dlt3 4 UP - IBM.ULTRIUM-HH6.002 /dev/nst1
drive - 3 dlt3 3 UP - IBM.ULTRIUM-HH6.003 /dev/nst3
drive - 4 dlt3 5 UP - IBM.ULTRIUM-HH6.004 /dev/nst4
drive - 5 dlt3 6 UP - IBM.ULTRIUM-HH6.005 /dev/nst0

[root@veritas bin]# /usr/openv/volmgr/bin/tpconfig -d
Id DriveName Type Residence
Drive Path Status
****************************************************************************
0 IBM.ULTRIUM-HH6.000 dlt3 TLD(0) DRIVE=1
/dev/nst2 DOWN
1 IBM.ULTRIUM-HH6.001 dlt3 TLD(0) DRIVE=2
/dev/nst5 UP
2 IBM.ULTRIUM-HH6.002 dlt3 TLD(0) DRIVE=4
/dev/nst1 UP
3 IBM.ULTRIUM-HH6.003 dlt3 TLD(0) DRIVE=3
/dev/nst3 UP
4 IBM.ULTRIUM-HH6.004 dlt3 TLD(0) DRIVE=5
/dev/nst4 UP
5 IBM.ULTRIUM-HH6.005 dlt3 TLD(0) DRIVE=6
/dev/nst0 UP

 

[root@veritas ~]# /usr/openv/volmgr/bin/vmoprcmd -up 0
oprd returned abnormal status (96)
IPC Error: Daemon may not be running


--------------------------------------------------------------------------------------------------------------------


Reports :

[root@veritas ~]# /usr/openv/volmgr/bin/tpautoconf -report_disc
======================= New Device (Robot) =======================
Inquiry = "SPECTRA PYTHON 2000"
Serial Number = 9111005888
Robot Path = /dev/sg9
Number of Drives = 6
Number of Slots = 120
Number of Media Access Ports = 8
Drive = 1, Drive Name = /dev/nst5, Serial Number = 1012005888
Drive = 2, Drive Name = /dev/nst2, Serial Number = 1013005888
Drive = 3, Drive Name = /dev/nst1, Serial Number = 1014005888
Drive = 4, Drive Name = /dev/nst3, Serial Number = 1015005888
Drive = 5, Drive Name = /dev/nst0, Serial Number = 1016005888
Drive = 6, Drive Name = /dev/nst4, Serial Number = 1017005888
======================= New Device (Drive) =======================
Inquiry = "IBM ULTRIUM-HH6 G9P1"
Serial Number = 1012005888
Drive Path = /dev/nst5
Found as drive=1 in new robot
Robot's inquiry = "SPECTRA PYTHON 2000"
Robot's Serial Number = 9111005888
Robot's device path = /dev/sg9
======================= New Device (Drive) =======================
Inquiry = "IBM ULTRIUM-HH6 G9P1"
Serial Number = 1017005888
Drive Path = /dev/nst4
Found as drive=6 in new robot
Robot's inquiry = "SPECTRA PYTHON 2000"
Robot's Serial Number = 9111005888
Robot's device path = /dev/sg9
======================= New Device (Drive) =======================
Inquiry = "IBM ULTRIUM-HH6 G9P1"
Serial Number = 1015005888
Drive Path = /dev/nst3
Found as drive=4 in new robot
Robot's inquiry = "SPECTRA PYTHON 2000"
Robot's Serial Number = 9111005888
Robot's device path = /dev/sg9
======================= New Device (Drive) =======================
Inquiry = "IBM ULTRIUM-HH6 G9P1"
Serial Number = 1013005888
Drive Path = /dev/nst2
Found as drive=2 in new robot
Robot's inquiry = "SPECTRA PYTHON 2000"
Robot's Serial Number = 9111005888
Robot's device path = /dev/sg9
======================= New Device (Drive) =======================
Inquiry = "IBM ULTRIUM-HH6 G9P1"
Serial Number = 1014005888
Drive Path = /dev/nst1
Found as drive=3 in new robot
Robot's inquiry = "SPECTRA PYTHON 2000"
Robot's Serial Number = 9111005888
Robot's device path = /dev/sg9
======================= New Device (Drive) =======================
Inquiry = "IBM ULTRIUM-HH6 G9P1"
Serial Number = 1016005888
Drive Path = /dev/nst0
Found as drive=5 in new robot
Robot's inquiry = "SPECTRA PYTHON 2000"
Robot's Serial Number = 9111005888
Robot's device path = /dev/sg9
[root@veritas ~]#


[root@veritas ltid]# /usr/openv/volmgr/bin/tpconfig -l
Device Robot Drive Robot Drive Device Second
Type Num Index Type DrNum Status Comment Name Path Device Path
robot 0 - TLD - - - - /dev/sg6
drive - 0 dlt3 1 DOWN - IBM.ULTRIUM-HH6.000 /dev/nst2
drive - 1 dlt3 2 UP - IBM.ULTRIUM-HH6.001 /dev/nst5
drive - 2 dlt3 4 UP - IBM.ULTRIUM-HH6.002 /dev/nst1
drive - 3 dlt3 3 UP - IBM.ULTRIUM-HH6.003 /dev/nst3
drive - 4 dlt3 5 UP - IBM.ULTRIUM-HH6.004 /dev/nst4
drive - 5 dlt3 6 UP - IBM.ULTRIUM-HH6.005 /dev/nst0

[root@veritas volmgr]# /usr/openv/volmgr/bin/tpautoconf -t
TPAC60 IBM ULTRIUM-HH6 G9P1 1012005888 -1 -1 -1 -1 /dev/nst5 - -
TPAC60 IBM ULTRIUM-HH6 G9P1 1017005888 -1 -1 -1 -1 /dev/nst4 - -
TPAC60 IBM ULTRIUM-HH6 G9P1 1015005888 -1 -1 -1 -1 /dev/nst3 - -
TPAC60 IBM ULTRIUM-HH6 G9P1 1013005888 -1 -1 -1 -1 /dev/nst2 - -
TPAC60 IBM ULTRIUM-HH6 G9P1 1014005888 -1 -1 -1 -1 /dev/nst1 - -
TPAC60 IBM ULTRIUM-HH6 G9P1 1016005888 -1 -1 -1 -1 /dev/nst0 - -


Tried this ....
###########################

Ran nbrbutil -resetMediaServer <media server>
Ran nbrbutil -resetall
Restarted NetBackup daemons on media server
Rebooted Media/Master Server
Rebooted Tape Library
Did not find any problems with Tape Library

Thanks,

Krishma.

0 Kudos
50 REPLIES 50

Krishma
Level 4

‎07-24-2017 12:27 PM

Processes currently running are :

[root@veritas daemon]# /usr/openv/netbackup/bin/bpps -x
NB Processes
------------
root 2850 1 0 11:15 ? 00:00:23 /usr/openv/db/bin/NB_dbsrv @/usr/openv/var/global/server.conf @/usr/openv/var/global/databases.conf -hn 8
root 2886 1 0 11:15 ? 00:00:02 /usr/openv/netbackup/bin/nbevtmgr
root 2915 1 0 11:15 ? 00:00:19 /usr/openv/netbackup/bin/nbemm
root 2919 1 0 11:15 ? 00:00:02 /usr/openv/netbackup/bin/nbrb
root 2969 1 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 2974 1 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/bpcompatd
root 3013 1 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/bpdbm
root 3014 3013 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/bpjobd
root 3016 1 0 11:15 ? 00:00:04 /usr/openv/netbackup/bin/nbjm
root 3026 3016 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbjm
root 3030 1 0 11:15 ? 00:00:01 /usr/openv/netbackup/bin/nbpem
root 3046 3030 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem
root 3056 1 0 11:15 ? 00:00:01 /usr/openv/netbackup/bin/nbstserv
root 3080 1 0 11:15 ? 00:00:03 /usr/openv/netbackup/bin/nbrmms
root 3120 1 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/nbsl
root 3144 1 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/nbvault
root 3156 1 0 11:15 ? 00:00:02 /usr/openv/netbackup/bin/nbsvcmon
root 3609 3013 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/bpdbm
root 3615 3609 0 11:15 ? 00:00:00 /usr/openv/netbackup/bin/bpdbm


MM Processes
------------
root 2946 1 0 11:15 ? 00:00:02 /usr/openv/volmgr/bin/ltid
root 2957 1 0 11:15 ? 00:00:00 vmd -v


Shared Symantec Processes
-------------------------
root 2512 1 0 11:14 ? 00:00:00 /opt/VRTSpbx/bin/pbx_exchange

0 Kudos

Krishma
Level 4
In response to Krishma

‎07-24-2017 12:35 PM

Output of scan command :

************************************************************
*********************** SDT_TAPE ************************
*********************** SDT_CHANGER ************************
*********************** SDT_OPTICAL ************************
************************************************************
------------------------------------------------------------
Device Name : "/dev/sg9"
Passthru Name: "/dev/sg9"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "SPECTRA PYTHON 2000"
Vendor ID : "SPECTRA "
Product ID : "PYTHON "
Product Rev: "2000"
Serial Number: "9111005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "SPECTRA PYTHON 9111005888"
Device Type : SDT_CHANGER
NetBackup Robot Type: 8
Removable : Yes
Device Supports: SCSI-3
Number of Drives : 6
Number of Slots : 120
Number of Media Access Ports: 8
Drive 1 Serial Number : "1012005888"
Drive 2 Serial Number : "1013005888"
Drive 3 Serial Number : "1014005888"
Drive 4 Serial Number : "1015005888"
Drive 5 Serial Number : "1016005888"
Drive 6 Serial Number : "1017005888"
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst5"
Passthru Name: "/dev/sg8"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1012005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1012005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst4"
Passthru Name: "/dev/sg7"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1014005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1014005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst3"
Passthru Name: "/dev/sg6"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1016005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1016005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst2"
Passthru Name: "/dev/sg5"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1017005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1017005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst1"
Passthru Name: "/dev/sg4"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1015005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1015005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst0"
Passthru Name: "/dev/sg2"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1013005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1013005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0

0 Kudos

Thiago_Ribeiro
Moderator
Moderator
Partner    VIP    Accredited

‎07-24-2017 12:36 PM

Hi,

Can you check this TN?

https://www.veritas.com/support/en_US/article.TECH70502

 

 

Regards,

 

Thiago

0 Kudos

Krishma
Level 4
In response to Thiago_Ribeiro

‎07-24-2017 12:37 PM

i already tried that . It did not fix the problem.

0 Kudos

Thiago_Ribeiro
Moderator
Moderator
Partner    VIP    Accredited
In response to Krishma

‎07-24-2017 12:54 PM

Hi,

1. Tape library is directly connected to the media server via SAN / SCSI?
2. OS is able to see the library and the drives?
3. Are you able to use robtest command?
4. Is this media server registered in the NetBackup EMM database using nbemmcmd command?


Did you check this TN?

https://www.veritas.com/support/en_US/article.TECH169477 - Troubleshooting Robot or Drive Issues in NetBackup

 

Regards,

 

Thiago

Marianne
Level 6
Partner    VIP    Accredited Certified
In response to Krishma

‎07-24-2017 08:56 PM

There are MM Processes missing:
avrd, tldcd, tldd.
Have you tried to stop and restart NBU?

Stop NBU and PBX.
Check that all processes have stopped.

Add VERBOSE into /usr/openv/volmgr/vm.conf.
Create daemon, reqlib and robots log folders under volmgr/debug if they do not exist already.
Check for any lock or lck files in volmgr/misc and delete them.

Start PBX and NBU.

Check MM processes to see if the missing processes are now started.
Check debug logs if not.

Handy NetBackup Links

Krishma
Level 4
In response to Marianne

‎07-31-2017 05:55 AM

Here is the update :

I found that Media server is not active for tape devices and veritas netbackup tape drives won't stay up


[root@veritas bin]# /usr/openv/volmgr/bin/vmoprcmd

HOST STATUS
Host Name Version Host Status
========================================= ======= ===========
veritas 656000 ACTIVE-DISK

PENDING REQUESTS


<NONE>

DRIVE STATUS

-------------------------------------------------------

[root@veritas bin]# /usr/openv/netbackup/bin/admincmd/nbemmcmd -updatehost -machinename veritas.pmhadley.com -machinetype media -machinestateop set_tape_active -masterserver veritas
NBEMMCMD, Version:6.5.6
Command completed successfully.

---------------------------------------------------------


[root@veritas bin]# /usr/openv/volmgr/bin/vmoprcmd

HOST STATUS
Host Name Version Host Status
========================================= ======= ===========
veritas 656000 ACTIVE

PENDING REQUESTS


<NONE>

DRIVE STATUS

Drive Name Label Ready RecMID ExtMID Wr.Enbl. Type
Host DrivePath Status
=============================================================================
IBM.ULTRIUM-HH6.000 No No No dlt3-Clean
veritas /dev/nst2 DOWN

IBM.ULTRIUM-HH6.001 No No No dlt3
veritas /dev/nst5 ACTIVE

IBM.ULTRIUM-HH6.002 No No No dlt3
veritas /dev/nst1 ACTIVE

IBM.ULTRIUM-HH6.003 No No No dlt3
veritas /dev/nst3 ACTIVE

IBM.ULTRIUM-HH6.004 No No No dlt3
veritas /dev/nst4 ACTIVE

IBM.ULTRIUM-HH6.005 No No No dlt3
veritas /dev/nst0 ACTIVE
--------------------------------------------------------------------------
[root@veritas bin]# /usr/openv/volmgr/bin/vmoprcmd -up 0
oprd returned abnormal status (96)
IPC Error: Daemon may not be running

---------------------------------------------------------------------------

Robtest commands that can be used to test the SCSI functionality of a robot

[root@veritas ~]# /usr/openv/volmgr/bin/robtest
Configured robots with local control supporting test utilities:
TLD(0) robotic path = /dev/sg6

Robot Selection
---------------
1) TLD 0
2) none/quit
Enter choice: 1

Robot selected: TLD(0) robotic path = /dev/sg6

Invoking robotic test utility:
/usr/openv/volmgr/bin/tldtest -rn 0 -r /dev/sg6

Opening /dev/sg6
MODE_SENSE complete
Enter tld commands (? returns help information)
s d
read_element_status ioctl() failed: Success

 What does this failed message means in robtest ? Nothing happens after that. 

 

 

 

0 Kudos

Krishma
Level 4
In response to Krishma

‎07-31-2017 05:56 AM

Yes , I stooped and restarted NBU .. Here is the result ... 

[root@veritas ~]# /usr/openv/netbackup/bin/bpps -x
NB Processes
------------
root 3101 1 9 08:12 ? 00:00:00 /usr/openv/db/bin/NB_dbsrv @/usr/openv/var/global/server.conf @/usr/openv/var/global/databases.conf -hn 8
root 3139 1 3 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbevtmgr
root 3172 1 9 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbemm
root 3179 1 2 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbrb
root 3204 1 0 08:12 pts/1 00:00:00 /usr/openv/netbackup/bin/bprd
root 3212 1 0 08:12 ? 00:00:00 /usr/openv/netbackup/bin/bpcompatd
root 3217 1 0 08:12 pts/1 00:00:00 /usr/openv/netbackup/bin/bpdbm
root 3218 3217 2 08:12 pts/1 00:00:00 /usr/openv/netbackup/bin/bpjobd
root 3222 1 1 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbjm
root 3236 1 11 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbpem
root 3266 3236 8 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem
root 3285 1 3 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbstserv
root 3306 1 4 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbrmms
root 3365 1 1 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbsl
root 3392 1 2 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbvault
root 3471 1 2 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbsvcmon
root 3553 3222 2 08:12 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbjm


MM Processes
------------
root 3190 1 0 08:12 pts/1 00:00:00 /usr/openv/volmgr/bin/ltid
root 3202 1 0 08:12 pts/1 00:00:00 vmd -v


Shared Symantec Processes
-------------------------
root 3040 1 0 08:11 ? 00:00:00 /opt/VRTSpbx/bin/pbx_exchange
[root@veritas ~]# /usr/openv/volmgr/bin/vmoprcmd

 

 

0 Kudos

Krishma
Level 4
In response to Krishma

‎07-31-2017 05:57 AM

This is a very good point 

There are MM Processes missing:
avrd, tldcd, tldd.

 

How do i start them ?

0 Kudos

quebek
Moderator
Moderator
   VIP    Certified
In response to Krishma

‎07-31-2017 06:05 AM

Hey

this output

Opening /dev/sg6
MODE_SENSE complete
Enter tld commands (? returns help information)
s d
read_element_status ioctl() failed: Success

is kinda uncool - is your special file /dev/sg6 still valid? if so after this s d you should see the drive status and there is nothing as you did notice...

can you please run /usr/openv/volmgr/bin/scan and check if robot and drives are visible there?

Marianne
Level 6
Partner    VIP    Accredited Certified
In response to Krishma

‎07-31-2017 06:11 AM

ltid is supposed to start the other Media Manager processes as part of NBU startup.

When you restart NBU, you need to carefully follow these steps (as per my previous post) :

Stop NBU and PBX.
Check that all processes have stopped.

Add VERBOSE into /usr/openv/volmgr/vm.conf.
Create daemon, reqlib and robots log folders under volmgr/debug if they do not exist already.
Check for any lock or lck files in volmgr/misc and delete them. 

(Confirm all of the above while NBU is down. )


Start PBX and NBU. 

Check MM processes to see if the missing processes are now started.
Check debug logs if not.


Handy NetBackup Links
0 Kudos

Marianne
Level 6
Partner    VIP    Accredited Certified
In response to Marianne

‎07-31-2017 06:27 AM

Some more from side (battling to catch up as this post is already a week old).

I think @quebek is on the right tract.
It seems that your device paths have changed.

scan sees the robot path as:
Device Name : "/dev/sg9"
Passthru Name: "/dev/sg9"

Robot config in NBU has robot 0 - TLD - - - - /dev/sg6 as device path.

scan reports this for /dev/sg6:

Device Name : "/dev/nst3"
Passthru Name: "/dev/sg6"
....

We see this for all the drives:
NetBackup Drive Type: Not Found(5)

You need to find what exactly has changed between OS and the robot/tape drives.

What does the OS see with this command?

cat /proc/scsi/scsi

 

 


Handy NetBackup Links
0 Kudos

Krishma
Level 4
In response to Marianne

‎07-31-2017 06:43 AM

For restarting NBU , i have followed same steps before but that did not help .
I can do it again if you want .

Not sure how device paths have changed.

I also noticed that Type of Device IBM.ULTRIUM-HH6.000 (the drive which is down) is dlt3-Clean and not "dlt3" .

Here is the output of /proc/scsi/scsi ----

[root@veritas ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 32 Lun: 00
Vendor: DP Model: BACKPLANE Rev: 1.07
Type: Enclosure ANSI SCSI revision: 05
Host: scsi0 Channel: 02 Id: 00 Lun: 00
Vendor: DELL Model: PERC H700 Rev: 2.10
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 00 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: PLDS Model: DVD+-RW DS-8A4S Rev: JD51
Type: CD-ROM ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 01 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 01 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi1 Channel: 00 Id: 02 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 02 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1

How do i check if special file /dev/sg6 still valid ?

Thank you for looking into this.

0 Kudos

Krishma
Level 4
In response to Krishma

‎07-31-2017 06:43 AM

[root@veritas ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 32 Lun: 00
Vendor: DP Model: BACKPLANE Rev: 1.07
Type: Enclosure ANSI SCSI revision: 05
Host: scsi0 Channel: 02 Id: 00 Lun: 00
Vendor: DELL Model: PERC H700 Rev: 2.10
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 00 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: PLDS Model: DVD+-RW DS-8A4S Rev: JD51
Type: CD-ROM ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 01 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 01 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi1 Channel: 00 Id: 02 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 02 Lun: 00
Vendor: IBM Model: ULTRIUM-HH6 Rev: G9P1
Type: Sequential-Access ANSI SCSI revision: 06
Host: scsi2 Channel: 00 Id: 02 Lun: 01
Vendor: SPECTRA Model: PYTHON Rev: 2000
Type: Medium Changer ANSI SCSI revision: 03
[root@veritas ~]#

0 Kudos

Krishma
Level 4
In response to Krishma

‎07-31-2017 06:48 AM

[root@veritas ~]# /usr/openv/volmgr/bin/scan
************************************************************
*********************** SDT_TAPE ************************
*********************** SDT_CHANGER ************************
*********************** SDT_OPTICAL ************************
************************************************************
------------------------------------------------------------
Device Name : "/dev/sg9"
Passthru Name: "/dev/sg9"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "SPECTRA PYTHON 2000"
Vendor ID : "SPECTRA "
Product ID : "PYTHON "
Product Rev: "2000"
Serial Number: "9111005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "SPECTRA PYTHON 9111005888"
Device Type : SDT_CHANGER
NetBackup Robot Type: 8
Removable : Yes
Device Supports: SCSI-3
Number of Drives : 6
Number of Slots : 120
Number of Media Access Ports: 8
Drive 1 Serial Number : "1012005888"
Drive 2 Serial Number : "1013005888"
Drive 3 Serial Number : "1014005888"
Drive 4 Serial Number : "1015005888"
Drive 5 Serial Number : "1016005888"
Drive 6 Serial Number : "1017005888"
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst5"
Passthru Name: "/dev/sg8"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1012005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1012005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst4"
Passthru Name: "/dev/sg7"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1015005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1015005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst3"
Passthru Name: "/dev/sg6"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1016005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1016005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst2"
Passthru Name: "/dev/sg5"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1014005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1014005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst1"
Passthru Name: "/dev/sg4"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1017005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1017005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name : "/dev/nst0"
Passthru Name: "/dev/sg2"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "IBM ULTRIUM-HH6 G9P1"
Vendor ID : "IBM "
Product ID : "ULTRIUM-HH6 "
Product Rev: "G9P1"
Serial Number: "1013005888"
WWN : ""
WWN Id Type : 0
Device Identifier: "IBM ULTRIUM-HH6 1013005888"
Device Type : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable : Yes
Device Supports: SCSI-6
Flags : 0x0
Reason: 0x0

0 Kudos

quebek
Moderator
Moderator
   VIP    Certified
In response to Krishma

‎07-31-2017 06:57 AM

Yes your device files has changed...

Now robot is reachable on 

Device Name : "/dev/sg9"
Passthru Name: "/dev/sg9"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry : "SPECTRA PYTHON 2000"
Vendor ID : "SPECTRA "
Product ID : "PYTHON "
Product Rev: "2000"
Serial Number: "9111005888"

 

while yoru NBU thinks its on sg6 - please remove all devices from this media server, rerun NBU device configuration guide...

good luck

Krishma
Level 4
In response to quebek

‎07-31-2017 07:00 AM

[root@veritas robots]# grep -i "ERROR" log.073117
07:56:49.497 [892] <16> emmlib_GetHost: (0) QueryMachine failed, emmError = 4005006, nbError = 0
07:56:49.497 [892] <16> BuildRobList: (-) Translating EMM_ERROR_DBServerDown(4005006) to 328 in the device management context
08:12:59.258 [3691] <2> emmlib_GetHost: (0) QueryMachine successful, emmError = 0, nbError = 0
08:23:08.865 [4008] <2> emmlib_GetHost: (0) QueryMachine successful, emmError = 0, nbError = 0


Last week : did not see it today yet

[root@veritas reqlib]# tail -200 log.072717
12:00:23.195 [25747] <2> getrequestack: server response: EXIT_STATUS 196
12:00:23.195 [25747] <16> vmdb_start_oprd: unable to send request to server: EMM database error (196)
12:00:23.195 [25747] <16> vmdb_start_oprd: vmdb_start_oprd request status: EMM database error (196)
[root@veritas reqlib]#

 

[root@veritas ltid]# grep -i error log.073117
08:04:37.998 [1771] <2> emmlib_GetHost: (0) QueryMachine successful, emmError = 0, nbError = 0
08:04:38.019 [1771] <2> emmlib_GetMachineAliasList: (0) GetMachineAliasList successful, emmError = 0, nbError = 0
08:04:42.188 [1771] <16> emmlib_GetDeviceList: (0) queryEmmDeviceConf failed, emmError = 2007004, nbError = 0
08:04:42.188 [1771] <16> InitLtidDeviceInfo: (-) Translating EMM_ERROR_SQLConnectionException(2007004) to 328 in the device management context
08:04:42.191 [1771] <16> emmlib_UpdateMachineState: (0) updateMachineState failed, Error < 0x3D1C8E >
08:04:42.191 [1771] <16> TermLtid: (-) Translating EMM_ERROR_DBServerDown(4005006) to 328 in the device management context
08:04:42.191 [1771] <16> TermLtid: Could not update machine state, error code = 328
08:07:50.109 [2479] <16> emmlib_GetHost: (0) QueryMachine failed, emmError = 4005006, nbError = 0
08:07:50.109 [2479] <16> AddAndVerifyHost: Failed to validate the existence of host veritas with EMM. EMM error code = 4005006
08:07:50.112 [2479] <16> emmlib_GetExtFileVersion: (0) queryExtFileVersion failed, emmError = 4005006, nbError = 0
08:07:50.112 [2479] <16> InitLtidEMM: (-) Translating EMM_ERROR_DBServerDown(4005006) to 328 in the device management context
08:08:08.137 [2479] <16> emmlib_UpdateMachineState: (0) updateMachineState failed, Error < 0x3D1C8E >
08:08:08.137 [2479] <16> InitLtidEMM: (-) Translating EMM_ERROR_DBServerDown(4005006) to 328 in the device management context
08:12:14.750 [3190] <2> emmlib_GetHost: (0) QueryMachine successful, emmError = 0, nbError = 0
08:12:14.769 [3190] <2> emmlib_GetMachineAliasList: (0) GetMachineAliasList successful, emmError = 0, nbError = 0
[root@veritas ltid]#


[root@veritas ltid]# grep ltid /var/log/messages
Jul 31 08:12:15 veritas ltid[3190]: ltid can not be started while resources are assigned to the host.

0 Kudos

Krishma
Level 4
In response to quebek

‎07-31-2017 07:03 AM

How did it change ? We have not done any activity on this system . We have rebooted Tape Library and this server (Media/Master server ) . Will that change the configuration ?

0 Kudos

Marianne
Level 6
Partner    VIP    Accredited Certified
In response to Krishma

‎07-31-2017 07:10 AM

You have not told us how the robot and tape drives are connected - SCSI or FC?

With FC, it is quite possible that a reboot will change OS device paths if Persistent Binding is not in place between FC hba and the OS.
Seems this could be causing the EMM (196) error.
(See https://vox.veritas.com/t5/Articles/Troubleshooting-EMM-Error-196/ta-p/810462 )

Your best best right now is to delete all existing devices in NBU config, restart NBU, then run NBU Device Config Wizard.

Let us know the result. 


Handy NetBackup Links