cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted

SAN Backup failed - backup active with error robot operation failed

Hi fellow friends;

I have issue with SAN backup.

Component:

  1. Master Server: Windows 2008R2 Server
  2. Media Server: IBM AIX 7.1
  3. Tape Library: Quantum i80, 4 units tape drives (LTO5)
  4. SAN Switch (Brocade)
  5. Veritas Netbackup v7.1

Issue:

When run backup 'policy' for Media server (IBM AIX 7.1), media keep mounting forever then got robot operation failed (status from activity monitor). This policy configure to use storage unit define to backup through SAN network.

If run backup for other client & using different storage unit, backup running OK and normally ended up status=0,  completed.

In summary: can conclude backup over SAN not OK while backup through LAN network is OK.

Appreciated advice/comments/recommendation. Thank you in Advance..

 

 

 

1 Solution

Accepted Solutions
Highlighted
Accepted Solution!

Hi Everyone, Manage to solve

Hi Everyone,

Manage to solve this issue after server been force to shut down (power outage) and rescan tape drives when reconfigure storage devices. Basically OS have some configuration mismatch with tape drives and manage to re-detect tape drives after server boot-up.

Hi Marianne,

I could not find 'mark solution' link

View solution in original post

43 Replies
Highlighted

hi, I just want to make

hi,

I just want to make sure

are you trying to use the FT media server or SAN media server?

 

Highlighted

Hi Nagalla, Thanks for quick

Hi Nagalla,

Thanks for quick reply.

Not sure how to answer this (sorry), but I guess I have Media Server. How to verify whether FT or SAN media server ?

 

Highlighted

sorry I guess I have SAN

sorry I guess I have SAN Media Server configured.

Highlighted

Are tape drives shared? Do

Are tape drives shared? Do you have SSO license added on master and media server? Did you use the Device Config wizard on the master to config devices for master and media server? Please post output from media server: /usr/openv/volmgr/bin/scan /usr/openv/volmgr/bin/tpconfig -l /usr/openv/volmgr/bin/vmoprcmd -d Ensure logging on media server is enabled as follows: bptm log folder exists in /usr/openv/netbackup/logs VERBOSE entry in /usr/openv/volmgr/vm.conf (followed by NBU restart) Ensure syslog is enabled at OS level.
Highlighted

Hi Marianne, Tape drives are

Hi Marianne,

Tape drives are shared. SSO license both install on  master & media.

Let me check on the device config. Below media server output as requested.

scan

oot@ibmdb1Smiley Embarassed./scan
************************************************************
*********************** SDT_TAPE    ************************
*********************** SDT_CHANGER ************************
************************************************************

------------------------------------------------------------
Device Name  : "/dev/rmt0.1"
Passthru Name: "/dev/rmt0.1"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry    : "IBM     DDS Gen7        VS11"
Vendor ID  : "IBM     "
Product ID : "DDS Gen7        "
Product Rev: "VS11"
Serial Number: "20150146"
WWN          : ""
WWN Id Type  : 0
Device Identifier: ""
Device Type    : SDT_TAPE
NetBackup Drive Type: Not Found(5)
Removable      : Yes
Device Supports: SCSI-3
Flags : 0x0
Reason: 0x0

 

tpconfig -l

root@ibmdb1Smiley Embarassed./tpconfig -l
Device Robot Drive       Robot                    Drive                 Device
Type     Num Index  Type DrNum Status  Comment    Name                  Path
robot      0    -    TLD    -       -  -          -                     backupserver
  drive    -    1 hcart2    4      UP  -          HP.ULTRIUM5-SCSI.002  /dev/rmt1.1
  drive    -    2 hcart2    3      UP  -          HP.ULTRIUM5-SCSI.003  /dev/rmt2.1
  drive    -    3 hcart2    2      UP  -          HP.ULTRIUM5-SCSI.000  /dev/rmt3.1
  drive    -    4 hcart2    1      UP  -          HP.ULTRIUM5-SCSI.001  /dev/rmt4.1
drive      -    0    pcd    -  DISABL  -          IBM.DDSGEN7.000       /dev/rmt0.1

 

vmoprcmd

root@ibmdb1Smiley Embarassed./vmoprcmd -d

                                PENDING REQUESTS

                                     <NONE>

                                  DRIVE STATUS

Drv Type   Control  User      Label  RecMID  ExtMID  Ready   Wr.Enbl.  ReqId
  1 hcart2   TLD                -                     No       -         0
  2 hcart2   TLD                -                     No       -         0
  3 hcart2   TLD                -                     No       -         0
  4 hcart2   TLD                -                     No       -         0

                             ADDITIONAL DRIVE STATUS

Drv DriveName            Shared    Assigned        Comment
  1 HP.ULTRIUM5-SCSI.002  Yes      -
  2 HP.ULTRIUM5-SCSI.003  Yes      -
  3 HP.ULTRIUM5-SCSI.000  Yes      -
  4 HP.ULTRIUM5-SCSI.001  Yes      -
root@ibmdb1Smiley Embarassed

 

Ok vm.conf have already add VERBOSE.

Over to you Marianne..thanks.

 

Highlighted

Seems 'scan' can only see the

Seems 'scan' can only see the internal tape drive?

 

Device Name  : "/dev/rmt0.1"
Passthru Name: "/dev/rmt0.1"
...
Inquiry    : "IBM     DDS Gen7        VS11"

drive      -    0    pcd    -  DISABL  -          IBM.DDSGEN7.000       /dev/rmt0.1

Please show us output of:

lsdev -C -c tape

Highlighted

Hi Marianne, yes indeed, OS

Hi Marianne,

yes indeed, OS (media Server, IBM AIX) only detect internal tape drive which is not use for backup.

lsdev -Cc tape output;

root@ibmdb1Smiley Embarassedlsdev -Cc tape
rmt0 Available 00-08-00 SAS 4mm Tape Drive
rmt1 Available 04-01-02 Other FC SCSI Tape Drive
rmt2 Available 04-01-02 Other FC SCSI Tape Drive
rmt3 Available 04-01-02 Other FC SCSI Tape Drive
rmt4 Available 04-01-02 Other FC SCSI Tape Drive

Over to you Marianne

Highlighted

If run backup for other

If run backup for other client & using different storage unit... different stu under same media server ? Can you try to move media from slot to drive via robtest..if no...what error? Can you post /usr/openv/volmgr/debug/tpreq logs o/p... Thanks, Giri.
Highlighted

Hi Giri, Thanks for reply.

Hi Giri,

Thanks for reply. What I mean is I have define 2 STU.

STU1 = for other client (for normal LAN backup, means client must go through Master Server for backup)

STU2 = for media server (which is for SAN backup - direct backup to Quantum i80 tape library)

Ok for robtest just to let you know, robtest command can nonly run from Master Server (which also as Media Server)

I can't do robtest from my Media Server (the IBM AIX Server)

There is no /tpreq folder in /volmgr/debug in Master but I can run robtest utility, and just now able to ove tape from slot to drive - with no error.

Hope this helps, let me know if you need additional info. Thanks.

 

 

Highlighted

I am curious to know how

I am curious to know how devices were configured if 'scan' does not pick up the drives.

We need to be able to somehow double-check that  /dev/rmt1.1 is indeed position 4 in the robot ,  /dev/rmt4.1 is indeed position 1 in the robot, etc.

Normally this can be easily confirmed with output from scan or 'tpautoconf -t' combined with 'scan -changer' output from the robot control host.

 

PS: Why exactly do you need 2 user id's on Connect? 
Once again you start a discussion as one user and then respond with your other id?

Highlighted

Hi Marianne, Sorry I should

Hi Marianne,

Sorry I should logout from imra2013 & log as Imra_backup. Can I delete my user ID (either one will do..)

I'm also puzzle how to explain this (scan does not pickup drive) when I assigned to this project. It was OK for Master Server but not for Media Server, maybe something was wrong and affect setting to media Server. Because previously we can do backup via SAN for Media Server. Now we can't even do robtest in Media Server (IBM AIX).

Hope this provide some background information (of the issue)

 

 

 

Highlighted

Hi Marianne, I would like to

Hi Marianne,

I would like to use imra2013 ID in Connect from now on.

Somehow can't use Imra_backup ID

Thanks for your reminder.

Sorry for inconvenienced caused.

 

Highlighted

Hi, Any helps ? Many thanks

Hi,

Any helps ?

Many thanks in advance.

 

Highlighted

Hi Marianne, Sorry below

Hi Marianne,

Sorry below output scan -changer (from Master Server)

C:\Program Files\Veritas\Volmgr\bin>scan -changer
************************************************************
*********************** SDT_CHANGER ************************
************************************************************
------------------------------------------------------------
Device Name  : ""
Passthru Name: ""
Volume Header: ""
Port: 3; Bus: 0; Target: 6; LUN: 1
Inquiry    : "QUANTUM Scalar i40-i80  140G"
Vendor ID  : "QUANTUM "
Product ID : "Scalar i40-i80  "
Product Rev: "140G"
Serial Number: "QUANTUMD1H0131319_LLA"
WWN          : ""
WWN Id Type  : 0
Device Identifier: "QUANTUM D1H0131319_LLA          "
Device Type    : SDT_CHANGER
NetBackup Robot Type: 8
Removable      : Yes
Device Supports: SCSI-3
Number of Drives : 4
Number of Slots  : 49
Number of Media Access Ports: 5
Drive 1 Serial Number      : "C38C1BE000"
Drive 2 Serial Number      : "C38C1BE004"
Drive 3 Serial Number      : "C38C1BE008"
Drive 4 Serial Number      : "C38C1BE00C"
Flags : 0x0
Reason: 0x0

 

and tpautoconf -t output

C:\Program Files\Veritas\Volmgr\bin>tpautoconf -t
TPAC60 HP      Ultrium 5-SCSI  Y5AZ C38C1BE004 3 0 4 0 Tape0 -
TPAC60 HP      Ultrium 5-SCSI  Y5AZ C38C1BE000 3 0 6 0 Tape2 -
TPAC60 HP      Ultrium 5-SCSI  Y5AZ C38C1BE00C 3 0 7 0 Tape3 -

Hope this helps.

 

 

 

Highlighted

Maybe I was not clear enough

Maybe I was not clear enough in my previous post:

 

We need to be able to somehow double-check that  /dev/rmt1.1 is indeed position 4 in the robot ,  /dev/rmt4.1 is indeed position 1 in the robot, etc.

Normally this can be easily confirmed with output from scan or 'tpautoconf -t' combined with 'scan -changer' output from the robot control host.

In the last statement I meant 'tpautoconf -t' on the media server.
This output will give us similar output as tpautoconf on the master and we will be able to see serial numbers for device names (rmt1.1, rmt2.1, etc).

We can then compare this with drive position and serial no's in scan output:
Drive 1 Serial Number      : "C38C1BE000"
Drive 2 Serial Number      : "C38C1BE004"
Drive 3 Serial Number      : "C38C1BE008"
Drive 4 Serial Number      : "C38C1BE00C"

which will finally enable us to compare with 'tpconfig -l' output to ensure there are no device mapping mismatches.

Highlighted

Hi Marianne, tpautoconf

Hi Marianne,

tpautoconf output from my Media Server.

root@ibmdb1Smiley Embarassed./tpautoconf -t
TPAC60 IBM     DDS Gen7        VS11 20150146 -1 -1 -1 -1 /dev/rmt0.1 - -

Looks Media Server does not detect LTO5 tape drives ..

 

Highlighted

Hi Marianne, Just to add,

Hi Marianne,

Just to add, have been reading SAN client & Fibre Transport Guide and I think all the while existing Media Server was not utilizing backup through SAN Network. My understanding we need to configure the machine as SAN client then would be able to utilize backup using fibre connectivity which suppose to be more faster than LAN connection. No wonder 900 GB data backup more than 9 hours whereas our LTO tape drives configured point to point at 8GB/s. Because all the while when backup can only wee 'Transport Type' as LAN.

Please correct me if I'm wrong.

Thanks in advance

 

Highlighted

You clearly have media server

You clearly have media server software installed with devices configured for AIX server.

There is a big difference between SAN Media server and SAN Client.

see: http://www.symantec.com/docs/TECH135896 and http://www.symantec.com/docs/TECH53815

Your current issue is with device access on AIX media server. 
You need to get that fixed.

Highlighted

Hi Marianne, Agree. My

Hi Marianne,

Agree.

My current setup is SAN Media Server (since media server software installed in AIX Server)

I have read both articles, looks SAN client require extra setting to configure. Maybe thats why our previous vendor setup our AIX media server as SAN Media Server. Anyway you are write, I need to solve device access on AIX Media Server.

Do you think I should refer to AIX support how to access the devices or Quantum support to provide drivers etc. Please advice.

Thanks in advance.sad