Forum Discussion

H_Sharma's avatar
H_Sharma
Level 6
10 years ago

Drive Issue

Hello Experts,

We have a drive it's control is showing down-tld. However, its writing fine. Its SSO. Few days back we had power cycled our library and we can observe

this thing since then.

Master 7.6.0.1 Windows 2008 R2

Media 7.6.0.1 Solaris 11 

We have 2 media server and 1 master.  Its happening for only one drive.

It i click on it, it shows below. 

1:-Drive 000  device host master and control TLD
2:-Drive 000 device host media server 1  and conrol Scan-TLD
3:-Drive 000 device host  media server 2 and control Active. Its writing with media server 2 as of now.

If i double click on it i could see Drive control as DOWN-TLD

I did the below troubleshooting.

1:- SCAN can see all the drives on master and 2 media servers.
2:- Vmoprcmd -d ds all the drive and up on master and media.
3:- Tpconfig -ld all the drives are up on master and both the media servers.
4:- Master server device manager can sense all the 8 drives.
5:- Media server OS cfgadm -al can see all the 8 drives.

Unable to understand what to do next as everything seems fine. If its showing downed state. I beleive somewhere, something is wrong. Pls help.

7.6
Backup and Recovery
NetBackup
Troubleshooting
Windows Server (2003-2008)
  • H_Sharma's avatar
    H_Sharma
    10 years ago

    Hi Marianne,

    Actually we have servers in cluster and both have their different issue.

    So this issue started with one drive. TSE restarted the Ltid services on this but after that this issue got resolved for this drive but issue came up on other drives. Any of the Drives were taking time to pick up the media however other backups were in queue so it was intermittently sometimes all drives were running fine as if nothing had happened to these.

     

    1:- TSE told to change the below changes


    C:\Program Files\Veritas\NetBackup\bin\admincmd>nbrbutil -changesettings RB_RESPECT_REQUEST_PRIORITY=true
    c:\Program Files\Veritas\NetBackup\bin\admincmd>nbrbutil -changesettings RB_BREAK_EVAL_ON_DEMAND=false

    It didnot work

    2 we have removed stale SERVER entries in Host Properties.

    It didnot work

    3:- We had updated the Firmware of drives.

    It didnot work as all drives were running on other cluser server.

    4:- We restarted netbackup services on problematic server.

    It worked.
     

15 Replies

Sort By
  • H_Sharma's avatar
    H_Sharma
    Level 6
    10 years ago

    Ok i tried to scan the host in device configuration and found master server could see 6 tapes drives path and media server could see 8 tapes drive and 1 robot. 

    and if i selected only master server and then auto detection is showing detected 4 drive paths and 1 robot.

    We have storage unit attached in master server as well. 

    Is this the problem? We have 8 drives. PFA

  • sdo's avatar
    sdo
    Moderator
    10 years ago

    Looks like you've found the problem.

    Another 'query/report/list' type command (which does not update anything) that you could try, on each master and media server is:

    tpautoconf -report_disc

    ...which is a report of device configuration dicsrepencies between what is configured and what is visible - essentially, you can think of it as reporting the difference between what the NetBackup (volmgr) 'scan' command can see (i.e. what the OS can see), and the device configuration(s) stored within EMM.

    One thing to remember is that... if a discrepency is reported then there could be at least four (possibly more) reasons for this:

    1) The device has disappeared from the OS, and thus 'scan' can no longer see it, but the device config is correct.

    ...or:

    2) The device is visible to the OS and to the 'scan' command, but missing (inadvertently or maybe even deliberately) from the EMM device configuration.

    ...or:

    3) Another typical cause, is that one (or more) tape drives have been replaced, but the EMM device configuration is out of step (for one or more NetBackup master/media servers) and so the tape drive serial number is out of step, and so NetBackup cannot 'serialize' (and manage one or more of the paths), and so logically downs the drive path that NetBackup thinks that it can no longer see.

    ...or:

    4) Drives have been added but when the 'Device Configuration Wizard' was last run, it didn't complete properly for one of the master or media servers, and so the EMM device configurations did not get populated correctly with the device (tape drive) paths.

    .

    Can I suggest that you run the tpautoconf -report_disc command above on all NetBackup master and media servers in that NetBackup domain - and see what you think is hapenning or has hapenned (in the past)?

  • sdo's avatar
    sdo
    Moderator
    10 years ago

    N.B:  Be careful with the 'tpautoconf' command.  Some variants of CLI switches and arguments on this command will make changes to EMM device configuration database.

    FYI - 'tpautoconf -report_disc' (on its own) does not make any updates, and is simply a query type command.

  • H_Sharma's avatar
    H_Sharma
    Level 6
    10 years ago

    Hi Sdo,

    I ran a command on master and could see 5 missing devices on master server in tpautoconf output.Drive 2,3,4,5,7 and if i run command on media servers. I didnot get any output on both the server. PFA

  • sdo's avatar
    sdo
    Moderator
    10 years ago

    IMO - I would look at running the NetBackup Device Configuration wizard, to re-create the EMM device configurations...

    ...but you may NOT want to do this... IF any part of your current device configuration was previously crafted by hand (using manual tpconfig commands)... BECAUSE device names and storage unit names may change (to NetBackup defaults).

    Having said that, not too many sites ever configure by hand, and usually (typically) smaller environments of just a few NetBackup Servers and a few tape drives will have previously used the device configuration wizard, in which case you should be ok to run it again - as long as your environment does not have any other issues which might impair the capabilities of the Device Configuration Wizard.

    N.B:  I am not a Symantec Support employee... so take any advice I may ever give with a pinch of salt, and consider your options carefully.  If you are fairly new to NetBackup, and are not too comfortable with understanding 'what could go wrong'... then can I suggest that you open a case with Symantec Support.

  • sdo's avatar
    sdo
    Moderator
    10 years ago

    If you are able/allowed/willing to show the discrepencies from the master, I (and/or others) might be able to help a bit more.

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago
    I would like to see output of these commands on the master server : tpconfig -l scan (both in volmgr\bin ) Please copy the text and post here in text format, rather than screenshots. If master has lost connectivity to some of the tape drives, there should be errors in Event Viewer System log along with indication of the cause.
  • jim_dalton's avatar
    jim_dalton
    Level 6
    10 years ago

    Start at the beginning...can all hosts see all drives, all correctly configured ? Thats the only sensible starting point. Using one tape that you dont care about and one drive at a time, put the tape in each drive and test you can read and write from the OS.Go through each drive on each OS Once thats done, the rest is trivial. IMO. Jim 

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago

    I have been wondering about this issue...

    Same environment where you had an issue with devices on clustered master server after failover? 
    You never replied to attempts to help over here:
     https://www-secure.symantec.com/connect/forums/cluster-failover-0 

    You may also want to go through the basic concepts of SSO again. Some URLs were shared here:
    https://www-secure.symantec.com/connect/forums/control-host 

    SSO is also covered in NetBackup Administrator's Guide, Volume II   

  • H_Sharma's avatar
    H_Sharma
    Level 6
    10 years ago

    Hi,

    Sorry for the delyed response,

    We have been experiening very stratnge issue. First we had a problem with one drive now 2-3 more drives are affected due to this.

    These drives are taking too much time to pick up the media. sometimes backups are in queued state and drive is free. Sometimes all the drives are working as if there is no issue.

    We had raised a ticket with symantec TSE and they had recommended to remove the drives and configure again also raised a ticket with Oracle (Library vendor) for such behaviour. Issue is still persistent.