Forum Discussion

MickBoosh's avatar
MickBoosh
Level 5
12 years ago

Disk storage server is down(2106)

Hi,

I've started getting the Disk storage server is down(2106) error. I'm having trouble trying to trouble shoot it. We have a staging area attached via fibre. There ar no windows event log errors for the time it went down. 

I've looked in bpdm and bptm but I can't find anything obvious in there.

Windows 2003 R2 x86

NBU 7.5.0.3

 

02/02/2013 13:08:23 - Info nbjm(pid=5848) starting backup job (jobid=4311605) for client XXXXXXXn, policy SQL_Backup_TransactionLogsNew, schedule Default-Application-Backup  
02/02/2013 13:08:23 - Info nbjm(pid=5848) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=4311605, request id:{B0C094C4-7A4A-44A9-B021-A983E41A9522})  
02/02/2013 13:08:23 - requesting resource Disk_Staging
02/02/2013 13:08:23 - requesting resource [MASTER].NBU_CLIENT.MAXJOBS.xxxxxxxxxn
02/02/2013 13:08:23 - requesting resource [MASTER].NBU_POLICY.MAXJOBS.SQL_Backup_TransactionLogsNew
02/02/2013 13:08:23 - awaiting resource Disk_Staging - Maximum job count has been reached for the storage unit
02/02/2013 13:10:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:14:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:18:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:20:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:24:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:26:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:30:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:32:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:34:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:36:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:38:23 - Info nbjm(pid=5848) Waiting in NetBackup scheduler work queue on server xxxx    
02/02/2013 13:41:04 - Error nbjm(pid=5848) NBU status: 2106, EMM status: Storage Server is down or unavailable  
Disk storage server is down(2106)
7.5
Backup and Recovery
NetBackup
  • Mark_Solutions's avatar
    Mark_Solutions
    12 years ago

    This may be a timeout caused, as you say, by the servers being very busy.

    It may be a lack of network ports or it may just be a communications timeout whilst checking the disks.

    Add the following file (just a flat file with no file extention - so caraful that you are viewing extentions and it doesnt get created as a text file) to the Master and Media Servers:

    <install path>\veritas\netbackup\db\config\DPS_PROXYDEFAULTRECVTMO

    Open it up in notepad and put the value of 800 in it.

    This just gives disk status checks a better timeout value which usually helps when things are very busy

  • RamNagalla's avatar
    RamNagalla
    Moderator
    12 years ago

    hi,

    what kind of storage server you are using?

    Pure disk or Data domain?

    let us know the output of below file

    Install path\veritas\netbackup\bin\nbdevquery -listdp -stype <storagetype -U

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Please tell us more about your Disk_Staging storage server.

    32-bit W2003? This means Advanced Disk, right? Not Basic Disk?
    Configured on Master or on a media server?

    Extract from Troubleshooter:

    Verify that all media servers configured for the storage server can communicate with the storage server. The bpstsinfo command queries the storage server periodically, so you can use the bpstsinfo log on the media server.
     
    Please create bpstsinfo folder under .../netbackup/logs
     
    Disclaimer: I cannot find any information on whether bpstsinfo is a legacy or unified log. I am hoping that creating legacy type log folder will work.
     
     
  • MickBoosh's avatar
    MickBoosh
    Level 5
    12 years ago

    I'm not sure we are using a storage server, I've never heard of this before. It's just a basic disk. That command says no disk pools found. Confused!

  • RamNagalla's avatar
    RamNagalla
    Moderator
    12 years ago

    please provide the nbemmcmd -listhost -verobose  for the media server which is in issue.

    and also, if that is the basic disk,

    just log into the server , go to the baisc disk directory and try to create one text file, and see how it works.

    if creating text file fails, than you need to work with your windows again, and storage team to fix that.

     

  • MickBoosh's avatar
    MickBoosh
    Level 5
    12 years ago

    It is intermittent I must add. The errors started on the 02/02/2013 and only lasted an hour. No errrors since then, but we have had them for a few weeks intermittently. It's configured on a master server.

     

    NBEMMCMD, Version: 7.5.0.3
    The following hosts were found:
    [master]
            MachineName = "[master]"
            FQName = "[master].[domain]"
            MachineDescription = ""
            MachineNbuType = server (6)
    [master]
            ClusterName = ""
            MachineName = "[master]"
            FQName = "[master].[domain]"
            GlobalDriveSeed = "VEND:#.:PROD:#.:IDX"
            LocalDriveSeed = ""
            MachineDescription = ""
            MachineFlags = 0xb7
            MachineNbuType = master (3)
            MachineState = active for tape and disk jobs (14)
            NetBackupVersion = 7.5.0.3 (750300)
            OperatingSystem = windows (11)
            ScanAbility = 5
    stasbkp
            ClusterName = ""
            MachineName = "[media]"
            FQName = "[media].[domain]"
            LocalDriveSeed = ""
            MachineDescription = ""
            MachineFlags = 0x17
            MachineNbuType = media (1)
            MachineState = active for tape and disk jobs (14)
            MasterServerName = "stas75"
            NetBackupVersion = 7.5.0.3 (750300)
            OperatingSystem = windows (11)
            ScanAbility = 5
    Command completed successfully.

     

  • RamNagalla's avatar
    RamNagalla
    Moderator
    12 years ago

    hi,

    if its working fine now, you are good to go..

    if you get this error in future, its better to practice, to start from the OS disk level,as its basic disk.

    check if the disk is accssible from the OS level or not, 

    then try to check the  the disk status. 

    make sure your disk is not reaching High disk I/O performance. 

  • MickBoosh's avatar
    MickBoosh
    Level 5
    12 years ago

    It is working now yes, but as I've had this problem for the last two weeks, it will happen again. It usually happens through the night so I will be unable to check.

    I have suspected it is when the disk is being highly utilised actually.

    Thanks for your help

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    This may be a timeout caused, as you say, by the servers being very busy.

    It may be a lack of network ports or it may just be a communications timeout whilst checking the disks.

    Add the following file (just a flat file with no file extention - so caraful that you are viewing extentions and it doesnt get created as a text file) to the Master and Media Servers:

    <install path>\veritas\netbackup\db\config\DPS_PROXYDEFAULTRECVTMO

    Open it up in notepad and put the value of 800 in it.

    This just gives disk status checks a better timeout value which usually helps when things are very busy