Forum Discussion

Bnj_V's avatar
Bnj_V
Level 3
10 years ago
Solved

Disk Volume is Down: Status 2074 - Can't be UP

Hello,

The error I've encountered is already common. I've read lot of forums and discussions about and I applied the solutions suggested.

But my drives are still down. May I ask for any suggestions for resolution please. (Note:Veritas NBU 6.5) 

Thanks in advance.

==============================================

Job Activity details:

Aug 11, 2015 2:59:55 PM - requesting resource zfs-p1nbu-incremental
Aug 11, 2015 2:59:55 PM - requesting resource PBCKSP01.NBU_CLIENT.MAXJOBS.pnet.ph.ds.amkor.com
Aug 11, 2015 2:59:55 PM - requesting resource PBCKSP01.NBU_POLICY.MAXJOBS.Windows_ATPNET
Aug 11, 2015 2:59:55 PM - Error nbjm (pid=15478) NBU status: 2074, EMM status: Disk volume is down
Disk volume is down  (2074)

===============================================

bperror:

1439276395 1 4 16 PBCKSP01 174136 174136 0 pnet.ph.ds.****.com nbpem backup of client pnet.ph.ds.****.com exited with status 2074 (Disk volume is down)
1439276506 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume disk_windows-amds:Internal_16 is down (not ok on root)  
1439276506 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume disk_windows-barcode:Internal_16 is down (not ok on root)  
1439276506 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume disk_windows-cimapps07:Internal_16 is down (not ok on root)  
1439277889 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume Lotus_Notes_Migration:Internal_16 is down (not ok on root)  
1439277889 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume disk_windows-notes:Internal_16 is down (not ok on root)  
1439280113 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume disk_Windows_Payroll_Full:Internal_16 is down (not ok on root)  
1439281496 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume zfs-p1nbu-incremental:Internal_16 is down (not ok on root)

================================================

/usr/openv/netbackup/bin/admincmd/nbdevquery -listdv -stype BasicDisk -U

Disk Pool Name : BMR-DISK Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaab Total Capacity (GB) : 1000.00 Free Space (GB) : 498.15 Use% : 50 Status : UP Flag : ReadOnWrite Flag : AdminUp Flag : InternalUp Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : BMR-atpxisp01n1 Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaae Total Capacity (GB) : 1000.00 Free Space (GB) : 498.15 Use% : 50 Status : UP Flag : ReadOnWrite Flag : AdminUp Flag : InternalUp Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : bmr_atpxi03 Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaad Total Capacity (GB) : 1000.00 Free Space (GB) : 498.15 Use% : 50 Status : UP Flag : ReadOnWrite Flag : AdminUp Flag : InternalUp Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : BMR-atpxisp01n2 Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaaf Total Capacity (GB) : 1000.00 Free Space (GB) : 498.15 Use% : 50 Status : UP Flag : ReadOnWrite Flag : AdminUp Flag : InternalUp Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : disk_Windows_Payroll_Full Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaam Total Capacity (GB) : 2048.00 Free Space (GB) : 1739.63 Use% : 15 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : StripTest-Disk Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaah Total Capacity (GB) : 1000.00 Free Space (GB) : 498.15 Use% : 50 Status : UP Flag : ReadOnWrite Flag : AdminUp Flag : InternalUp Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : disk_windows-barcode Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaai Total Capacity (GB) : 2048.00 Free Space (GB) : 1739.63 Use% : 15 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : disk_windows-cimapps07 Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaaj Total Capacity (GB) : 2048.00 Free Space (GB) : 1739.63 Use% : 15 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : disk_windows-cimapps04 Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaak Total Capacity (GB) : 2048.00 Free Space (GB) : 1739.63 Use% : 15 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : disk_windows-amds Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaal Total Capacity (GB) : 2048.00 Free Space (GB) : 1739.63 Use% : 15 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : zfs-p1nbu-incremental Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaaq Total Capacity (GB) : 19.70 Free Space (GB) : 2.91 Use% : 85 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : Lotus_Notes_Migration Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaar Total Capacity (GB) : 19.70 Free Space (GB) : 2.84 Use% : 85 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

Disk Pool Name : disk_windows-notes Disk Type : BasicDisk Disk Volume Name : Internal_16 Disk Media ID : @aaaas Total Capacity (GB) : 19.70 Free Space (GB) : 2.84 Use% : 85 Status : DOWN Flag : ReadOnWrite Flag : AdminUp Flag : InternalDown Num Read Mounts : 0 Num Write Mounts : 1 Cur Read Streams : 0 Cur Write Streams : 0

================================================

I tried runing this command to all the down disk pool name:

./nbdevconfig -changestate -stype BasicDisk -dp [disk-pol-name] -dv Internal_16 -state UP

successfully changed the state of disk volume. But when running nbdevquery again.. the status are still DOWN(Same as above)

7.1.x and Earlier
Backing Up
Backup and Recovery
Configuring
Error messages
NetBackup
Solaris
Troubleshooting
  • RiaanBadenhorst's avatar
    RiaanBadenhorst
    10 years ago

    PLease post df -h of the server, or show the file system layout where these volumes are lying.

    "1439281496 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume zfs-p1nbu-incremental:Internal_16 is down (not ok on root)"

    Is the basic disk configured on the / file system?

  • Bnj_V's avatar
    Bnj_V
    10 years ago

    Thanks Riaan and Marianne!

    /zfs-p1nbu/incremental/  and /p3_nbu_copy is the cause of my problem. no mount points found on df -h.

    Those are shared paths from our NetApp Filers.our previous storage admin did not informed us that those paths were removed/not used already. So I have to remove now the policies on those affected clients.

     

8 Replies

Sort By
  • RiaanBadenhorst's avatar
    RiaanBadenhorst
    Level 6
    10 years ago

    PLease post df -h of the server, or show the file system layout where these volumes are lying.

    "1439281496 1 1536 8 psk01 0 0 0 *NULL* nbemm Volume zfs-p1nbu-incremental:Internal_16 is down (not ok on root)"

    Is the basic disk configured on the / file system?

  • Bnj_V's avatar
    Bnj_V
    Level 3
    10 years ago

    Sample drive:

    root@TPBACKUPSP01 # /usr/openv/netbackup/bin/admincmd/nbdevquery -listdv -stype BasicDisk -dp zfs-p1nbu-incremental -U
    Disk Pool Name      : zfs-p1nbu-incremental
    Disk Type           : BasicDisk
    Disk Volume Name    : Internal_16
    Disk Media ID       : @aaaaq
    Total Capacity (GB) : 19.70
    Free Space (GB)     : 2.91
    Use%                : 85
    Status              : DOWN
    Flag                : ReadOnWrite
    Flag                : AdminUp
    Flag                : InternalDown
    Num Read Mounts     : 0
    Num Write Mounts    : 1
    Cur Read Streams    : 0
    Cur Write Streams   : 0

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago

    Please show us output of 'df -h' and 'bpstulist -L'

    The path specified in the STU must be on a different filesystem (as per STU config)

  • Bnj_V's avatar
    Bnj_V
    Level 3
    10 years ago

    Thanks you for the comment.

    Here's the output. 

    Here's the output:
     
    bpstulist -L
     
    Label:                atpsk01-hcart3-robot-tld-0-atpnas01-3020
    Storage Unit Type:    NDMP
    Host Connection:      atpsk01
    Number of Drives:     2
    On Demand Only:       no
    Max MPX/drive:        1
    Density:              hcart3 - 1/2 Inch Cartridge 3
    Robot Type/Number:    TLD / 0
    Max Fragment Size:    1048576 MB
    NDMP attach host:     atpnas01-3020
     
    Label:                atpsk01-hcart3-robot-tld-0-atpnas02-3020
    Storage Unit Type:    NDMP
    Host Connection:      atpsk01
    Number of Drives:     4
    On Demand Only:       no
    Max MPX/drive:        1
    Density:              hcart3 - 1/2 Inch Cartridge 3
    Robot Type/Number:    TLD / 0
    Max Fragment Size:    1048576 MB
    NDMP attach host:     atpnas02-3020
     
    Label:                atpsk01-hcart3-robot-tld-2
    Storage Unit Type:    Media Manager
    Host Connection:      atpsk01
    Number of Drives:     4
    On Demand Only:       no
    Max MPX/drive:        1
    Density:              hcart3 - 1/2 Inch Cartridge 3
    Robot Type/Number:    TLD / 2
    Max Fragment Size:    1048576 MB
     
    Label:                atpsk01-hcart3
    Storage Unit Type:    Media Manager
    Host Connection:      atpsk01
    Number of Drives:     4
    On Demand Only:       no
    Max MPX/drive:        1
    Density:              hcart3 - 1/2 Inch Cartridge 3
    Robot Type:           (not robotic)
    Max Fragment Size:    1048576 MB
     
    Label:                BMR-DISK
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/bmr"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                BMR-atpxisp01n1
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/bmr/atpxisp01n1"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                bmr_atpxi03
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/bmr/atpxi03"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                BMR-atpxisp01n2
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/bmr/atpxisp01n2"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                disk_Windows_Payroll_Full
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/p3_nbu_copy/Windows_Payroll_Full"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                StripTest-Disk
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/bmr/StripTest"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                disk_windows-barcode
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/p3_nbu_copy/windows-barcode"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                disk_windows-cimapps07
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/p3_nbu_copy/windows-cimapps07"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                disk_windows-cimapps04
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/p3_nbu_copy/windows-cimapps04"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                disk_windows-amds
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      1
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/p3_nbu_copy/windows-amds"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                zfs-p1nbu-incremental
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      4
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/zfs-p1nbu/incremental/"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                Lotus_Notes_Migration
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      4
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/zfs-p1nbu/incremental/Lotus_Notes"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
     
    Label:                disk_windows-notes
    Storage Unit Type:    Disk
    Storage Unit Subtype: Basic
    Host Connection:      atpsk01
    Concurrent Jobs:      4
    On Demand Only:       yes
    Max MPX:              1
    Path:                 "/zfs-p1nbu/incremental"
    Max Fragment Size:    524288 MB
    Stage data:           no
    Block Sharing:        no
    High Water Mark:      98
    Low Water Mark:       80
    Ok On Root:           no
    root@ATPBACKUPSP01 # 
    =====================================
    root@ATPBACKUPSP01 # df -h
    Filesystem             size   used  avail capacity  Mounted on
    /dev/dsk/c0t0d0s0       20G   9.2G    10G    48%    /
    /devices                 0K     0K     0K     0%    /devices
    ctfs                     0K     0K     0K     0%    /system/contract
    proc                     0K     0K     0K     0%    /proc
    mnttab                   0K     0K     0K     0%    /etc/mnttab
    swap                    13G   1.5M    13G     1%    /etc/svc/volatile
    objfs                    0K     0K     0K     0%    /system/object
    /platform/sun4u-us3/lib/libc_psr/libc_psr_hwcap1.so.1
                            20G   9.2G    10G    48%    /platform/sun4u-us3/lib/libc_psr.so.1
    /platform/sun4u-us3/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                            20G   9.2G    10G    48%    /platform/sun4u-us3/lib/sparcv9/libc_psr.so.1
    fd                       0K     0K     0K     0%    /dev/fd
    /dev/dsk/c0t0d0s3       15G   720M    14G     5%    /var
    swap                    13G   176K    13G     1%    /tmp
    swap                    13G    64K    13G     1%    /var/run
    /dev/dsk/c0t0d0s4       15G    12G   2.3G    85%    /opt
    /dev/dsk/c0t0d0s5      7.9G   4.4G   3.4G    57%    /export/home
    atpnas07-3020P:/vol/fv_atp3backupsp01
                           2.8T   1.6T   1.2T    58%    /p1_nbu_copy
    192.168.1.1:/vol/fv_bmr
                          1000G   502G   498G    51%    /bmr
    =================================================
     cat /etc/vfstab
    #device         device          mount           FS      fsck    mount   mount
    #to mount       to fsck         point           type    pass    at boot options
    #
    fd      -       /dev/fd fd      -       no      -
    /proc   -       /proc   proc    -       no      -
    /dev/dsk/c0t0d0s1       -       -       swap    -       no      -
    /dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1       no      -
    /dev/dsk/c0t0d0s3       /dev/rdsk/c0t0d0s3      /var    ufs     1       no      -
    /dev/dsk/c0t0d0s5       /dev/rdsk/c0t0d0s5      /export/home    ufs     2       yes     -
    /dev/dsk/c0t0d0s4       /dev/rdsk/c0t0d0s4      /opt    ufs     2       yes     -
    192.168.1.1:/vol/fv_bmr -       /bmr    nfs     -       yes     -
    #atpnas03-r200:/vol/fv_p1nbu/atpbackupsp01    -       /p3_nbu_copy    nfs     -       yes     -
    atpnas07-3020P:/vol/fv_atp3backupsp01   -       /p1_nbu_copy    nfs     -       yes     -
    /devices        -       /devices        devfs   -       no      -
    ctfs    -       /system/contract        ctfs    -       no      -
    objfs   -       /system/object  objfs   -       no      -
    swap    -       /tmp    tmpfs   -       yes     -      
    192.168.1.1:/vol/vol1 -         /root/testmount nfs     -       yes      - 
    atpnas6040-n2:/vol/fv_restore   -       /restore        nfs     -       yes     -
     
  • RiaanBadenhorst's avatar
    RiaanBadenhorst
    Level 6
    10 years ago

    Sorry, not to familiar with zfs. Where is /zfs-p1nbu/incremental/ located, i don't see in the df -h output

  • Bnj_V's avatar
    Bnj_V
    Level 3
    10 years ago

    Thanks Riaan and Marianne!

    /zfs-p1nbu/incremental/  and /p3_nbu_copy is the cause of my problem. no mount points found on df -h.

    Those are shared paths from our NetApp Filers.our previous storage admin did not informed us that those paths were removed/not used already. So I have to remove now the policies on those affected clients.