cancel
Showing results for 
Search instead for 
Did you mean: 

Disk Pool Up but Storage Server Down

Sorin
Level 3

Hi all,

We had a failed volume for about 1 month. We changed the disks and the volume is fine in Windows. However in NBU is a different story. 

PS C:\Program Files\Veritas\NetBackup\bin\admincmd> ./nbdevquery -liststs -stype PureDisk -storage_server hypbak01 -U
Storage Server : hypbak01
Storage Server Type : PureDisk
Storage Type : Formatted Disk, Network Attached
State : UP
Flag : OpenStorage
Flag : CopyExtents
Flag : AdminUp
Flag : InternalUp
Flag : LifeCycle
Flag : CapacityMgmt
Flag : FragmentImages
Flag : Cpr
Flag : FT-Transfer
Flag : OptimizedImage

PS C:\Program Files\Veritas\NetBackup\bin\admincmd> ./nbdevquery -listdp -stype PureDisk -dp hypbak01_dedup_1 -U
Disk Pool Name : hypbak01_dedup_1
Disk Pool Id : hypbak01_dedup_1
Disk Type : PureDisk
Status : UP
Flag : Patchwork
Flag : Visible
Flag : OpenStorage
Flag : SingleStorageServer
Flag : CopyExtents
Flag : AdminUp
Flag : InternalUp
Flag : LifeCycle
Flag : CapacityMgmt
Flag : FragmentImages
Flag : Cpr
Flag : FT-Transfer
Flag : OptimizedImage
Raw Size (GB) : 28614.00
Usable Size (GB) : 28614.00
High Watermark : 98
Low Watermark : 80
Num Volumes : 1
Max IO Streams : 40
Comment :
Storage Server : hypbak01 (DOWN)

I also tried 

PS C:\Program Files\Veritas\NetBackup\bin\admincmd> ./nbdevconfig -changests -storage_server hypbak01 -stype PureDisk -s
tate UP
Storage server hypbak01 has been successfully changed
PS C:\Program Files\Veritas\NetBackup\bin\admincmd> ./nbdevconfig -changests -storage_server hypbak01 -stype PureDisk -s
tate RESET
Storage server hypbak01 has been successfully changed
PS C:\Program Files\Veritas\NetBackup\bin\admincmd>

And restarting of the services.

Any other idea?

BIG THANKS in advance

 

9 REPLIES 9

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@Sorin

Does  'failed volume '  mean that everything on the dedupe volume was lost?

I guess this is the case as I have noticed this in your output:

Raw Size (GB) : 28614.00
Usable Size (GB) : 28614.00

 

Have you perhaps performed any of these steps as per the NBU Dedupe manual?

Chapter 8: 
This chapter includes the following topics:
■ About recovering the MSDP catalog
■ Restoring the MSDP catalog from a shadow copy
■ Recovering from an MSDP storage server disk failure
■ Recovering from an MSDP storage server failure

Hi,

And thanks for your answer.

By "failed volume" I mean that one or two of the underline disks failed and the storage device made the volume unavailable for the OS. Once we replaced the failed drives all the data was back and available now in Windows Explorer.

As I browse the volume now it sais 16TB of used data on the volume.

Nicolai
Moderator
Moderator
Partner    VIP   

there might be data consistency issues with the MSDP volume

Take a look at the SPAD log file :

/storage_path/log/spad/spad.log

If its a reasonable size attach it to a post , do not bulk debug log text in a post

Hi,

Here is the log

Thanks,

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Could you please check/share the spoold.log ( storage_path\log\spoold) as well? 

I notice that there is nothing in the spad log since 10 Aug. 

Was NBU restarted after re-mounting the volume?

Can you check if MSDP processes are running?
NetBackup Deduplication Engine - spoold
NetBackup Deduplication Manager - spad

 

Hello again,

And thanks for helping me with this.

I've attached the log. After remounting the volume I've restarted both the media server and the master server.

Deduplication Engine and Deduplication Manager are both running on the media server.

 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

No errors in spoold log.

I'm running out of ideas....

Last thing to check (that I can think of) :
Please run this command on the master server:
nbemmcmd -listhosts -verbose 

What does MachineState show for this media server? 

If output looks good, I suggest that you log a Support call with Veritas.

hypbak01
        ClusterName = ""
        MachineName = "hypbak01"
        FQName = "hypbak01"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x17
        MachineNbuType = media (1)
        MachineState = active for tape and disk jobs (14)
        MasterServerName = "asppbu02"
        NetBackupVersion = 8.0.0.0 (800000)
        OperatingSystem = windows (11)
        ScanAbility = 5

hypbak01
        MachineName = "hypbak01"
        FQName = "hypbak01"
        MachineDescription = "PureDisk"
        MachineFlags = 0x2
        MachineNbuType = ndmp (2) (storage_server)

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Output looks good. 

Time to log a Support call?