2106: disk storage server is down and 2074 EMM status: Disk volume is down.

[Amel2]

We are using Netbackup 9.1.0.1 and NBU model 5240. 

We have 2 errors that appear under the Master Server (reports > problems) 

First error that appeared: PureDiskVolume is marked down. Storage server {rtmo}

Second error: NBU Status: 2074 EMM status: Disk volume is down. 

From our failed disk backup, it states 2106: disk storage server is down. 

We also saw from the host properties > media server > vnetd proxy encountered an error. 

We have rebooted the media server and restarted the services on the master server. But the same issue has appeared again after 2 days. We did not make any changes other than enabling the duplication jobs for all backup 1 month ago.  We have limited logs from the media and master server as the logging settings is set as 'Mininum logging'.

Any clue on why this issue suddenly occured? 

[Nicolai]

It's difficult to say what's wrong, i think the way forward is to inspect logs. But before starting very network connectivity is present and without issues.

This command will increase the logging level for the DSM (Disk Service Manager)

vxlogcfg -a -p 51216 -o 178 -s DebugLevel=5 -s DiagnosticLevel=6

After finding the issues reset the logging level with 

vxlogcfg -a -p 51216 -o 178 -s DebugLevel=1 -s DiagnosticLevel=6

To see the log you use the vxlogview command , see : https://www.veritas.com/support/en_US/article.100017099

My suggestion is to do :

vxlogview -p 51216 -o 178 -t 24:00:00

This command will list all logs entries from DSM 24 hours back.

[quebek]

Hi

I would checked

• Appliance/Status
• Manage/Storage/Show All
• Support/Disk - to see if all msdp related things are mounted...

But if you are unfamiliar with outcomes from above it can be hard to tell what is wrong...

[Michal_Mikulik1]

I guess when vnetd error is observed, then the network is a problem, not the appliance or its storage itself.

Test if the appliance is working as a mere client or not. So create a simple test policy where the appliance is the client, storage unit is outside the appliance (for example ad-hoc Basic Storage unit folder on the Master Server) and a few small files as Backup Selection. Run a backup and we will see according to the status you will get from this job.

Regards

Michal

[Nicolai]

or just run a ping test with a large package size

ping acme.com  -l 8192 -t

let the ping test run thru the night. A single lost ping package is not a issues, but if there is multiple lost pings in a row, it indicate issues with the network.

[Amel2]

Thanks for the reply all. 

We ran the bptestbpcd.exe from master to media server, it shows that we have good connections. In addition to that, our backup and deduplication jobs are running fine however, there are some days that we encounter vnetd errors. Is there any cause for the vnetd service to stop in the media server? How do we know that the vnetd service is overloaded with our job connections?

Also, for the debug logs, how much space would level 5 logging take on the Media server? Is level 3 sufficient to get some hints? 

[Nicolai]

Hi @Amel2 

Vnetd is usually a very stable service plus traffic has been moved to 1556/pbx in modern version of Netbackup.

I can't say how much space is needed for logging, it depend on the activity. But start out with a 3 and then increase to 5. Remember is only one service that has increased logging.

[Amel2]

Hi Nicolai, 

Thanks for the reply, 

Our backup has been failing every alternate day, we have been rebooting the media server when it fails. The backup is successful for that day, then it fails the next day. 

We have checked our hardware and noticed that there is permanent amber light on the hard disk. But overall NBU health shows green light. 

Is this due to some hardware failure? or it is software issue? 

Thank you

[Alexis_Jeldrez]

Hi Amel2.

You need to make sure the hardware is 100% ok before using software.

If you have any doubts, consult with your hardware vendor and only afterwards check the software.

[Amel2]

Hi Alexis_Jeldrez,

We have ran the hardware test on the media server (Main > Support > Test hardware) and it passed even thou there is amber light on the hard disk. 

Would you know where else we can check the logs for hard disk error? 

Thank you