cancel
Showing results for 
Search instead for 
Did you mean: 

Netbackup Appliance Error

cc1702004
Level 4

Hello,

Can someone help me ?

I need help..suddenly I have issued with our customer NBU appliance version 3.0/netbackup 8.0. When I opened the Netbackup Java Console for the Appliance, i see these options greyed out

1) Configure Disk Storage Servers
2) Configure Cloud Storage Servers
3) Configure Disk Pool

When I click on the Policy, I got the below error:

NBARSMgmt_i::queryApplicationMetadata: Error detected during
execution of query. Please check the logs for more
details (2)
(Status 2)

When I click on Activity Monitor, i get this error message :

The Activity Monitor failed because of
Connection to bpjob deamon on server failed.
For more details, refer to the log files

When i click on Device Monitor, I get the below message :


Device Monitor,
Database system error(220)
Database server is down. check for low disk space

(Don't know why i see this message, because the NBU appliance is very free : 300 GB used out of 13000 GB

When i click on the Login Activity shortcut, i see System Error 130.

This appliance we have stopped all the backup but customer is keeping it for restoration purposes.

Attached together with this is the nbu processes currently running, issued via command line.

I also received email notification on this

Your appliance has encountered the following error(s):
• The MSDP is configured, however the system failed to retrieve MSDP getmode or tlog status information. Contact Veritas Technical Support for more assistance.
o Time of event: 2021-01-27 18:41:32 (+08:00)
o UMI Event code: V-475-109-1000
o Component Type: MSDP
o Component: MSDP Partition
o Status: Not Available
o State: CRITICAL
o Additional information about this error is available at following link:
V-475-109-1000

 

11 REPLIES 11

cc1702004
Level 4

[Info] Performing sanity check on disks and partitions... (5 mins approx)
----------------------------------------------------------------------------------
Partition | Total | Available | Used | %Used | Status
----------------------------------------------------------------------------------
AdvancedDisk | 130 GB | 113.92 GB | 16.08 GB | 13 | Optimal
Configuration | 100 GB | 87.90 GB | 12.10 GB | 13 | Optimal
MSDP | 13.08 TB | 12.80 TB | 285.48 GB | 3 | Optimal
MSDP Catalog | 47 GB | 46.55 GB | 465.27 MB | 1 | Optimal
NetBackup Catalog | 250 GB | 245.53 GB | 4.47 GB | 2 | Optimal
Share | 0 GB | 0 GB | 0 GB | 0 | Not Configured
Unallocated | 49.8 GB | - | - | - | -

AdvancedDisk
-------------------------------------------------------------------
Partition | Total | Available | Used | %Used | Status
-------------------------------------------------------------------
0 | 130 GB | 113.92 GB | 16.08 GB | 13 | Mounted

Check shows MDSP partition is very free

 

quebek
Moderator
Moderator
   VIP    Certified
Hi
Are all nbu services up & running? Please go to support
Than processes
NetBackup show
If you have no idea how this should have looked like do
NetBackup start
It wont harm.
Worse case scenario
From maintenance run reboor
Login via ipmi interface and observe that reboot

i ran a software test from the appliance

.Support> Test software

Running Validation tests on Master Appliance tuaap02dmzbkup.

Checking whether serial number is present ... [OK]
Checking whether installation is successful ... [OK]
Checking whether critical kernel drivers exist and can work ... [OK]
Checking whether SSH is enabled ... [OK]
Checking whether hostname is set ... [OK]
Checking whether IP address is set ... [OK]
Checking whether appliance user accounts exist ... [OK]
Checking whether required packages are installed ... [OK]
Checking whether BIOS firmware is compatible with the software ... [WARNING]
Checking whether registration information is properly set ... [OK]
Checking whether NetBackup is configured and running ... - [Error] Unable to get list of disk pools from master server 't'.
- [Error] Unable to get list of disk pools from master server 'tuaap02dmzbkup'.
[OK]
Checking whether database service is running ... [OK]
Checking whether message queue service is running ... [OK]
Checking whether web server service is running ... [OK]
Checking whether storage units and diskpools have been created ... - [Error] Unable to get list of disk pools from master server 'tuaap02dmzbkup'.
- [Error] Unable to get list of disk pools from master server '.
- [Error] Unable to get list of disk pools from master server '.
- [Error] Unable to get list of disk pools from master server .
- [Error] Unable to get list of disk pools from master server ''.
[NA]
Checking test backup operations ... [FAIL]
Checking test restore operations ... [NA]
Expiring test backup images and deleting test policy ... [OK]

Completed running tests on the appliance.
SELF TEST RESULT : [FAIL]

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@cc1702004 

If I look at your attachment, it seems that 'Processes NetBackup Show' returned nothing?

That means that NetBackup is not running.

Have you tried to start NBU with  Processes NetBackup Start ?

PS:
not sure if you know this, but there is a separate NBU Appliance forum: https://vox.veritas.com/t5/NetBackup-Appliance/bd-p/netbackup-appliance

My apologies..not sure why previous attachment has nothing..reattaching the processes..yes the netbackup services are running

DPeaco
Moderator
Moderator
   VIP   

If this were my nbu server? I'd shutdown Netbackup, shutdown pbx_exchange, make sure that all processes for nbu were down (bpps -x would show no active processes) and then I'd start vxpbx_exchanged, and then I'd run bp.start_all.

Check all processes via bpps -x (at an elevated prompt). Sometimes a clean restart of netbackup will fix the issues we wrestle with. IMHO, a clean start of netbackup is a great place to start. Also, if your appliance hasn't been rebooted in a long time (more than 90 days), I'd look at rebooting it.

Also, there are many fixes and enhancements in 3.2/8.2 that helps with things like this and your firmware may also need updating. Just some suggestions is all. :) 

Thanks,
Dennis

quebek
Moderator
Moderator
   VIP    Certified

Hi

In posted output I cant see bpjobd deamon... IMHO restart NBU or reboot whole appliance.

Hi @cc1702004 

Have you checked the NetBackup license to make sure it is okay - that is often a reason why things become greyed out in the GUI when a license expires.

Also I noticed in the process listing that you seem to have a NAS device mounted on the appliance as a backup target "/mnt/remote/SynNasArchive/Jan" and there are many bpdm processes attempting to run a volume_cleanup on this (and are presumably stuck). What is this volume and how is it mounted (and is it a recent change)?

Also - the process listing shown is indicating the appliance hostname (short) is nbu02p while the validation tests output is from the master server tuaap02dmzbkup. Are these the same system?

I agree with the others though that a reboot of the appliance is not going to hurt matters and may well resolve the issue if it is transient.

Cheers
David

I rebooted the nbu appliance..

Some of the things seem to come up..for example when i click on Device Manager, I can see the tape library, then i can activate the disabled policy, but still could not see Storage unit..

After 10 minutes, there is a message pop-up, saying there is a database system error, complaining low disk space..i don;t understand this NBU appliance we disabled all backups, 10 TB of free space...

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@cc1702004 

Where do you see the 10TB free space?  The Dedupe pool?

Please check all filesystems - I cannot remember off-hand where NBU EMM database resides on an appliance.
EMM and the relational database will shut down when file system is low on available space to prevent database corruption.

Krutons
Moderator
Moderator
   VIP   

Have you elevated access on the appliance and looked at the individual file systems?