Netbackup services are down in Master Serevr

[john10]

Hello All,

We are running NBU 7.0.1 on win2K8, last week we had some Hard ware problem with Tape Drives, for the same we changed them today, and early morning we got alert from NetBackup master which is attached below, after change of Hard ware 3 Netbackup services(NetBackup Resource Broker,NetBackup Enterprise Media Manager,NetBackup Relational Database Manager) are not comming, and i alos added "NetBackup Enterprise Server" Licence from another server which is working fine i tried to recycle NBU services still not able to Launch the NBU console, and please find the eventvwr logs below.

Eventvwr alerts:

The description for Event ID 0 from source NetBackup cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

EMM interface initialization failed, status = 334

the message resource is present but the message is not found in the string/message table

[john10]

Hi All,

Please find the Bpps O/P below.

D:\Program Files\Veritas\NetBackup\bin>bpup.exe -v -f

NetBackup 7.0 -- Startup Utility

Starting services
> NetBackup Legacy Network Service
> NetBackup Legacy Network Service -- STARTED
> NetBackup Client Service
> NetBackup Client Service -- STARTED
> NetBackup Legacy Client Service
> NetBackup Legacy Client Service -- STARTED
> NetBackup SAN Client Fibre Transport Service
> NetBackup SAN Client Fibre Transport Service -- NOT STARTED
> NetBackup Event Manager
> NetBackup Event Manager -- STARTED
> NetBackup Relational Database Manager
> NetBackup Relational Database Manager -- NOT STARTED
> NetBackup Enterprise Media Manager
> NetBackup Enterprise Media Manager -- NOT STARTED
> NetBackup Resource Broker
> NetBackup Resource Broker -- NOT STARTED
> NetBackup Volume Manager
> NetBackup Volume Manager -- STARTED
> NetBackup Device Manager
> NetBackup Device Manager -- STARTED
> NetBackup Remote Manager and Monitor Service
> NetBackup Remote Manager and Monitor Service -- STARTED
> NetBackup Deduplication Manager
> NetBackup Deduplication Manager -- NOT STARTED
> NetBackup Deduplication Engine
> NetBackup Deduplication Engine -- NOT STARTED
> NetBackup Audit Manager
> NetBackup Audit Manager -- NOT STARTED
> NetBackup Database Manager
> NetBackup Database Manager -- STARTED
> NetBackup Compatibility Service
> NetBackup Compatibility Service -- STARTED
> NetBackup Request Daemon
> NetBackup Request Daemon -- STARTED
> NetBackup Job Manager
> NetBackup Job Manager -- STARTED
> NetBackup Policy Execution Manager
> NetBackup Policy Execution Manager -- STARTED
> NetBackup Service Layer
> NetBackup Service Layer -- STARTED
> NetBackup Vault Manager
> NetBackup Vault Manager -- STARTED
> NetBackup Key Management Service
> NetBackup Key Management Service -- STARTED
> NetBackup Storage Lifecycle Manager
> NetBackup Storage Lifecycle Manager -- STARTED
> NetBackup Agent Request Server
> NetBackup Agent Request Server -- NOT STARTED
> NetBackup Service Monitor
> NetBackup Service Monitor -- STARTED
> NetBackup Bare Metal Restore Master Server
> NetBackup Bare Metal Restore Master Server -- NOT STARTED
> NetBackup Bare Metal Restore Boot Server
> NetBackup Bare Metal Restore Boot Server -- STARTED
Start up of one or more services failed.

D:\Program Files\Veritas\NetBackup\bin>

[Marianne]

Running an old-old-old 7.0.1 version of NBU with evaluation/demo license?

If you feel that you have valid, permanent licenses for this installation, find your license key certificate(s) and compare with 'bpminlicense -list_keys -verbose'.

If you still have issues, contact your local Symantec partner or Customer Care (link at the bottom of this page).

[john10]

Yes Marianne,

we had permanent Licence from Symanetc, for the same i copied from another server and added in the server which we are facing issue. and below is the O/P, as i removed the Licence key.

FYI

D:\Program Files\Veritas\NetBackup\bin\admincmd>bpminlicense -list_keys -verbose
<Licece Key removed by me>
  file version    = 0x05000000
  time added      = 0x537c6eb2 05/21/14 17:15:30
  hostname        = ch71w0022
  product ID      = 6 NetBackup
  serial number   = 74264
  key version     = 0x05000000
  count           = 20
  server platform = 2 Any Windows platform
  client platform = 0 Any platform
  server tier     = 5 NetBackup Server
  client tier     = 0 No tier
  license type    = 1 Permanent
  Site ID         = 2006 Unknown site: 2006
  Feature ID      = 83 PureDisk MS Exchange Agent +
  Feature ID      = 80 PureDisk MS SQL Server Agent +
  Feature ID      = 75 Bare Metal Restore +
  Feature ID      = 74 Encryption +
  Feature ID      = 73 Open File Backup +
  Feature ID      = 71 Remote Client Support +
  Feature ID      = 70 Robotic Library Sharing Support +
  Feature ID      = 68 Microsoft RSM Robotic Libraries +
  Feature ID      = 61 MS SharePoint Agent +
  Feature ID      = 58 Inline Tape Copy +
  Feature ID      = 49 Library Based Tape Drives +
  Feature ID      = 46 DataStore +
  Feature ID      = 43 Encryption (Legacy DES 56-bit) +
  Feature ID      = 42 Encryption (Legacy DES 40-bit) +
  Feature ID      = 38 Sybase extension +
  Feature ID      = 36 Oracle extension +
  Feature ID      = 35 Lotus Notes extension +
  Feature ID      = 34 DB2 extension +
  Feature ID      = 33 MS SQL Server extension +
  Feature ID      = 32 MS Exchange extension +
  Feature ID      = 22 Additional clients +
  Feature ID      = 20 Base NetBackup +
<Licece Key removed by me>
  file version    = 0x05000000
  time added      = 0x537c6879 05/21/14 16:48:57
  hostname        = ch71w0022
  product ID      = 7 NetBackup Server
  serial number   = 14343
  key version     = 0x05000000
  count           = 0
  server platform = 0 Any platform
  client platform = 0 Any platform
  server tier     = 0 No tier
  client tier     = 0 No tier
  license type    = 1 Permanent
  Site ID         = 2006 Unknown site: 2006
  Feature ID      = 73 Open File Backup +
  Feature ID      = 71 Remote Client Support +
  Feature ID      = 70 Robotic Library Sharing Support +
  Feature ID      = 68 Microsoft RSM Robotic Libraries +
  Feature ID      = 58 Inline Tape Copy +
  Feature ID      = 48 Additional Drives/Robotics +
  Feature ID      = 46 DataStore +
  Feature ID      = 20 Base NetBackup +
<Licece Key removed by me>
  file version    = 0x05000000
  time added      = 0x537c3f04 05/21/14 13:52:04
  hostname        = ch71w0022
  product ID      = 7 NetBackup Server
  serial number   = 14326
  key version     = 0x05000000
  count           = 5
  server platform = 0 Any platform
  client platform = 0 Any platform
  server tier     = 0 No tier
  client tier     = 0 No tier
  license type    = 1 Permanent
  Site ID         = 2006 Unknown site: 2006
  Feature ID      = 75 Bare Metal Restore +
  Feature ID      = 74 Encryption +
  Feature ID      = 73 Open File Backup +
  Feature ID      = 71 Remote Client Support +
  Feature ID      = 70 Robotic Library Sharing Support +
  Feature ID      = 68 Microsoft RSM Robotic Libraries +
  Feature ID      = 58 Inline Tape Copy +
  Feature ID      = 49 Library Based Tape Drives +
  Feature ID      = 48 Additional Drives/Robotics +
  Feature ID      = 46 DataStore +
  Feature ID      = 43 Encryption (Legacy DES 56-bit) +
  Feature ID      = 42 Encryption (Legacy DES 40-bit) +
  Feature ID      = 22 Additional clients +
  Feature ID      = 20 Base NetBackup +
<Licece Key removed by me>
  file version    = 0x05000000
  time added      = 0x537c3efb 05/21/14 13:51:55
  hostname        = ch71w0022
  product ID      = 6 NetBackup
  serial number   = 378114
  key version     = 0x05000000
  count           = 0
  server platform = 0 Any platform
  client platform = 0 Any platform
  server tier     = 5 NetBackup Server
  client tier     = 0 No tier
  license type    = 1 Permanent
  Site ID         = 2006 Unknown site: 2006
  Feature ID      = 83 PureDisk MS Exchange Agent +
  Feature ID      = 81 Enterprise Vault Agent +
  Feature ID      = 80 PureDisk MS SQL Server Agent +
  Feature ID      = 61 MS SharePoint Agent +
  Feature ID      = 38 Sybase extension +
  Feature ID      = 37 Informix extension +
  Feature ID      = 36 Oracle extension +
  Feature ID      = 35 Lotus Notes extension +
  Feature ID      = 34 DB2 extension +
  Feature ID      = 33 MS SQL Server extension +
  Feature ID      = 32 MS Exchange extension +
  Feature ID      = 22 Additional clients +
<Licece Key removed by me>
  file version    = 0x05000000
  time added      = 0x537c3ef1 05/21/14 13:51:45
  hostname        = ch71w0022
  product ID      = 7 NetBackup Server
  serial number   = 219831
  key version     = 0x05000000
  count           = 0
  server platform = 0 Any platform
  client platform = 0 Any platform
  server tier     = 0 No tier
  client tier     = 0 No tier
  license type    = 1 Permanent
  Site ID         = 2006 Unknown site: 2006
  Feature ID      = 81 Enterprise Vault Agent +
  Feature ID      = 73 Open File Backup +
  Feature ID      = 71 Remote Client Support +
  Feature ID      = 70 Robotic Library Sharing Support +
  Feature ID      = 68 Microsoft RSM Robotic Libraries +
  Feature ID      = 58 Inline Tape Copy +
  Feature ID      = 46 DataStore +
  Feature ID      = 20 Base NetBackup +

D:\Program Files\Veritas\NetBackup\bin\admincmd>

[john10]

Hello Marianne,

for testing puppose i added NBU 7.5 Licence from command prompt and recycled NBU services still the GUI is not able to Launch due to same services not running, any more suggestions for this issue ?

[Marianne]

I don't think 7.5 license will work on 7.0 installation.

You will need to find your 7.0 license key certificates.

Or contact Customer Care.
They will be able to get your 7.0 keys from your IBR (Install Base Report).

Once you have confirmed that license keys are correct, stop and start NBU again.

If still not coming up, check nbemm log:
vxlogview -o 111 -t:00:10:00

(The above will show logs for last 10 minutes. Adjust -t parameter to include the most recent startup attempt.)

Please also check server.log.
(in <install-path>\Veritas\NetBackupDB\log\)

[john10]

Hello Marianne,

For the same we raised a case with Symanetc, they provided the TN, after following the TN the issue was resolved thanks for your time.

FYI

http://www.symantec.com/business/support/index?page=content&id=TECH162344

[Marianne]

We would probably see evidence of log issue in server.log (as per my previous post).

Logs are truncated when hot catalog backup is successful.

Are you running regular catalog backups? At least once per day?

[john10]

Yes Marianne we are running Catalog Backup every day.