communication issues after /usr/openv/netbackup/bin/update_clients

[Bendoumah_Abdel]

Hi everyone,

after executing /usr/openv/netbackup/bin/update_clients some clients are not backed  up showing me error 58 can't connect to client any idea?

best regards.

[Mark_Solutions]

Sounds like somethimng may have gone wrong - you need to check those clients to ensure that their NetBackup daemons are running.

What output did you get when you ran the command?

[RamNagalla]

when you run the /usr/openv/netbackup/bin/update_clients, did you see update successfull output?

if update got failed there is a possiblity of corrept netbackup in client, which need end up with re-installing the netbackup client.

[Marianne]

What did you answer to 'update bp.conf' question? Check failing clients' bp.conf files if all media servers are still there. Ensure bpcd log folder exists on these clients. After next failure, copy bpcd log on one of these clients to bpcd.txt and post as File attachment.

[Bendoumah_Abdel]

Hi Marianne,

is this the right path of the directory : /usr/openv/netbackup/logs/bpcd??

best regards.

[Bendoumah_Abdel]

Hi, 

this is an output of the policy jib:

02/19/2013 10:52:19 - Info nbjm (pid=13882) starting backup job (jobid=88523) for client prod1-kcc, policy KPSA_prod1-kcc_FS, schedule FULL

02/19/2013 10:52:19 - Info nbjm (pid=13882) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=88523, request id:{6E929F7E-7A82-11E2-968D-0021283C0178})

02/19/2013 10:52:19 - requesting resource mob-archive-hcart3-robot-acs-0

02/19/2013 10:52:19 - requesting resource mob-archive.NBU_CLIENT.MAXJOBS.prod1-kcc

02/19/2013 10:52:19 - requesting resource mob-archive.NBU_POLICY.MAXJOBS.KPSA_prod1-kcc_FS

02/19/2013 10:52:19 - granted resource  mob-archive.NBU_CLIENT.MAXJOBS.prod1-kcc

02/19/2013 10:52:19 - granted resource  mob-archive.NBU_POLICY.MAXJOBS.KPSA_prod1-kcc_FS

02/19/2013 10:52:19 - granted resource  LF8926

02/19/2013 10:52:19 - granted resource  IBM.ULTRIUM-TD3.001

02/19/2013 10:52:19 - granted resource  mob-archive-hcart3-robot-acs-0

02/19/2013 10:52:20 - estimated 183900571 kbytes needed

02/19/2013 10:52:20 - begin Parent Job

02/19/2013 10:52:20 - begin Bare Metal Restore: Start Notify Script

02/19/2013 10:52:20 - Info RUNCMD (pid=12971) started

02/19/2013 10:52:20 - Info RUNCMD (pid=12971) exiting with status: 0

Operation Status: 0

02/19/2013 10:52:20 - end Bare Metal Restore: Start Notify Script; elapsed time 0:00:00

02/19/2013 10:52:20 - begin Bare Metal Restore: Bare Metal Restore Save

02/19/2013 10:52:21 - started process bpbrm (pid=12978)

02/19/2013 10:52:36 - Info bpbrm (pid=12978) connect failed STATUS (18) CONNECT_FAILED

02/19/2013 10:52:36 - Info bpbrm (pid=12978) status: FAILED, (43) CONNECT_RESET; system: (131) Connection reset by peer; FROM 0.0.0.0 TO prod1-kcc 10.96.12.20 bpcd VIA pbx

02/19/2013 10:52:36 - Info bpbrm (pid=12978) status: FAILED, (42) CONNECT_REFUSED; system: (146) Connection refused; FROM 0.0.0.0 TO prod1-kcc 10.96.12.20 bpcd VIA vnetd

02/19/2013 10:52:36 - Info bpbrm (pid=12978) status: FAILED, (42) CONNECT_REFUSED; system: (146) Connection refused; FROM 0.0.0.0 TO prod1-kcc 10.96.12.20 bpcd

02/19/2013 10:52:36 - Error bpbrm (pid=12978) Cannot connect to prod1-kcc

02/19/2013 10:52:36 - Info bpbkar (pid=0) done. status: 58: can't connect to client

02/19/2013 10:52:36 - end writing

Operation Status: 58

02/19/2013 10:52:36 - end Bare Metal Restore: Bare Metal Restore Save; elapsed time 0:00:16

02/19/2013 10:52:36 - begin Bare Metal Restore: Policy Execution Manager Preprocessed

Operation Status: 58

02/19/2013 10:58:01 - end Bare Metal Restore: Policy Execution Manager Preprocessed; elapsed time 0:05:25

02/19/2013 10:58:01 - begin Bare Metal Restore: Stop On Error

Operation Status: 0

02/19/2013 10:58:01 - end Bare Metal Restore: Stop On Error; elapsed time 0:00:00

02/19/2013 10:58:01 - begin Bare Metal Restore: End Notify Script

02/19/2013 10:58:02 - Info RUNCMD (pid=14848) started

02/19/2013 10:58:02 - Info RUNCMD (pid=14848) exiting with status: 0

Operation Status: 0

02/19/2013 10:58:02 - end Bare Metal Restore: End Notify Script; elapsed time 0:00:01

Operation Status: 58

02/19/2013 10:58:02 - end Parent Job; elapsed time 0:05:42

can't connect to client  (58)

best regards.

[Mark_Solutions]

This may show a refusal as well - check the clients bp.conf file (/usr/openv/netbackup) - i think Marianne was hinting at this.

You bpcd log directory was correct - do you have the log from there following this failed backup?

[Marianne]

bpcd folder needs to be created under /usr/openv/netbackup/logs/.

Log file will be created in bpcd folder that should indicate reason for failed connection.

[Vickie]

Check the bp.conf file exist under "Install-path\netbackup\bp.conf". 

Do this file contains the correct entry of master and media servers ?

Example of bp.conf file is mentioned below,

SERVER = Master server name

SERVER = Media server name

CLIENT_NAME = Client server name

REQUIRED_INTERFACE = Client backup interface name (if any configured)

[Bendoumah_Abdel]

I already create this folder but it's still empty:

root@prod1-kcc[/usr/openv/netbackup/bin/goodies/bpcd]# pwd

/usr/openv/netbackup/bin/goodies/bpcd

root@prod1-kcc[/usr/openv/netbackup/bin/goodies/bpcd]# ls -lrt

total 0

i run a backup but no log is generated in this folder.

also the file bp.conf is correct it contains all the media server and the client.!!!!

[Mark_Solutions]

you are in the goodies folder!

usr/openv/netbackup/logs/bpcd/

[Bendoumah_Abdel]

root@prod1-kcc[/usr/openv/netbackup/logs/bpcd]# pwd

/usr/openv/netbackup/logs/bpcd

root@prod1-kcc[/usr/openv/netbackup/logs/bpcd]# ls -lrt

total 0

[Mark_Solutions]

OK - I need you to do five things and show me the outputs please:

1. On the client run /usr/openv/netbackup/bin/bpps -x

2. Show us the bp.conf on the client

3. Ping the client from the Master and the Media Servers and show us the outputs

4, Ping the Master and the Media Servers from the client and show us the outputs.

5. Run ifconfig on the Master and Media Servers and Clients and show us the outputs

Thanks

[Bendoumah_Abdel]

1.root@prod1-kcc[/NBU/NetBackup_7.5_CLIENTS]# /usr/openv/netbackup/bin/bpps -x

ksh: /usr/openv/netbackup/bin/bpps:  not found

2.root@prod1-kcc[/NBU/NetBackup_7.5_CLIENTS]# cat /usr/openv/netbackup/bp.conf

SERVER = mob-archive

SERVER = media-arch

SERVER = ict-prod

SERVER = ict-medbackup

SERVER = mobicom1

SERVER = mobicom2

SERVER = mobapp

SERVER = mobinfo

SERVER = detcall

SERVER = m-form

SERVER = sappprd

SERVER = saprd

SERVER = freepart

SERVER = bscs-lh

SERVER = rtx-lh

SERVER = sun-restore-ce0

SERVER = bscs-lh-ge

SERVER = rtx-lh-ge

SERVER = rtx

SERVER = fe-end1

SERVER = mq-lh

SERVER = oracle-lh

SERVER = fe-end2

SERVER = saraprod2

SERVER = ict-prod.ce0

SERVER = sun-restore

SERVER = sun-restore-ge

SERVER = sun-restore-ge1

SERVER = m9000-bscs

SERVER = m9000-fe-end1

SERVER = m9000-rtx

SERVER = m9k1-dom0

SERVER = m9000-fe-end2

SERVER = movix-m9k1-dom0

SERVER = movix-m9k1-dom1

SERVER = movix-m9k2-dom0

SERVER = srvbkp-ge

SERVER = srvbkp-ge.iamdg.net.ma

SERVER = srvvc

SERVER = m9k1-dom7

CLIENT_NAME = prod1-kcc

3.root@prod1-kcc[/NBU/NetBackup_7.5_CLIENTS]# ping mob-archive

mob-archive is alive

root@mob-archive[/tmp]#ping prod1-kcc

prod1-kcc is alive

4.root@mob-archive[/tmp]#ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1

        inet 127.0.0.1 netmask ff000000 

nxge0: flags=1001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,FIXEDMTU> mtu 1500 index 2

        inet 10.80.0.77 netmask ffffff80 broadcast 10.80.0.127

        ether 0:21:28:3c:1:78 

nxge3: flags=1001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,FIXEDMTU> mtu 1500 index 3

        inet 204.1.1.36 netmask ffffff00 broadcast 204.1.1.255

        ether 0:21:28:3c:1:7b 

nxge3:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

        inet 192.168.13.77 netmask ffffff00 broadcast 192.168.13.255

nxge3:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

        inet 204.1.4.77 netmask ffffff00 broadcast 204.1.4.255

nxge3:3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

        inet 192.168.1.77 netmask ffffff00 broadcast 192.168.1.255

nxge3:4: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

        inet 192.168.0.77 netmask ffffff00 broadcast 192.168.0.255

mob-archive is the master server.

best regards.

[Mark_Solutions]

OK - didn't make things clear enough ...

I need to know what the master and media servers resolve the ip address of the client as and what the client resolves the master and media servers as (you obviously have a huge number of media servers so maybe just pick one or two that may try and back this client up)

Can you check that bpps exists on the client in the usr/openv/netbackup/bin directory - if so try it without any switches or using the -a swicth and show the output

[Bendoumah_Abdel]

bpps doesn't exist in this path:

root@prod1-kcc[/usr/openv/netbackup/bin]# ls

bparchive        bpcd_old         goodies          jnbSA            version

bpcd             bpdynamicclient  jbpSA            tar              vnetd

even with a master server and one media server in bp.conf i'm facing the same error can't connect to client.

[Mark_Solutions]

There should be a whole lot more in that directory!!

What ever went wrong when you did the upgrade has left only half a client installation.

I would see if you can re-install it locally or uninstall it and install it afresh - but do it locally so that you have full control and visiblity over the process.

But this does explain your problem

[Bendoumah_Abdel]

even to reinstall it the installation is hanging in this point.

root@prod1-kcc[/NBU/NetBackup_7.5_CLIENTS]# ./install

Symantec Installation Script

Copyright 1993 - 2012 Symantec Corporation, All Rights Reserved.

        Installing NetBackup Client Software

Please review the SYMANTEC SOFTWARE LICENSE AGREEMENT located on

the installation media before proceeding.  The agreement includes

details on the NetBackup Product Improvement Program.

For NetBackup installation and upgrade information specific to your

platform and to find out if your installed EEBs or hot fixes are

contained in this release, check out the Symantec Operations Readiness

Tools (SORT) Installation and Upgrade Checklist and Hot fix and EEB

Release Auditor, respectively, at https://sort.symantec.com/netbackup.

Do you wish to continue? [y,n] (y) y

Do you want to install the NetBackup client software for this client? [y,n] (y) y

This package will install Solaris/Solaris10 client.

NetBackup client is currently at release level 7.5.

This package will install NetBackup client 7.5.

Client binaries are located in /NBU/NetBackup_7.5_CLIENTS/NBClients/anb/Clients/usr/openv/netbackup/client/Solaris/Solaris10.

NetBackup Deduplication software is installed.

Saving client binaries for Solaris/Solaris10.

Installing PBX...

Same version is already installed. Nothing to upgrade for package VRTSpbx

Installation log located here: /var/tmp/installpbx-27076-021913161359.log

Unpacking SYMCnbclt package.

Checking for pre-existing SYMCnbclt package.

Installing SYMCnbclt package.

[Marianne]

Something wrong on your client...

Please show output of:

pkginfo |grep SYMC

If I may ask - was Client package extracted with gtar? See http://www.symantec.com/docs/TECH15408

[Bendoumah_Abdel]

Hi Marianne,

root@prod1-kcc[/root]# pkginfo |grep SYMC

application SYMCnbjava                       NetBackup Java Console

application SYMCnbjre                        NetBackup Java JRE

application SYMCpddea                        Symantec Deduplication Agent

Regards.