Error Job starting

[pretorian]

Hi all,

We have Netbackup 7 (platform windows) i'm getting log error like this :

5/27/2010 11:26:26 AM - requesting resource netbackup-srv-hcart-robot-tld-0
5/27/2010 11:26:26 AM - requesting resource netbackup-srv.NBU_CLIENT.MAXJOBS.netbackup-srv
5/27/2010 11:26:26 AM - requesting resource netbackup-srv.NBU_POLICY.MAXJOBS.Copy_Permanent
5/27/2010 11:26:26 AM - granted resource netbackup-srv.NBU_CLIENT.MAXJOBS.netbackup-srv
5/27/2010 11:26:26 AM - granted resource netbackup-srv.NBU_POLICY.MAXJOBS.Copy_Permanent
5/27/2010 11:26:26 AM - granted resource 0038L4
5/27/2010 11:26:26 AM - granted resource IBM.ULT3580-TD4.000
5/27/2010 11:26:26 AM - granted resource netbackup-srv-hcart-robot-tld-0
5/27/2010 11:26:26 AM - estimated 25006334 Kbytes needed
5/27/2010 11:26:26 AM - started
5/27/2010 11:26:27 AM - started process bpbrm (4072)
5/27/2010 11:26:27 AM - conne

Please, wating for your valuable suggestion ?
Thanks in advance.


Regards,


Ahmad

[Andy_Welburn]

Your output looks normal altho' appears to have 'stopped' at connecting. Or is that the issue?

[pretorian]

Hi Andy,

Herewith the next error appears :

5/27/2010 11:26:20.656 [NBUErrorTranslator::getErrorDetails()] NB error code: 25 NB localized Error text: cannot connect on socket(../NBUErrorTranslator.cpp:46)
5/27/2010 11:26:20.656 [ExceptionManager::fillExceptionObject()] Error:ClientQueryFacetImpl:opulateOutput()::../ClientQueryFacetImpl.cpp(145): errNo: 25 localizedStr: 0000000002DC3A60 additionalStr: 0000000000000000(../ExceptionManager.cpp:66)
5/27/2010 11:26:20.656 [CORBAExceptionTranslator::getErrorDetails()] Exception name:NBSLOpException(../CORBAExceptionTranslator.cpp:45)
5/27/2010 11:26:20.656 [CORBAExceptionTranslator::getErrorDetails()] Rethrowing NBSLOpException exception(../CORBAExceptionTranslator.cpp:83)
5/27/2010 11:26:20.656 [AsyncCollector_T::handle_input] Caught a CORBA exception while loopinguser exception, ID 'IDLymantec/NetBackup/SL/NBSLOpException:1.0'(h:\70\src\nb\nbsl\server\BH_AsyncCollector.cpp:80)
5/27/2010 11:26:25.844 [DefaultQueue::queueRequest] queueing PEMretryJob : jobid=9529, parentid=9529, retryType=2 -- retry count=-1(../CallbackQueue.cpp:1269)
5/27/2010 11:26:25.844 [DefaultQueue::handle_input] sending PEMretryJob : jobid=9529(../CallbackQueue.cpp:1392)
5/27/2010 11:26:25.844 [DefaultQueue::handle_input] (0000000001526920) CallbackQueue now empty(../CallbackQueue.cpp:1476)
5/27/2010 11:26:25.844 [JobManager_i::getJobById] (00000000019BDFF0) job not found in map, jobid=9529(../JobManager_i.cpp:880)
5/27/2010 11:26:25.859 V-116-254 [PolicyClientTask::cancel] scheduling for policy Copy_Permanent, client netbackup-srv, has been abandoned because of execution of new job, will recalculate after job completes
5/27/2010 11:26:25.859 [Info] V-116-41 running session_start_notify
5/27/2010 11:26:25.859 [PemTask::taskComplete] (ID:0000000001B75988) Active subtask count=0(../PemTask.cpp:527)
5/27/2010 11:26:25.859 V-116-248 [ProcessDefinition::runIt] spawned command "C:\Program Files\Veritas\NetBackup\bin\session_start_notify.cmd", pid=6500
5/27/2010 11:26:26.375 V-116-249 [ProcessDefinition::notify] command "C:\Program Files\Veritas\NetBackup\bin\session_start_notify.cmd", pid=6500, completed with status 0
5/27/2010 11:26:26.406 [RecoverableJob::registerForEmail] (ID:0000000001B4DF88) Registered for email(../RecoverableJob.cpp:2682)
5/27/2010 11:26:26.406 [Info] V-116-25 job with jobid=9529 restarted as jobid=9533
5/27/2010 11:26:26.406 V-118-227 [ResBroker_i::requestResources] received resource request ID {1095D8A2-24EB-4B03-B437-B83FCEB5629F}, priority 0, secondary priority 26347, description THE_BACKUP_JOB-9533-{1095D8A2-24EB-4B03-B437-B83FCEB5629F}
5/27/2010 11:26:26.406 [ResBroker_i::requestResources] RequestSeq:
Request=0 provider=MPXProvider resourcename=MpxRequest-9533 MPXGroupRequest maxmpx=1
    SubRequest=0 provider=DriveOperationProvider resourcename=netbackup-srv-hcart-robot-tld-0 userSequence=0 RB:torageUnitRequest { StorageUnitRequest { storageUnit=netbackup-srv-hcart-robot-tld-0 mediaPool=Copy_Permanent retentionLevel=9 mustUseLocalMediaServer=no failOnError=no mpxRequired=no mustBeNdmp=no getMaxFreeSpace=no minFreeSpaceKBytes=25006334 usageType=1 client=netbackup-srv shareGroup=*ANY* } }
Request=1 provider=NamedResourceProvider resourcename=netbackup-srv.NBU_CLIENT.MAXJOBS.netbackup-srv CountedResourceRequest { max=1 }
Request=2 provider=NamedResourceProvider resourcename=netbackup-srv.NBU_POLICY.MAXJOBS.Copy_Permanent CountedResourceRequest { max=2147483647 }

I don't know why the job is can't starting again ?

Regards,

Abbas

[Andy_Welburn]

Has the backup ever worked?

I can certainly see an EC25 "Cannot connect on socket" error.

What is the OS of the client? If Windows, can you confirm that the NB client service is running?

Firewall interfering or possible network connectivity or name resolution issues - see the following TechNote on the use of bpclntcmd, telnet etc etc to help resolve these:
In-depth Troubleshooting Guide for Exit Status Code 25

You could also try bptestbpcd -client client -verbose -debug from the master & post the output here.

[Rajesh_s1]

From the logs it says  "cannot connect on socket" , this clearly stats that there is some communication problem between your master server and client .

Try to browse the client properties from master server .

check the belwo commands ,

Run the command from your master server : bpclntcmd -hn (client name)

bpclntcmd -get_pbx_port (client name)

Tell me the windows version of your client . 

Regards,
rajesh

[pretorian]

Hai Andy,

The client_name is netbackup-svr and herewith the output 'bptestbpcd -client netbackup-srv -verbose -debug" :
1 1 1
1.1.1.99:1293 -> 1.1.1.99:13724
1.1.1.99:1294 -> 1.1.1.99:13724
PEER_NAME = netbackup-srv.xxxxxx.com
HOST_NAME = netbackup-srv
CLIENT_NAME = netbackup-srv
VERSION = 0x07000000
PLATFORM = win_x64
PATCH_VERSION = 7.0.0.0
SERVER_PATCH_VERSION = 7.0.0.0
MASTER_SERVER = netbackup-srv
EMM_SERVER = netbackup-srv
1.1.1.99:1304 -> 1.1.1.99:13724

And Rajesh :
Herewith the output  :
C:\Program Files\Veritas\NetBackup\bin>bpclntcmd -hn netbackup-srv
host netbackup-srv: netbackup-srv.xxxxxx.com at 1.1.1.99 (0x63010101)
host netbackup-srv: netbackup-srv.xxxxxx.com at 1.4.0.2 (0x2000401)
host netbackup-srv: netbackup-srv.xxxxxx.com at 192.168.11.108 (0x6c0ba8c0)

C:\Program Files\Veritas\NetBackup\bin> bpclntcmd -get_pbx_port netbackup-srv
1556

The client platform is windows 2003

Regards,

Abbas

[Marianne]

Three IP addresses assigned to the same host name? 
host netbackup-srv: netbackup-srv.xxxxxx.com at 1.1.1.99 (0x63010101)
host netbackup-srv: netbackup-srv.xxxxxx.com at 1.4.0.2 (0x2000401)
host netbackup-srv: netbackup-srv.xxxxxx.com at 192.168.11.108 (0x6c0ba8c0)

Which interface is TCP/IP going to use when connecting to netbackup-srv? Which interface must be used for connect-back?
Each interface should be bound to a unique hostname and own IP address.

[Rajesh_s1]

Hi ,

As per the output PBX is listening properly . But i think due to three network interfaces it's getting confused .

Ping the client from your system and findout excatly which is the IP it's using . Then you add the same IP in master server host file (i.e. system32\drivers\etc\hosts)  , in host file you 1st add the IP address and then host name , save and close the host file . After this try the backup , it would resolve your issue . If still you have issue then try the below steps ,


open the client properties of "netbackup-srv"  from master server and then go to the universal setting tab in that right side you can see that a option 'use specified network interface' in that give the IP address 192.168.11.108 which is from the output given above . Then the client will communicate only with this IP address . Then you can try backup .

Regards,
Rajesh

[Rajesh_s1]

HI ,

One more clarification from the bptestbpcd -client command output i can able to see the client name is same as master server name .

Can u plz tell me the master server name , because master and client names are showing the similar names.


Regards,
Rajesh

[pretorian]

Hi Marianne,

host netbackup-srv: netbackup-srv.xxxxxx.com at 1.1.1.99 (0x63010101)  --> IP standby (still active) and never used
host netbackup-srv: netbackup-srv.xxxxxx.com at 1.4.0.2 (0x2000401) --> Line transfer backup from agent or client nbu (LAN transfer backup)
host netbackup-srv: netbackup-srv.xxxxxx.com at 192.168.11.108 (0x6c0ba8c0) --> IP for line copy sharing data and for line remote assistence

and Rajesh,
You are correct. nbu server and nbu client have same hostname (netbackup-srv).

Before I upgrade from version 6.5 to 7. Job error never happened like this. Whether this is a bug from netbackup 7 ?

Regards,

Abbas

[Marianne · Accepted solution]

I have seen this causing issues since NBU 3.1 days.
You can see that the 1st IP address that is returned for hostname netbackup-srv is 1.1.1.99. This IP is returned by TCP/IP and NetBackup tries to use it.
Follow Rajesh's advice and add hosts entries. Give another name in the hosts file to the standby IP - something like
1.1.1.99 netbackup-srv-sb
1.4.0.2 netbackup-srv
192.168.11.108 netbackup-srv-remote

Only assign the netbackup-srv name to the IP that must be used for outgoing and incoming NBU comms.

[pretorian]

Ok. I try follow Rajesh advice. and send the result.
Thanks for your quick respond.

Regards,

Abbas