Backup failing with error 41(network connection timed out)

[Planters]

Hi,

One of my server backup is getting failed with error 'network connection timed out(41). Some times this backup is getting completed. I know that the issue is related with network but i have a doubt, why the job "exited with status 150: termination requested by administrator" i didnt cancelled this jobs at all. Please find the detail jobs below.

Netbackup version: 7.1

Client server is windows 2000

 5/21/2012 7:50:23 PM - Info nbjm(pid=3648) starting backup job (jobid=1583288) for client USEAIHN133B, policy NT_USEAIHN133_SYSTEM_Test, schedule Weekly_Full 
5/21/2012 7:50:23 PM - Info nbjm(pid=3648) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=1583288, request id:{95E46826-310B-4B59-8078-6198B4477EC5}) 
5/21/2012 7:50:23 PM - requesting resource ISEAN403_404
5/21/2012 7:50:23 PM - requesting resource isean403.NBU_CLIENT.MAXJOBS.USEAIHN133B
5/21/2012 7:50:23 PM - requesting resource isean403.NBU_POLICY.MAXJOBS.NT_USEAIHN133_SYSTEM_Test
5/21/2012 7:50:23 PM - granted resource isean403.NBU_CLIENT.MAXJOBS.USEAIHN133B
5/21/2012 7:50:23 PM - granted resource isean403.NBU_POLICY.MAXJOBS.NT_USEAIHN133_SYSTEM_Test
5/21/2012 7:50:23 PM - granted resource BMC676
5/21/2012 7:50:23 PM - granted resource HP.ULTRIUM3-SCSI.012
5/21/2012 7:50:23 PM - granted resource isean403-hcart3-robot-tld-0
5/21/2012 7:50:23 PM - estimated 1143919240 Kbytes needed
5/21/2012 7:50:23 PM - Info nbjm(pid=3648) started backup job for client USEAIHN133B, policy NT_USEAIHN133_SYSTEM_Test, schedule Weekly_Full on storage unit isean403-hcart3-robot-tld-0
5/21/2012 7:50:26 PM - Info bpbrm(pid=7896) USEAIHN133B is the host to backup data from    
5/21/2012 7:50:26 PM - Info bpbrm(pid=7896) telling media manager to start backup on client    
5/21/2012 7:50:26 PM - Info bptm(pid=1468) using 262144 data buffer size       
5/21/2012 7:50:26 PM - Info bptm(pid=1468) using 64 data buffers        
5/21/2012 7:50:30 PM - Info bpbrm(pid=3924) sending bpsched msg: CONNECTING TO CLIENT FOR USEAIHN133B_1337629823    
5/21/2012 7:50:30 PM - connecting
5/21/2012 7:50:32 PM - Info bptm(pid=5552) setting receive network buffer to 1049600 bytes     
5/21/2012 7:50:32 PM - Info bpbrm(pid=3924) start bpbkar32 on client        
5/21/2012 7:50:32 PM - connected; connect time: 00:00:02
5/21/2012 7:50:32 PM - begin writing
5/21/2012 7:50:33 PM - Info bpbkar32(pid=0) Backup started          
5/21/2012 7:50:33 PM - Info bpbrm(pid=3924) Sending the file list to the client     
5/21/2012 7:55:33 PM - Info bpbrm(pid=7896) child done, status 41        
5/21/2012 7:55:33 PM - Info bpbrm(pid=7896) sending message to media manager: STOP BACKUP USEAIHN133B_1337629823    
5/21/2012 7:55:34 PM - Info bpbrm(pid=7896) media manager for backup id USEAIHN133B_1337629823 exited with status 150: termination requested by administrator
5/21/2012 7:55:34 PM - end writing; write time: 00:05:02
network connection timed out(41)

[revarooo]

Look at this in the output from above:

5/21/2012 7:50:33 PM - Info bpbrm(pid=3924) Sending the file list to the client     

5/21/2012 7:55:33 PM - Info bpbrm(pid=7896) child done, status 41   

That is exactly 5 minutes. I am betting you are hitting a CLIENT_READ_TIMEOUT which is by default set to 300 (5 minutes). This means it is taking the client more than 5 minutes to send the data to the media serrver so the media server gives up. Try changing the CLIENT_READ_TIMEOUT:

Check this technote, it mentions changing the setting on the Master server but you should change it on the media server doing the backup. It explains how to change it. If the media server is Unix/Linux, edit the /usr/openv/netbackup/bp.conf and at the bottom of this file add CLIENT_READ_TIMEOUT = 1800 

http://www.symantec.com/business/support/index?page=content&id=HOWTO50656

Increase it to 1800 and wait 30 minutes after initiating the backup - hopefully it will actually do something before that time elapses.

[Planters]

Thanks for your response.

I have already increased the client read time out to 5000 again getting the same error. i think some thing related with media server and client server. Mainly network related only, but need to clarify.

[revarooo]

Sounds very likely it's network related. What happens if you try backing up the client to a different media server?

also when this backup of this client does fail to this media server, what is logged in the clients netbackup\logs\bpbkar\ log ?

[Planters · Accepted solution]

Hi Revaroo,

Thanks for your support. I have found out the reason why it is failed. In this policy backup selection have 4 streams. But in the policy "Allow multiple data streams is not enabled".

I have checked "Allow multiple data streams" and the backup is happing fine.