the backup failed to back up the requested files(6)

[trakool]

Hi, Our enviroment we have master server version 7.5.0.5 in windows 2008r2 . Last week we are getting a error code 6 form NBU with the following error:

29/04/2017 2:29:29 AM - started process bpbrm (10656)
29/04/2017 2:29:30 AM - Info bpbrm(pid=10656) clientname is the host to backup data from
29/04/2017 2:29:30 AM - Info bpbrm(pid=10656) reading file list from client
29/04/2017 2:29:30 AM - connecting
29/04/2017 2:29:31 AM - Info bpbrm(pid=10656) listening for client connection
29/04/2017 2:29:37 AM - Info bpbrm(pid=10656) INF - Client read timeout = 300
29/04/2017 2:29:37 AM - Info bpbrm(pid=10656) accepted connection from client
29/04/2017 2:29:37 AM - Info bphdb(pid=2518) Backup started
29/04/2017 2:29:37 AM - Info bptm(pid=12760) start
29/04/2017 2:29:37 AM - Info bptm(pid=12760) using 1048576 data buffer size
29/04/2017 2:29:37 AM - Info bptm(pid=12760) setting receive network buffer to 4195328 bytes
29/04/2017 2:29:37 AM - Info bptm(pid=12760) using 30 data buffers
29/04/2017 2:29:37 AM - connected; connect time: 00:00:07
29/04/2017 2:29:38 AM - Info bptm(pid=12760) start backup
29/04/2017 2:29:39 AM - Info bptm(pid=12760) backup child process is pid 12392.13620
29/04/2017 2:29:39 AM - Info bptm(pid=12392) start
29/04/2017 2:29:39 AM - begin writing
29/04/2017 2:29:40 AM - Info bphdb(pid=2518) dbclient(pid=2518) wrote first buffer(size=262144)
29/04/2017 3:55:59 AM - Info bphdb(pid=2518) done. status: 6
29/04/2017 3:56:01 AM - Info bphdb(pid=2518) done. status: 6: the backup failed to back up the requested files
29/04/2017 3:56:01 AM - end writing; write time: 01:26:22
the backup failed to back up the requested files(6)

Attached is the DBClient log error

Could you please help me to resolve this?

and the error occurs for many time.

Thank you,

Trakool

[Tape_Archived]

Please share bpcd and bpbkar logs on the client. Make sure all the connection between client, master and media servers are correct. Check if all basic configuration setting like bp.conf is set right.

[Tousif]

Hello,

Sharing my finding from dbclient logs.

Error:
01:11:52.271 [2477] <16> readCommFile: ERR - timed out after 900 seconds while reading from /usr/openv/netbackup/logs/user_ops/dbext/logs/2477.
0.1493388024

Cause:
This message indicates that dbclient did not receive an expected update in the comm file before a timer expired.
The timeout is fatal if it occurs before the backup image transfer begins.

Solution:
    
The following items should be considered before adjusting the Client Read Timeout or Server Read Timeout settings.

 
1)     Determine the reason for the backup delay before adjusting timeouts, and also the maximum length of the observed delay.

2)     Whenever possible, resolve the root cause of the delay instead of increasing the timeouts; backup and restore operations will work more efficiently and consistently.

3)     A temporary increase to one of the timeouts may be useful to allow successful operations while performing root cause analysis.  Be sure to revert to the original timeout value after the root cause is resolved.

Note: The NBU 7.5.x, 7.6.x are EOSL. Kindly upgrade your NetBackup environment as soon as possible.

Regards,

[Marianne]

I found similar errors in this post:
https://vox.veritas.com/t5/NetBackup/Problem-backup-with-Oracle-11-on-Oracle-Linux/td-p/608436

It is a lengthy discussion - good idea to read through it and then see if the solution could also help in your situation.

[trakool]

hello, thanks for your reply. I have not to create the bpcd and bpbkar logs on the client. I ensure for the connection between client, master and media servers are correct because this error it has happen to only full backup in some time.

Regards,

Trakool

[Tousif]

Hello,

Thanks for the update.

As you said, its intermittent issue. I would recommend to increase the client timeout value to 3600 and check if it work. Most of the time network hiccup or busy client causes this problem.

If it not fix the issue then need to monitor the behavior of failure.

Regards,

[trakool]

hello, can i try to increase the client timeout value to 3600 with GUI (administration console) to change the properties of this client?

Now, master server is version 7.5.0.5 if not, Cloud you please provide me to detail.

Many thanks for your support

Trakool

[Tousif]

Hello,

Can i try to increase the client timeout value to 3600 with GUI (administration console) to change the properties of this client?

Yes, you can change client read timeout using GUI. The client read timeout should set on media server use to take backup of that client. Media server reflect value to client.

 Recommendation:

 Admin Console -> Host Properties -> Media Server -> Timeouts -> Client read timeout : 3600

 Admin Console -> Host Properties -> Problematic Client Server -> Timeouts -> Client read timeout : 3600

 Reference:

https://www.veritas.com/support/en_US/article.TECH28396

Regards

[trakool]

Sorry for reply very late.After I already try to increase the client timeout that I not found "readCommFile: ERR - timed out after 900 seconds while reading from /usr/openv/netbackup/logs/user_ops/dbext/logs/2477.

0.1493388024" 

But the last logs I am getting a similar error "writeToServer: ERR - send() to server on socket failed: Connection timed out (110)"

Please see more log in attached file.

thank for advance

Trakool

[trakool]

Sorry for reply very late.After I already try to increase the client timeout that I not found "readCommFile: ERR - timed out after 900 seconds while reading from /usr/openv/netbackup/logs/user_ops/dbext/logs/2477.

0.1493388024" 

But the last logs I am getting a similar error "writeToServer: ERR - send() to server on socket failed: Connection timed out (110)"

Please see more log in attached file.

thank for advance

Trakool

[Marianne]

Client Connect Timeout and Client Read Timeout should go on the master and media server.

Please copy full dbclient log to dbclient.txt as well as bptm and bpbrm on media server (as bptm.txt and bpbrm.txt) and upload here.
bprd on the master server will also help to see where the break in communication is.

[trakool]

Sorry for reply very late.After I already try to increase the client timeout that I not found "readCommFile: ERR - timed out after 900 seconds while reading from /usr/openv/netbackup/logs/user_ops/dbext/logs/2477.

0.1493388024"

But the last logs I am getting a similar error "writeToServer: ERR - send() to server on socket failed: Connection timed out (110)"

Please see more log in attached file.

Thank for advance

Trakool

[Tousif · Accepted solution]

Hello,

In provided log, it seems like client still have some issue with network.

Snippet:

05:34:48.890 [32058] <2> writeToServer: bytesSent = 133092 bytes
05:34:48.890 [32058] <2> writeToServer: start send -- try=1
05:34:48.890 [32058] <16> writeToServer: ERR - send() to server on socket failed: Connection timed out (110)
05:34:48.890 [32058] <16> dbc_put: ERR - failed sending data to server

Client able to send some amount of data but after some time it failed to send data to media server.

I would recommend to involve your OS and Network team to investigate. I don't think troubleshooting from NBU site will help to resolve this issue.

Regards, 

[trakool]

Hello, Marianne

Thank you for your support, But I still not create directory for bptm, bpbrm and bprd log on master server.

I will create that and after it has error on next time. I will reply again.

For full DBClient log, It has not open from text editor because file size is very big.

Best regards,

Trakool

[trakool]

Hello, Tousif

After monitoring network, I found the network usage is full bandwidth in during full backup and I try to fix this issue by adjust job policy to reduce network load. It seems to work.

Thank you for your suggestion

Trakool

[Tousif]

Hello Trakool,

Great to know that issue has fixed.

Please mark this thread as resolved. So It will not be in open list.

Thanks & Regards,