hello Guys,2 of my clients started failing recently with error in job details as below Feb 19, 2021 8:57:58 PM - started process bpbrm (pid=72139)Feb 19, 2021 9:58:09 PM - Error bpbrm (pid=72139) [PROXY] Connecting host: frontier123Feb 19, 2021 9:58:09 PM - Error bpbrm (pid=72139) [PROXY] ConnectionId: {0E8CEB40-731F-11EB-B0C8-19FDBA47E32D}:OUTBOUNDFeb 19, 2021 9:58:09 PM - Error bpbrm (pid=72139) [PROXY] pid: 19555Feb 19, 2021 9:58:09 PM - Error bpbrm (pid=72139) [PROXY] Received status: 41 with message The requested I/O operation has timed out.Feb 19, 2021 9:58:09 PM - Error bpbrm (pid=72139) [PROXY] Encountered error (CERT_PROTOCOL_READING_JSON_LENGTH) while processing(CertProtocol).Feb 19, 2021 9:58:09 PM - Error bpbrm (pid=72139) bpcd on Frontier123 exited with status 41: network connection timed out nbU version s 8.1.2 on allClient is Windows 2016Master/media NBU 5240

In all honesty - the vast amount of hostnames for the one client is confusing me... may NBU as well... It also seems as if the client has 2 IP addresses with the same hostname? host DMC-FRONTIERHTZ: DMC-FrontierHTZ.dmcnt1.local at 155.139.194.179host DMC-FRONTIERHTZ: DMC-FrontierHTZ.dmcnt1.local at 192.168.10.104 At least the uppercase hostname is consistent on client and master - DMC-FRONTIERHTZWith same IP address used for comms between the master and client: 155.139.194.179If you use this hostname consistently everywhere, at least hostname lookup should be fine. What you really need to troubleshoot is the fact that connection request never reached the client (as seen in yesterday's bpcd log). Something that you did not post with bptestbpcd output is the 1st line that shows the connect options.This will tell you/us how NBU is trying to connect. (See https://www.veritas.com/support/en_US/doc/15263389-127350397-0/v14667804-127350397) You can then try to involve network, server and/or security admin to find out if something in the environment or on the client changed that could block NBU connection attempts (e.g. Windows Firewall or AV software).

Hey can you ping from master these two clients and vice - versa? How about telnet from master to bpcd port on clients?

yea it does backup1:/home/maintenance # telnet dmc-frontierhtz bpcdTrying 155.139.194.179...Connected to dmc-frontierhtz.Escape character is '^]'.^CConnection closed by foreign host.backup1:/home/maintenance # telnet dmc-frontier123.dmc.org bpcdTrying 155.139.194.141...Connected to dmc-frontier123.dmc.org.Escape character is '^]'.^CConnection closed by foreign host.backup1:/home/maintenance #backup1:/usr/openv/netbackup/bin/support/output/nbsu # ping dmc-frontierhtzPING frontierhtz.dmc.org (155.139.194.179) 56(84) bytes of data.64 bytes from frontierhtz.dmc.org (155.139.194.179): icmp_seq=1 ttl=127 time=0.602 ms64 bytes from frontierhtz.dmc.org (155.139.194.179): icmp_seq=2 ttl=127 time=0.400 ms^C--- frontierhtz.dmc.org ping statistics ---2 packets transmitted, 2 received, 0% packet loss, time 1001msrtt min/avg/max/mdev = 0.400/0.501/0.602/0.101 msbackup1:/usr/openv/netbackup/bin/support/output/nbsu # ping dmc-frontier123PING dmc-frontier123.dmc.org (155.139.194.141) 56(84) bytes of data.64 bytes from dmc-frontier123.dmc.org (155.139.194.141): icmp_seq=1 ttl=127 time=0.673 ms64 bytes from dmc-frontier123.dmc.org (155.139.194.141): icmp_seq=2 ttl=127 time=0.581 ms64 bytes from dmc-frontier123.dmc.org (155.139.194.141): icmp_seq=3 ttl=127 time=0.743 ms

OK so these are reachable... What is bpbrm log telling for this job: grep 72139 /usr/openv/netbackup/logs/bpbrm/log_from_job_date increase verbosity to 3 and retry if above is clueless.

I would also have enabled logging for bpcd on client end and possible bpbkar...

Agent Backup failing|bptestbpcd hung

12 Replies

quebek
Moderator
5 years ago
Hey

can you ping from master these two clients and vice - versa? How about telnet from master to bpcd port on clients?
- fazhar
  Level 3
  5 years ago
  yea it does
  
  backup1:/home/maintenance # telnet dmc-frontierhtz bpcd
  Trying 155.139.194.179...
  Connected to dmc-frontierhtz.
  Escape character is '^]'.
  ^CConnection closed by foreign host.
  backup1:/home/maintenance # telnet dmc-frontier123.dmc.org bpcd
  Trying 155.139.194.141...
  Connected to dmc-frontier123.dmc.org.
  Escape character is '^]'.
  ^CConnection closed by foreign host.
  backup1:/home/maintenance #
  
  backup1:/usr/openv/netbackup/bin/support/output/nbsu # ping dmc-frontierhtz
  PING frontierhtz.dmc.org (155.139.194.179) 56(84) bytes of data.
  64 bytes from frontierhtz.dmc.org (155.139.194.179): icmp_seq=1 ttl=127 time=0.602 ms
  64 bytes from frontierhtz.dmc.org (155.139.194.179): icmp_seq=2 ttl=127 time=0.400 ms
  ^C
  --- frontierhtz.dmc.org ping statistics ---
  2 packets transmitted, 2 received, 0% packet loss, time 1001ms
  rtt min/avg/max/mdev = 0.400/0.501/0.602/0.101 ms
  backup1:/usr/openv/netbackup/bin/support/output/nbsu # ping dmc-frontier123
  PING dmc-frontier123.dmc.org (155.139.194.141) 56(84) bytes of data.
  64 bytes from dmc-frontier123.dmc.org (155.139.194.141): icmp_seq=1 ttl=127 time=0.673 ms
  64 bytes from dmc-frontier123.dmc.org (155.139.194.141): icmp_seq=2 ttl=127 time=0.581 ms
  64 bytes from dmc-frontier123.dmc.org (155.139.194.141): icmp_seq=3 ttl=127 time=0.743 ms
  - quebek
    Moderator
    5 years ago
    OK so these are reachable...
    
    What is bpbrm log telling for this job:
    
    grep 72139 /usr/openv/netbackup/logs/bpbrm/log_from_job_date
    
    increase verbosity to 3 and retry if above is clueless.

Forum Discussion

Agent Backup failing|bptestbpcd hung

12 Replies

Related Content

Media server to Client - Bptestbpcd failed with 25

bptestbpcd cmd Connectivity from mediaserver to client Failed

backup container bptestbpcd failed: 7647

bptestbpcd socked failed 23

Oracle agent