01-18-2019 01:48 AM
Hello Team,
Backups are getting failed with error code 84 for few of the clients.
Detailed status of the job as below
Jan 16, 2019 6:30:02 PM - Info nbjm (pid=39108) starting backup job (jobid=1190904) for client citisft-lx-usswdc-pr3-st2, policy tx931_prd_mo_net_std_linux_su_crebm4700_grp_1, schedule 1m_si_incr
Jan 16, 2019 6:30:02 PM - Info nbjm (pid=39108) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=1190904, request id:{A5CB351E-19E6-11E9-ADF6-FD182554582B})
Jan 16, 2019 6:30:02 PM - requesting resource CREBM4700_FS
Jan 16, 2019 6:30:02 PM - requesting resource crebm4700.NBU_CLIENT.MAXJOBS.citisft-lx-usswdc-pr3-st2
Jan 16, 2019 6:30:02 PM - requesting resource crebm4700.NBU_POLICY.MAXJOBS.tx931_prd_mo_net_std_linux_su_crebm4700_grp_1
Jan 16, 2019 6:30:12 PM - awaiting resource CREBM4700_FS. Maximum job count has been reached for the storage unit.
Jan 16, 2019 7:39:06 PM - granted resource crebm4700.NBU_CLIENT.MAXJOBS.citisft-lx-usswdc-pr3-st2
Jan 16, 2019 7:39:06 PM - granted resource crebm4700.NBU_POLICY.MAXJOBS.tx931_prd_mo_net_std_linux_su_crebm4700_grp_1
Jan 16, 2019 7:39:06 PM - granted resource MediaID=@aaaab;DiskVolume=ddswdccsp01_crebm4700_lsu1;DiskPool=crebm4700_disk_lsu1_ddswdccsp01;Path=ddswdccsp01_crebm4700_lsu1;StorageServer=ddswdccsp01;MediaServer=crems4706
Jan 16, 2019 7:39:06 PM - granted resource crebm4700_disk_lsu1_ddswdccsp01
Jan 16, 2019 7:39:06 PM - estimated 23620046 kbytes needed
Jan 16, 2019 7:39:06 PM - Info nbjm (pid=39108) started backup (backupid=citisft-lx-usswdc-pr3-st2_1547685546) job for client citisft-lx-usswdc-pr3-st2, policy tx931_prd_mo_net_std_linux_su_crebm4700_grp_1, schedule 1m_si_incr on storage unit crebm4700_disk_lsu1_ddswdccsp01
Jan 16, 2019 7:39:07 PM - Info bpbrm (pid=52317) citisft-lx-usswdc-pr3-st2 is the host to backup data from
Jan 16, 2019 7:39:07 PM - Info bpbrm (pid=52317) reading file list for client
Jan 16, 2019 7:39:07 PM - started process bpbrm (pid=52317)
Jan 16, 2019 7:39:08 PM - Info bpbrm (pid=52317) starting bpbkar on client
Jan 16, 2019 7:39:08 PM - Info bpbkar (pid=43742) Backup started
Jan 16, 2019 7:39:08 PM - Info bpbrm (pid=52317) bptm pid: 52813
Jan 16, 2019 7:39:08 PM - Info bptm (pid=52813) start
Jan 16, 2019 7:39:08 PM - connecting
Jan 16, 2019 7:39:08 PM - connected; connect time: 0:00:00
Jan 16, 2019 7:39:09 PM - Info bptm (pid=52813) using 524288 data buffer size
Jan 16, 2019 7:39:09 PM - Info bptm (pid=52813) setting receive network buffer to 393216 bytes
Jan 16, 2019 7:39:09 PM - Info bptm (pid=52813) using 256 data buffers
Jan 16, 2019 7:39:10 PM - Info bptm (pid=52813) start backup
Jan 16, 2019 7:39:12 PM - Info bptm (pid=52813) backup child process is pid 53124
Jan 16, 2019 7:39:12 PM - begin writing
Jan 17, 2019 12:04:17 PM - Critical bptm (pid=52813) image write failed: error 2060046: plugin error
Jan 17, 2019 12:04:17 PM - Critical bptm (pid=52813) sts_get_image_prop failed: error 2060046: plugin error
Jan 17, 2019 12:04:17 PM - Critical bptm (pid=52813) sts_delete_image of image citisft-lx-usswdc-pr3-st2_1547685546_C1_F1 failed: error 2060046 plugin error
Jan 17, 2019 12:04:17 PM - Critical bptm (pid=52813) image delete failed: error 2060046: plugin error
Jan 17, 2019 12:05:46 PM - Error bptm (pid=52813) cannot write image to disk, Invalid argument
Jan 17, 2019 12:05:46 PM - Info bptm (pid=52813) EXITING with status 84 <----------
Jan 17, 2019 12:06:26 PM - Info bpbkar (pid=43742) done. status: 84: media write error
Jan 17, 2019 12:06:26 PM - end writing; write time: 16:27:14
media write error (84)
Also checked the bptm logs on media server and found below error. Please assist
04:14:32.912 [100675] <16> 1194358:bptm:100675:crems4706: [18943:1071510] ddcl_nfs_PLU_query_with_key(): OST secure query RPC failed with 5028
(DDBoost OST_QUERY_SECURE RPC not available 10) -- RPC error 10; falling back to legacy RPC
04:14:32.912 [100675] <16> 1194358:bptm:100675:crems4706: [18943:1071510] ddcl_PLU_generate_psk_key(): Using legacy non-PSK authentication mec
hanism
Solved! Go to Solution.
01-22-2019 03:49 AM
Thanks Marianne,
The Issue got resolved after increasing the timeout value on DD
01-18-2019 07:06 AM
Please see if you can find the bptm entries for (pid=52813) on media server crems4706.
This is the failing PID that we see in Job Details.
The PID that you found [100675] points to an issue with comms to verify DD credentials.
Maybe a good idea to re-add DD credentials on this media server and double-check firewall settings between media server and DD.
01-22-2019 03:49 AM
Thanks Marianne,
The Issue got resolved after increasing the timeout value on DD
11-23-2021 09:20 PM
please advise which command you use to increasing the timeout value on DD
i am facing same problem
01:23:15.955 [25551] <2> Orb::init: caching EndpointSelectorFactory(Orb.cpp:1164)
01:23:15.955 [25551] <2> Orb::setOrbConnectTimeout: timeout seconds: 60(Orb.cpp:1668)
01:23:15.955 [25551] <2> Orb::setOrbRequestTimeout: timeout seconds: 1800(Orb.cpp:1677)
01:23:15.955 [25551] <2> ProxyConnector::delegateConnect: checking reverse connect for host [nbu-juf-master.iga.gov.bh] and host IP is [10.100.3.1](ProxyConnector.cpp:56)
01:23:15.955 [25551] <2> ProxyConnector::delegateConnect: Reverse connection disabled, PBX direct connect will be tried(ProxyConnector.cpp:92)
01:23:15.955 [25551] <2> TAO: PBXIOP (25551|25551) - PBXIOP_Connector::make_connection, proxy connector found
01:23:15.955 [25551] <2> TAO: PBXIOP (25551|25551) - PBXIOP_Connector::make_connection, will try plain connect
01:23:15.980 [25551] <2> Orb::destroyOrb: destroying Orb [EMMlib_Orb](Orb.cpp:1877)
01:23:15.982 [25551] <2> debuglog: <16> sts_log: nbfsd_irp (4): [63CF:206B9F0] ddcl_nfs_PLU_query_with_key(): OST secure query RPC failed with 5028 (unrecoverable rpc failure for procedure=OST_QUERY_SECURE, retval=10 on ip=10.100.2.1) -- RPC error 10; falling back to legacy RPC
01:23:15.982 [25551] <2> debuglog: <16> sts_log: nbfsd_irp (4): [63CF:206B9F0] ddcl_PLU_generate_psk_key(): Using legacy non-PSK authentication mechanism
01:23:16.013 [25551] <2> debuglog: <8> sts_log: nbfsd_irp (2): [63CF:206B9F0] NFS reconnection with IFGROUP IP
01:23:16.014 [25551] <2> debuglog: <16> sts_log: nbfsd_irp (4): [63CF:206B9F0] ddcl_nfs_PLU_query_with_key(): OST secure query RPC failed with 5028 (DDBoost OST_QUERY_SECURE RPC not available 10 ) -- RPC error 10; falling back to legacy RPC
01:23:16.014 [25551] <2> debuglog: <16> sts_log: nbfsd_irp (4): [63CF:206B9F0] ddcl_PLU_generate_psk_key(): Using legacy non-PSK authentication mechanism
01:23:39.632 [21263] <2> ConnectionCache::connectAndCache: Acquiring new connection for host nbu-juf-master.iga.gov.bh, query type 79
01:23:39.633 [21263] <2> vnet_pbxConnect_ex: pbxConnectExEx Succeeded
01:23:39.646 [21263] <2> logconnections: PROXY CONNECT FROM 10.100.3.2.52467 TO 10.100.3.1.1556 fd = 31
01:23:39.646 [21263] <2> logconnections: BPDBM CONNECT FROM 127.0.0.1.58475 TO 127.0.0.1.60796 fd = 31
01:23:39.740 [21263] <2> closeConnection: Caching connection for query type 79 for reuse
01:2