Forum Discussion

jos_hua's avatar
jos_hua
Level 3
4 years ago

VM backup failing with (11) system call failed

My nbu v8.1.2 daily backups of VMware & MSSQL polices have been running successfully. Suddenly, I had encountered the below error only for VMware policy: Error bptm cannot add fragment to image dat...
8.1.2
Windows Server 2016
  • jos_hua's avatar
    jos_hua
    4 years ago

    Marianne 

    Unfortunately my veritas support had lapsed recently. :smileysad:

    But good news! I did the following steps to resolve the backup issue. :smileyhappy:

    1) Deactivate the CDPUserSvc service via Registry Editor

    • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\CDPUserSvc

    2) Set the value Start to 4

    3) Invole an administrative command prompt window and enter the following command:

    • sc config cdpusersvc type=own

    4) Reboot the both NBU Master & Media servers

    5) Check that CDPUserSvc service is no longer running via Services control panel applet.

    It seems the KB4048953 from Microsoft Windows Update had caused my NBU services to go haywire.

jos_hua's avatar
jos_hua
Level 3

I checked the logs in bprd folder and noticed this error in the log:

00:01:52.255 [15764.15204] <2> emmlib_QueryAppCredentials: (0) EMM_AppCredentials_Record => { MediaServerName => vcenter.abc.net AppServerName => vcenter.abc.net AppServerType => 10 (CEMM_NBU_VIRTUAL_MACHINE) AppServerSubType => 1 PortNumber => 443 UserName => nbuadmin Password => ****** PasswordKey => ****** CredentialType => 1563766176 (INVALID_CREDENTIAL_TYPE) SubTypeName => DisplayName =>

However, i could login to my vcenter using the same account & password and browse the vcenter to select additional VMs to add to my VMware_Backup policy.

Marianne's avatar
Marianne
Level 6
4 years ago

The line that you saw in bprd is probably a 'red herring'. Plus it is a <2> - debug, not a warning <8>, error <16> or severe error <32>.

I would start with with bpbrm log on the media server - level 3.
This looks like comms error between bpbrm on media server and bpdbm on the master:

... cannot add fragment to image database, error = cannot connect on socket

  • jos_hua's avatar
    jos_hua
    Level 3
    4 years ago

    Hi Marianne,

    I have tried to filter thru both logs for <16> & <32> : bpbrm from the media server and bpdbm from the master

    I search the knowledgebase for solution on "The proxy protocol has been violated."

    https://www.veritas.com/support/en_US/article.100043072

    I checked my Netbackup Client Service was as per default set to start as LocalSystem and running.

    bpdbm log (master server)

    11:34:10.840 [7532.7860] <16> dump_proxy_info: statusmsg: The proxy protocol has been violated. The remote connector of connection ([local] [AF_INET:[10.20.34.38]:1556] -- [remote] [AF_INET:[10.20.36.21]:58215]) is not correctly using the proxy., nbu status = 7647, severity = 2, Additional Message: [PROXY] Encountered error (CERT_PROTOCOL_READING_JSON_LENGTH) while processing(CertProtocol)., nbu status = 2, severity = 1

    bpbrm log (media server)

    10:41:07.906 [11364.11332] <32> bpbrm Exit: unexpected termination of client cute_cats.abc.net
    10:41:07.906 [11364.11332] <2> set_job_details: Tfile (219762): LOG 1603680067 32 bpbrm 11364 unexpected termination of client cute_cats.abc.net
    10:41:07.906 [11364.11332] <2> send_job_file: job ID 219762, ftype = 3 msg len = 91, msg = LOG 1603680067 32 bpbrm 11364 unexpected termination of client cute_cats.abc.net
    10:41:07.906 [11364.11332] <2> ConnectionCache::connectAndCache: Acquiring new connection for host masterserver01.abc.net, query type 1

     

    Regards

    Terence

    • Marianne's avatar
      Marianne
      Level 6
      4 years ago

      jos_hua 

      You need to find log entries that correspond with this timestamp:

      23 Oct, 2020 8:26:31 cannot add fragment to image database, error = cannot connect on socket

      Best to locate the PID in Job details, then extract all entries for that PID.
      Among the bpbrm entries, there will be attemp to connect to bpbdm.

      If you cannot post full logs, please log a call with Veritas Support and submit logs to them.