Forum Discussion

jos_hua's avatar
jos_hua
Level 3
5 years ago

VM backup failing with (11) system call failed

My nbu v8.1.2 daily backups of VMware & MSSQL polices have been running successfully.

Suddenly, I had encountered the below error only for VMware policy:

Error bptm cannot add fragment to image database, error = cannot connect on socket
(11) system call failed

My MSSQL backups are still running successfully.

i had checked firewall rules are already opened.

tcp 443 & 902 for nbu master/media & vmware vcenter & ESX hosts
- nbu master/media able to telnet vcenter tcp port 443 & ESX tcp port 902

tcp 1556 & 13724 for nbu master/media
- able to telnet nbu master/media tcp ports 1556/13724

I had searched the knowledgebase and tried the technotes.

https://www.veritas.com/support/en_US/article.100001950
https://www.veritas.com/support/en_US/article.100013415
https://vox.veritas.com/t5/NetBackup/cannot-connect-on-socket-25/td-p/620192
https://vox.veritas.com/t5/NetBackup/My-FS-Bakup-is-failing-with-25/td-p/525005
https://vox.veritas.com/t5/NetBackup/VMware-Backups-fails-with-status-11/m-p/342301#M69430

Does anyone have similiar error and managed to resolve?

Regards
Joshua

8.1.2
Windows Server 2016
  • jos_hua's avatar
    jos_hua
    5 years ago

    Marianne 

    Unfortunately my veritas support had lapsed recently. :smileysad:

    But good news! I did the following steps to resolve the backup issue. :smileyhappy:

    1) Deactivate the CDPUserSvc service via Registry Editor

    • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\CDPUserSvc

    2) Set the value Start to 4

    3) Invole an administrative command prompt window and enter the following command:

    • sc config cdpusersvc type=own

    4) Reboot the both NBU Master & Media servers

    5) Check that CDPUserSvc service is no longer running via Services control panel applet.

    It seems the KB4048953 from Microsoft Windows Update had caused my NBU services to go haywire.

  • jos_hua's avatar
    jos_hua
    Level 3
    5 years ago

    I checked the logs in bprd folder and noticed this error in the log:

    00:01:52.255 [15764.15204] <2> emmlib_QueryAppCredentials: (0) EMM_AppCredentials_Record => { MediaServerName => vcenter.abc.net AppServerName => vcenter.abc.net AppServerType => 10 (CEMM_NBU_VIRTUAL_MACHINE) AppServerSubType => 1 PortNumber => 443 UserName => nbuadmin Password => ****** PasswordKey => ****** CredentialType => 1563766176 (INVALID_CREDENTIAL_TYPE) SubTypeName => DisplayName =>

    However, i could login to my vcenter using the same account & password and browse the vcenter to select additional VMs to add to my VMware_Backup policy.

    • Marianne's avatar
      Marianne
      Level 6
      5 years ago

      The line that you saw in bprd is probably a 'red herring'. Plus it is a <2> - debug, not a warning <8>, error <16> or severe error <32>.

      I would start with with bpbrm log on the media server - level 3.
      This looks like comms error between bpbrm on media server and bpdbm on the master:

      ... cannot add fragment to image database, error = cannot connect on socket

      • jos_hua's avatar
        jos_hua
        Level 3
        5 years ago

        Hi Marianne,

        I have tried to filter thru both logs for <16> & <32> : bpbrm from the media server and bpdbm from the master

        I search the knowledgebase for solution on "The proxy protocol has been violated."

        https://www.veritas.com/support/en_US/article.100043072

        I checked my Netbackup Client Service was as per default set to start as LocalSystem and running.

        bpdbm log (master server)

        11:34:10.840 [7532.7860] <16> dump_proxy_info: statusmsg: The proxy protocol has been violated. The remote connector of connection ([local] [AF_INET:[10.20.34.38]:1556] -- [remote] [AF_INET:[10.20.36.21]:58215]) is not correctly using the proxy., nbu status = 7647, severity = 2, Additional Message: [PROXY] Encountered error (CERT_PROTOCOL_READING_JSON_LENGTH) while processing(CertProtocol)., nbu status = 2, severity = 1

        bpbrm log (media server)

        10:41:07.906 [11364.11332] <32> bpbrm Exit: unexpected termination of client cute_cats.abc.net
        10:41:07.906 [11364.11332] <2> set_job_details: Tfile (219762): LOG 1603680067 32 bpbrm 11364 unexpected termination of client cute_cats.abc.net
        10:41:07.906 [11364.11332] <2> send_job_file: job ID 219762, ftype = 3 msg len = 91, msg = LOG 1603680067 32 bpbrm 11364 unexpected termination of client cute_cats.abc.net
        10:41:07.906 [11364.11332] <2> ConnectionCache::connectAndCache: Acquiring new connection for host masterserver01.abc.net, query type 1

         

        Regards

        Terence