The backup failed to back up the requested files

[msandrosov59]

Hi!

I'm running netbackup 8.1.1.

This message only appears for one vm backup. I can backup another vm on the same datastore without a problem. 

Mar 19, 2024 12:52:35 AM - Info nbjm (pid=6784) starting backup job (jobid=38578) for client tp-gmp-budgetpl2, policy Infrostructure_OKVS, schedule Inc_dayly
Mar 19, 2024 12:52:35 AM - Info nbjm (pid=6784) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=38578, request id:{1D0A6BA5-5AF6-432B-A6BA-16984ED1DA31})
Mar 19, 2024 12:52:35 AM - requesting resource  Robots
Mar 19, 2024 12:52:35 AM - requesting resource  srv-nbu01i.msa.com.NBU_CLIENT.MAXJOBS.tp-gmp-budgetpl2
Mar 19, 2024 12:52:35 AM - requesting resource  srv-nbu01i.msa.com.NBU_POLICY.MAXJOBS.Infrostructure_OKVS
Mar 19, 2024 12:52:35 AM - granted resource  srv-nbu01i.msa.com.NBU_CLIENT.MAXJOBS.tp-gmp-budgetpl2
Mar 19, 2024 12:52:35 AM - granted resource  srv-nbu01i.msa.com.NBU_POLICY.MAXJOBS.Infrostructure_OKVS
Mar 19, 2024 12:52:35 AM - granted resource  T369L5
Mar 19, 2024 12:52:35 AM - granted resource  IBM.ULT3580-TD5.000
Mar 19, 2024 12:52:35 AM - granted resource  srv-nbu01i-hcart2-robot-tld-0
Mar 19, 2024 12:52:35 AM - estimated 31849843 kbytes needed
Mar 19, 2024 12:52:35 AM - Info nbjm (pid=6784) started backup (backupid=tp-gmp-budgetpl2_1710798755) job for client tp-gmp-budgetpl2, policy Infrostructure_OKVS, schedule Inc_dayly on storage unit srv-nbu01i-hcart2-robot-tld-0 using backup host srv-nbu01i.msa.com
Mar 19, 2024 12:52:35 AM - started process bpbrm (pid=10660)
Mar 19, 2024 12:52:36 AM - connecting
Mar 19, 2024 12:52:36 AM - connected; connect time: 0:00:00
Mar 19, 2024 12:52:36 AM - Info bpbrm (pid=10660) tp-gmp-budgetpl2 is the host to backup data from
Mar 19, 2024 12:52:36 AM - Info bpbrm (pid=10660) reading file list for client
Mar 19, 2024 12:52:36 AM - Info bpbrm (pid=10660) starting bpbkar32 on client
Mar 19, 2024 12:52:39 AM - Info bpbkar32 (pid=11024) Backup started
Mar 19, 2024 12:52:39 AM - Info bpbkar32 (pid=11024) archive bit processing:<enabled>
Mar 19, 2024 12:52:39 AM - mounting T369L5
Mar 19, 2024 12:52:39 AM - Info bptm (pid=7672) start
Mar 19, 2024 12:52:39 AM - Info bptm (pid=7672) using 262144 data buffer size
Mar 19, 2024 12:52:39 AM - Info bptm (pid=7672) using 256 data buffers
Mar 19, 2024 12:52:39 AM - Info bptm (pid=7672) start backup
Mar 19, 2024 12:52:39 AM - Info bptm (pid=7672) Waiting for mount of media id T369L5 (copy 1) on server srv-nbu01i.msa.com.
Mar 19, 2024 12:52:42 AM - Info bpbkar32 (pid=11024) INF - Backing up vCenter server srv-vcenter02.dmzmsa.com, ESX host esx54.dmzmsa.com, BIOS UUID 4209da86-5982-57a6-0437-3af5ee488116, Instance UUID 50097925-6f1c-fa6e-507d-d31fdd6efa7b, Display Name tp-gmp-budgetpl2, Hostname tp-gmp-budgetpl.dmzmsa.com
Mar 19, 2024 12:53:30 AM - Info bptm (pid=7672) media id T369L5 mounted on drive index 0, drivepath {10,0,0,0}, drivename IBM.ULT3580-TD5.000, copy 1
Mar 19, 2024 12:53:30 AM - mounted T369L5; mount time: 0:00:51
Mar 19, 2024 12:53:30 AM - positioning T369L5 to file 77
Mar 19, 2024 12:54:34 AM - positioned T369L5; position time: 0:01:04
Mar 19, 2024 12:54:34 AM - begin writing
Mar 19, 2024 1:01:28 AM - Error bpbrm (pid=10660) from client tp-gmp-budgetpl2: ERR - Error opening the snapshot disks using given transport mode: san:hotadd:nbd:nbdssl Status 23
Mar 19, 2024 1:01:29 AM - Info bpbkar32 (pid=11024) bpbkar waited 0 times for empty buffer, delayed 0 times.
Mar 19, 2024 1:01:35 AM - Error bpbrm (pid=10660) could not send server status message to client
Mar 19, 2024 1:01:37 AM - Critical bpbrm (pid=10660) unexpected termination of client tp-gmp-budgetpl2
Mar 19, 2024 1:01:37 AM - Info bpbkar32 (pid=0) done. status: 6: the backup failed to back up the requested files
Mar 19, 2024 1:01:37 AM - end writing; write time: 0:07:03
the backup failed to back up the requested files  (6)

Two days ago, the backup of this virtual machine was performed without problems. No changes were made to the policy settings.

[Michal_Mikulik1]

Hello,

check VMware side (Tasks/Events for the failing VM), a root cause must be there.

Also try to search history of this forum, issues like this were being solved many times here.

M.

[msandrosov59]

Everything I see in the VMware log:

‎03‎/‎19‎/‎2024‎ ‎1‎:‎01‎:‎44‎ ‎AM NetBackup: Backup failed for tp-gmp-budgetpl2. (Job 38575; Client srv-nbu01i.msa.com; Master srv-nbu01i.msa.com; Policy Infrostructure_OKVS; Schedule Inc_dayly; Type INCR; Duration 7 hrs 4 min 5 sec; Status 6)

Related events:

There are no related events.

[Hamza_H · Accepted solution]

Hi,
This is a transport mode issue, I can see that all of the 4 types are selected, which transport was working with before? If it is san, check if lun volumes are accessible for the backup host, if nbd, check communication over port 902 between esx and backup host as well as name resolution.
If you are sure there is no error there, the it might be a permission issues for the user used in vcenter.
The easiest way to find out the root cause, is enabling vxms logs on backup host.
Kind regards

[Slartybardfast]

As explained by Hamza_H this is a transport issue and the vmxs log will identify the issue. By the way the version you are using EOSL on 28/06/2023. I would recommend that you upgrade as soon as possible, you will not be able to raise a case with Veritas if you need to in the future.