cancel
Showing results for 
Search instead for 
Did you mean: 

VM bakups failing with Error Code 11 Continuously

skrp
Level 4
Certified

Hi Team,

VM backups in our environment are failing with Error code 11 after writing some data. 

Master Server is 7.5.0.7. It acts as both Master and Media servers.

I've also checked the logs bpcd, bprd, vnetd but they don't show much information. Please guide me with this issue.

Hiere is the error from Netbackup Detailed Status.

14/07/2020 23:36:12 - positioned 2434L3; position time: 00:00:01
14/07/2020 23:36:12 - begin writing
15/07/2020 00:25:22 - Info bpbkar32(pid=15264) bpbkar waited 202 times for empty buffer, delayed 214 times.
15/07/2020 00:25:27 - Error bpbrm(pid=11572) could not send server status message
15/07/2020 00:25:27 - Critical bpbrm(pid=11572) unexpected termination of client KOSLOAPP001
15/07/2020 00:25:28 - Error bpbrm(pid=11572) cannot send mail to root on client ktslobak001.kt.group.local
15/07/2020 00:25:28 - Info bpbkar32(pid=0) done. status: 11: system call failed
15/07/2020 00:25:28 - end writing; write time: 00:49:16
system call failed(11)

 

9 REPLIES 9

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@skrp 

Have a look at the processes that are involved in this backup:
bpbrm
bpbkar32

So, the logs that you need to check first are bpbrm and bpbkar. 

skrp
Level 4
Certified

Hi Marianne,

Thanks for the update. I'm unable to open the file as it is too big. Can you suggest some editors please.

 

skrp
Level 4
Certified

Here are the logs(bpbrm and bpbkar) as an attachment. Below are the backups that are failing with this error.

KOSLOAPP001

KWSLOWBE005L

KWSLOWBE053

KWSLOWBE002

KWSLOWBE004L

Please help me with this as I'm unable to resolve issue with KOSLOAPP001 and couldn't secure a backup for this client since one month.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Try TextPad.

Hopefully your logging level is not set to 5?

I only enable this logging level if requested by Veritas Support.

Level 3 is normally fine for 99% of troubleshooting.

My 7.5.x way-back machine isn't working so well this morning. In current terminology, a VMware policy has a backup host or a restore host. I think it used to be called a proxy host. At any rate, that's where bpbkar32 executes.

Q1: In your case, does bpbkar32 execute on the same master/media server where bpbrm executes?

Q2: Do you see anything significant in the Windows application event log? Did bpbkar32 or another NetBackup process crash?

For Windows client processes such as bpfis, bpbkar32, and tar32, the important logging level is the one you configure under Windows client host properties. It should be set to 2, which is the max.

Q3: If bpbkar32 didn't crash, what errors do you find in its log? If bpbkar32 did crash, what are the last 20 log lines before the crash? Please post such a summary here.

Q4: Are your Windows and VMware versions (VCenter, Esx server, VDDK) compatible with NetBackup 7.5.0.7?

Upon reflection, I think I may have been incorrect in assuming you are doing VMware policy backups. Are you actually doing MS-Windows policy backups? Please share your policy information. Q2 and Q3 in my previous post still apply.

skrp
Level 4
Certified

Thank you Marianne. I've installed Textpad. 

Logging level is 0. I've checked both the files but don't see anything that I think is the cause of the issue. May be I can't find it. Will you be able to check the log(bpbrm) which I uploaded?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@skrp 

The log file is way too big to download on my (slow) home network. 

You say that you have uploaded (bpbrm and bpbkar), but there is only one large file? 

What you need to look for in the logs are the PID's seen in Activity Monitor.

e.g. bpbrm log file for 15 July, look for PID 11572 in square brackets [ ] .
bpbkar on 15 July, look for all PID entries [15264]. 

So, if you want to troubleshoot failures that happened on 15 July, there is no use looking at a log dated 13 July (071320.log). 

Hamza_H
Moderator
Moderator
   VIP   

You may want to check if your master (which is I presume it is your backup host) if it has enough space, also verify its resources after the execution of backups.

Please share bpbrm (level=3 of verbosity) and you may want to set vxms logs on the master server to get more information.

VxMS logging on a Windows backup host

To configure VxMS logging on a Windows backup host

  1. Create the VxMS log directory:

    install_path\NetBackup\logs\vxms

    Note:

    For logging to occur, the VxMS folder must exist.

    Note:

    If you have run the NetBackup mklogdir.bat command, the VxMS log directory already exists.

    See NetBackup logging for VMware.

  2. In the Windows registry, create the DWORD registry entry VXMS_VERBOSE in the following location:

    HKEY_LOCAL_MACHINE > SOFTWARE > Veritas > NetBackup > CurrentVersion > Config

  3. To configure the logging level, set the numeric value of VXMS_VERBOSE to 0 or greater. Larger numbers result in more verbose logs (set it to 5 it's enough)

more information: https://www.veritas.com/support/en_US/doc/21902280-127283730-0/v23649764-127283730

good luck