Netbackup 7.6.1.1 VMware Snapshot Jobs stuck waiting for deletion

[OxBrOps]

Hi,

We have just upgraded to 7.6.1.1 and have a new issue whereby every day 5-10 of 50 or so of our VMware Snapshot backups sit waiting for the snapshot created at the start of the job to delete.

The child job to back up the snapshot completes sucesfully and VMware does sucesfully delete the snapshot created for netbackup, but for some reason the message that this deletion has completed doesn't ever get back to the netbackup console and the jobs just sit there indefinately until they are cancelled by the administrator.

It effects differernt cilents every day, across several different polices and on different media servers.

Has anyone come across this problem? It seems to be new with 7.6.1.1

If anyone can give me and idea of what logs might be useful to look in that would be really helpful.

Although the backup are working correctly the hung jobs are taking up precious slots in our backup window and stopping others from running.

Helen

[sdo · Accepted solution]

The LBN for NetBackup v7.6.1:

https://support.symantec.com/en_US/article.TECH227580.html

...lists this:

(ET 3735610) <<Fix Downloadable>> After upgrading to NetBackup 7.6.1/7.6.1.1 VMware Snapshots are intermittently hanging on Windows VMware Backup Host
 http://www.symantec.com/docs/TECH230113

...wondering if this is worth a read?  Might be related/similar to your issue.

HTH.

[OxBrOps]

sdo -

Thanks - it does sound similar.

I'll get my head around the location of the bpfis log and see if I can match up the errors.

:)

[OxBrOps]

I have created the bpfis logs on the media server as they wern't there already so we will see what they show when we get a hung job, but it does sound like it could be it.

As an aside:

I did a bit of cross referecning between the detailed job status and the PID of the bpfis processes on the media server. For one of the jobs that has been stuck wating for confirmation of the deletion i went and killed the corresponding bpfis process on the media server. Hey presto, back in the admin console the job now completes sucesfully.

So for some reason the bpfis.exe (windows media server) process doesn't temrminate even though it's seemingly finished doing whatever.

Atleast I can use this as a a work around to stop the failure reports until I confirm it is the issue and apply the EEB if it is.

Ta

[thevmwareguy]

You should apply the hotfix to stop this issue occuring again as per user sdo comment. It has been tested.

http://www.symantec.com/docs/TECH230113

[Dollypee]

Hi, So far, I have'nt experience this issue in my environment also running 7.6.1.1. But Symantec has acknowledge this issue which is due to be fix in next maintenance release. I believe there's currently an EEB as a work around. Thanks

[OxBrOps]

Just a note to say that in order to generate the bpfis logs on the media server required to prove the issue you need to do the following:

Create the bpfis logging directory

Open the BAR console, go to File,  NetBackup Client Properties, Trouble shooting Tab, set General = 2 and Verbose = 5.

What to look for in the logs are the following lines.

14:16:19.390 [144.2944] <2> onlfi_vfms_logf: INF - VThreadBase detected multiple threads.
14:17:08.015 [144.1276] <2> send_keep_alive: INF - sending keep alive

You should the "sending keep alive" repeat

In our case we have applied the EBB now after a conversation with symantec support.

Thanks all for your help