06-29-2017 11:04 PM
Any ideas on this apart from disable/re-enable NDMP and killing all active jobs?
Snapshots created by NDMP backups not being deleted. NetApp support tried and gave up ..."its a NetBackup problem"
NetApp = 3.3P1
NBU = 7.7.3
Solved! Go to Solution.
07-04-2017 05:49 AM - edited 07-04-2017 05:52 AM
NetApp eventually responded with a solution: Bounce the controller and/or upgrade to ontap 9, its a known bug in 8.x. A NetBackup process has locked the snapshot and the process cannot be killed by any natural means, a bit like a zombie process. Bouncing the device is the best way to get rid of them.
Given the low frequency of these occurring I suspect these processes only hang around when the backup starts at midnight plus there is combination of a full moon, low tide and a cloudless evening.
06-29-2017 11:22 PM
Pretty sure is not just a Netbackup problem, as the snapshot is done by NetApp.
If I remember correctly there is some ndmpd commands which can show hanging ndmp processes/backups and cancel/kill them. Might require root access to the NetApp system.
06-30-2017 05:53 AM
I don't think Netbackup can delete it...
Deletion will be by NetApp using command for example
snap delete -a vol1
now requirement for stopping ndmp session would be to release the control over that snapshot so then Netapp can delete it..
07-01-2017 04:11 PM
It was Netapp Support saying something like "we can't help its a NetBackup problem...sort it out with them"
Our NetApp admin is not a novice and knows what he is doing.
07-03-2017 12:16 AM
Hello,
NDMP is protocol use by many backup application. Creating and delete the snapshot is done by NAS storage.
After successful backup the NBU just inform storage server to delete the snapshot
Could you please try to manual delete the snapshot by using NetApp command.
The following logs may help you to isolate the issue.
NBU:
Media Server:
bpbrm
bptm
ndmp and ndmpagent
/usr/openv/netbackup/bin/vxlogcfg -r -p 51216 -o 134 -s DebugLevel=6 -s DiagnosticLevel=6
/usr/openv/netbackup/bin/vxlogcfg -r -p 51216 -o 151 -s DebugLevel=6 -s DiagnosticLevel=6
NetApp:
System log
Thanks & Regards
07-04-2017 05:49 AM - edited 07-04-2017 05:52 AM
NetApp eventually responded with a solution: Bounce the controller and/or upgrade to ontap 9, its a known bug in 8.x. A NetBackup process has locked the snapshot and the process cannot be killed by any natural means, a bit like a zombie process. Bouncing the device is the best way to get rid of them.
Given the low frequency of these occurring I suspect these processes only hang around when the backup starts at midnight plus there is combination of a full moon, low tide and a cloudless evening.