Forum Discussion

Jim-90's avatar
Jim-90
Level 6
8 years ago
Solved

NetApp orphaned snapshots from NDMP backups can't be deleted.

Any ideas on this apart from disable/re-enable NDMP and killing all active jobs?

Snapshots created by NDMP backups not being deleted.  NetApp support tried and gave up ..."its a NetBackup problem"  

NetApp = 3.3P1

NBU = 7.7.3

 

7.7
Other
  • Jim-90's avatar
    Jim-90
    8 years ago

    NetApp eventually responded with a solution: Bounce the controller and/or upgrade to ontap 9, its a known bug in 8.x.   A NetBackup process has locked the snapshot and the process cannot be killed by any natural means, a bit like a zombie process. Bouncing the device is the best way to get rid of them.  

    Given the low frequency of these occurring I suspect these processes only hang around when the backup starts at midnight plus there is combination of a full moon, low tide and a cloudless evening. 

5 Replies

Sort By
  • Michael_G_Ander's avatar
    Michael_G_Ander
    Level 6
    8 years ago

    Pretty sure is not just a Netbackup problem, as the snapshot is done by NetApp.

    If I remember correctly there is some ndmpd commands which can show hanging ndmp processes/backups and cancel/kill them. Might require root access to the NetApp system.

  • nbutech's avatar
    nbutech
    Level 6
    8 years ago

    I don't think Netbackup can delete it...

    Deletion will be by NetApp using command for example

    snap delete -a vol1 

    now requirement for stopping ndmp session would be to release the control over that snapshot so then Netapp can delete it.. 

    • Jim-90's avatar
      Jim-90
      Level 6
      8 years ago

      It was Netapp Support saying something like "we can't help its a NetBackup problem...sort it out with them"

      Our NetApp admin is not a novice and knows what he is doing.

  • Tousif's avatar
    Tousif
    Level 6
    8 years ago

    Hello,

     

    NDMP is protocol use by many backup application. Creating and delete the snapshot is done by NAS storage.

    After successful backup the NBU just inform storage server to delete the snapshot

    Could you please try to manual delete the snapshot by using NetApp command.

    The following logs may help you to isolate the issue.

    NBU:

    Media Server:

    bpbrm

    bptm

    ndmp and ndmpagent

    /usr/openv/netbackup/bin/vxlogcfg -r -p 51216 -o 134 -s DebugLevel=6 -s DiagnosticLevel=6
    /usr/openv/netbackup/bin/vxlogcfg -r -p 51216 -o 151 -s DebugLevel=6 -s DiagnosticLevel=6

    NetApp:

    System log

     

    Thanks & Regards

    • Jim-90's avatar
      Jim-90
      Level 6
      8 years ago

      NetApp eventually responded with a solution: Bounce the controller and/or upgrade to ontap 9, its a known bug in 8.x.   A NetBackup process has locked the snapshot and the process cannot be killed by any natural means, a bit like a zombie process. Bouncing the device is the best way to get rid of them.  

      Given the low frequency of these occurring I suspect these processes only hang around when the backup starts at midnight plus there is combination of a full moon, low tide and a cloudless evening.