Snapshot error encountered(156)

[NicoScheepers]

Good day,

Please assist I use Netbackup 7.5 to backup Server 2008 R2 VM's on a 2012 HyperV server and encounter the following when doing a backup of the one M:

cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.Server_1376761390.1.0

An existing snapshot is no longer valid and cannot be mounted for subsequent operations
8/17/2013 7:44:08 PM - end writing
     
 

8/17/2013 7:43:10 PM - Info nbjm(pid=5708) starting backup job (jobid=9753) for client , policy , schedule WEEKLY 
8/17/2013 7:43:10 PM - Info nbjm(pid=5708) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=9753, request id:{50221D43-28F1-45C8-9AF5-952B8F8C28CD}) 
8/17/2013 7:43:10 PM - requesting resource -msdp-stu-01
8/17/2013 7:43:10 PM - requesting resource .NBU_CLIENT.MAXJOBS.DWN-OTDSQL-PROD
8/17/2013 7:43:10 PM - requesting resource .NBU_POLICY.MAXJOBS.DWN-PROD-CLS-02
8/17/2013 7:43:10 PM - granted resource .NBU_CLIENT.MAXJOBS.DWN-OTDSQL-PROD
8/17/2013 7:43:10 PM - granted resource .NBU_POLICY.MAXJOBS.DWN-PROD-CLS-02
8/17/2013 7:43:10 PM - granted resource MediaID=@aaaad;DiskVolume=PureDiskVolume;DiskPool=-msdp-pool-01;Path=PureDiskVolume;StorageServer=;M...
8/17/2013 7:43:10 PM - granted resource dwn-prod-bck1-msdp-stu-01
8/17/2013 7:43:10 PM - estimated 0 Kbytes needed
8/17/2013 7:43:10 PM - begin Parent Job
8/17/2013 7:43:10 PM - begin Hyper-V, Start Notify Script
8/17/2013 7:43:10 PM - Info RUNCMD(pid=9000) started           
8/17/2013 7:43:10 PM - Info RUNCMD(pid=9000) exiting with status: 0        
Status 0
8/17/2013 7:43:10 PM - end Hyper-V, Start Notify Script; elapsed time: 00:00:00
8/17/2013 7:43:10 PM - begin Hyper-V, Step By Condition
Status 0
8/17/2013 7:43:10 PM - end Hyper-V, Step By Condition; elapsed time: 00:00:00
8/17/2013 7:43:10 PM - begin Hyper-V, Read File List
Status 0
8/17/2013 7:43:10 PM - end Hyper-V, Read File List; elapsed time: 00:00:00
8/17/2013 7:43:10 PM - begin Hyper-V, Create Snapshot
8/17/2013 7:43:10 PM - started
8/17/2013 7:43:11 PM - started process bpbrm (15080)
8/17/2013 7:43:22 PM - Info bpbrm(pid=15080)  is the host to backup data from    
8/17/2013 7:43:22 PM - Info bpbrm(pid=15080) reading file list from client       
8/17/2013 7:43:22 PM - Info bpbrm(pid=15080) start bpfis on client        
8/17/2013 7:43:22 PM - Info bpbrm(pid=15080) Starting create snapshot processing        
8/17/2013 7:43:23 PM - Info bpfis(pid=4836) Backup started          
8/17/2013 7:43:57 PM - Critical bpbrm(pid=15080) from client : FTL - snapshot processing failed, status 156  
8/17/2013 7:43:57 PM - Critical bpbrm(pid=15080) from client : FTL - snapshot creation failed, status 156  
8/17/2013 7:43:57 PM - Warning bpbrm(pid=15080) from client : WRN - ALL_LOCAL_DRIVES is not frozen   
8/17/2013 7:43:57 PM - Info bpfis(pid=4836) done. status: 156         
8/17/2013 7:43:57 PM - end Hyper-V, Create Snapshot; elapsed time: 00:00:47
8/17/2013 7:43:57 PM - Info bpfis(pid=0) done. status: 156: snapshot error encountered      
8/17/2013 7:43:57 PM - end writing
Status 156
8/17/2013 7:43:57 PM - end Parent Job; elapsed time: 00:00:47
8/17/2013 7:43:57 PM - begin Hyper-V, Stop On Error
Status 0
8/17/2013 7:43:57 PM - end Hyper-V, Stop On Error; elapsed time: 00:00:00
8/17/2013 7:43:57 PM - begin Hyper-V, Delete Snapshot
8/17/2013 7:43:57 PM - started process bpbrm (12260)
8/17/2013 7:44:05 PM - Info bpbrm(pid=12260) Starting delete snapshot processing        
8/17/2013 7:44:05 PM - Info bpfis(pid=0) Snapshot will not be deleted       
8/17/2013 7:44:07 PM - Info bpfis(pid=5096) Backup started          
8/17/2013 7:44:07 PM - Critical bpbrm(pid=12260) from client : cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.server_1376761390.1.0     
8/17/2013 7:44:08 PM - Info bpfis(pid=5096) done. status: 1542         
8/17/2013 7:44:08 PM - end Hyper-V, Delete Snapshot; elapsed time: 00:00:11
8/17/2013 7:44:08 PM - Info bpfis(pid=0) done. status: 1542: An existing snapshot is no longer valid and cannot be mounted for subsequent operations
8/17/2013 7:44:08 PM - end writing
Status 1542
8/17/2013 7:44:08 PM - end Parent Job; elapsed time: 00:00:58
8/17/2013 7:44:08 PM - begin Hyper-V, End Notify Script
8/17/2013 7:44:08 PM - Info RUNCMD(pid=15204) started           
8/17/2013 7:44:09 PM - Info RUNCMD(pid=15204) exiting with status: 0        
Status 0
8/17/2013 7:44:09 PM - end Hyper-V, End Notify Script; elapsed time: 00:00:01
Status 156
8/17/2013 7:44:09 PM - end operation
snapshot error encountered(156)

[RamNagalla]

please delete all files in C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\

they try the backup again.. see how it works.

if it fails look for the errors in HyperV level.

[quebek]

Create bpbrm and bpfis legacy logging - verbose = 5 (if you are not sure how to do this lookup here http://www.symantec.com/business/support/index?page=content&id=HOWTO72880&actp=search&viewlocale=en_US&searchid=1376835206150 ), retry the job and look into these logs for next guidelines...

[NicoScheepers]

Hi,

I had a look at  C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\ and the directory is already empty....

[Marianne]

 Info bpfis(pid=0) done. status: 1542: An existing snapshot is no longer valid and cannot be mounted for subsequent operations

We need to see bpfis log on HyperV server... 
Also check Event Viewer logs on HyperV server for VSS/snapshot errors.

http://www.symantec.com/docs/HOWTO73036

[Pamela_S1]

First, on the Hyper V host (if it is a CSV, in all hosts)  run vssadmin List Writers, the VSS Hyper V Writer has to stable and with no errors. if it says something like

Writer name: 'Microsoft Hyper-V VSS Writer'
   Writer Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
   Writer Instance Id: {ccf11414-9848-4055-9b78-24c27d913e0c}
   State: [1] Stable
   Last error: Retryable error

It means that as long as you don´t solve this, no backups.

Fo fix it, stop any backups related to the host and guests.  Then run Vssadmin List Shadows, and it will show you shadows that may be "hung up" or corrupted.

So, you have to delete de corrupted shadow, in order to get the writer stable and with no errors. To delete the corrupted shadow, we use the diskshadow command.

We get this issue everytime that Hyper - V or Exchange backup fails due connectivity problems, when you solve the connectivity problem, the snapshot doesn´t get flushed, and wont allow you to get backups after that .

Hope this helps

P.