cancel
Showing results forΒ 
Search instead forΒ 
Did you mean:Β 

snapshot error 156 of file system policy

TJN06
Level 3

File system backups are failing with 156 error in file system having multiple drivers backups are running in one point of time backups are failing with 156 error then how can we identifiy which drive got failed. 

Note : Allow multie streaming option is already disabled. and we have clear instructions should not enable the multie streaming option to FS policy kindly help us how can we identifiy which drive is failing and where we need to check.

Job details

Jul 27, 2020 7:00:01 PM - Info nbjm (pid=15428) starting backup job (jobid=377261) for client cvksvs491.xxx.xxx.com, policy MSFS_SCCVPBLRDBS_OS_FULL_006, schedule DAILY_INCR
Jul 27, 2020 7:00:01 PM - Info nbjm (pid=15428) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=377261, request id:{F91DD5F8-D23F-45A2-86B1-AAE76423D304})
Jul 27, 2020 7:00:01 PM - requesting resource  DCD2D004_MSFS001_BKP_STU
Jul 27, 2020 7:00:01 PM - requesting resource  cvssvni01.xxxx.xxx.com.NBU_CLIENT.MAXJOBS.cvksvs491.xxx.xxx.com
Jul 27, 2020 7:00:27 PM - granted resource  cvssvni01.xxxxx.xxx.com.NBU_CLIENT.MAXJOBS.cvksvs491.xxx.xxx.com
Jul 27, 2020 7:00:28 PM - granted resource  MediaID=@aaaaB;DiskVolume=DCD2D004_NBU_MSFS001;DiskPool=DCD2D004_MSFS001_DP;Path=DCD2D004_NBU_MSFS001;StorageServer=172.16.96.216;MediaServer=cvsvks10-b.xxxxx.xxxxx.com
Jul 27, 2020 7:00:28 PM - granted resource  DCD2D004_MSFS001_BKP_STU
Jul 27, 2020 7:02:19 PM - estimated 1412999503 kbytes needed
Jul 27, 2020 7:02:19 PM - Info nbjm (pid=15428) started backup (backupid=cvksvs491.xxx.xxx.com_1595856632) job for client cvksvs491.xxx.xxx.com, policy MSFS_SCCVPBLRDBS_OS_FULL_006, schedule DAILY_INCR on storage unit DCD2D004_MSFS001_BKP_STU
Jul 27, 2020 7:02:25 PM - started process bpbrm (pid=295316)
Jul 27, 2020 7:02:28 PM - Using resilient connections
Jul 27, 2020 7:02:30 PM - Info bpbrm (pid=295316) cvksvs491.xxx.xxx.com is the host to backup data from
Jul 27, 2020 7:02:30 PM - Info bpbrm (pid=295316) reading file list for client
Jul 27, 2020 7:02:33 PM - Info bpbrm (pid=295316) accelerator enabled
Jul 27, 2020 7:02:37 PM - connecting
Jul 27, 2020 7:02:40 PM - Info bpbrm (pid=295316) starting bpbkar32 on client
Jul 27, 2020 7:02:41 PM - connected; connect time: 0:00:00
Jul 27, 2020 7:02:42 PM - Info bpbkar32 (pid=8012) Backup started
Jul 27, 2020 7:02:42 PM - Info bpbkar32 (pid=8012) change time comparison:<enabled>
Jul 27, 2020 7:02:42 PM - Info bpbkar32 (pid=8012) accelerator enabled backup, archive bit processing:<disabled>
Jul 27, 2020 7:02:43 PM - Info bptm (pid=165352) start
Jul 27, 2020 7:02:44 PM - Info bptm (pid=165352) using 262144 data buffer size
Jul 27, 2020 7:02:44 PM - Info bptm (pid=165352) using 30 data buffers
Jul 27, 2020 7:02:46 PM - Info bptm (pid=165352) start backup
Jul 27, 2020 7:02:47 PM - Info bptm (pid=165352) backup child process is pid 119140.15492
Jul 27, 2020 7:02:47 PM - Info bptm (pid=119140) start
Jul 27, 2020 7:02:47 PM - begin writing
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <C:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <D:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <F:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <H:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <L:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <M:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <T:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) not using change journal data for <System State:\>: not supported for non-local volumes / file systems
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) not using change journal data for <_BACKUP_SPECIAL_OBJECTS AFTER System State:>: not supported for non-local volumes / file systems
Jul 27, 2020 7:03:38 PM - Info bpbkar32 (pid=8012) not using change journal data for <C:\>: unable to locate change journal data
Jul 27, 2020 7:11:28 PM - Info bpbkar32 (pid=8012) not using change journal data for <D:\>: unable to locate change journal data
Jul 27, 2020 7:11:39 PM - Info bpbkar32 (pid=8012) not using change journal data for <F:\>: unable to locate change journal data
Jul 27, 2020 7:14:05 PM - Info bpbkar32 (pid=8012) not using change journal data for <H:\>: unable to locate change journal data
Jul 27, 2020 8:13:00 PM - Info bpbkar32 (pid=8012) not using change journal data for <L:\>: unable to locate change journal data
Jul 27, 2020 8:13:02 PM - Error bpbrm (pid=295316) from client cvksvs491.xxx.xxx.com: ERR - failure reading file: L:\LOGFILES\DSM_log.ldf (WIN32 21: The device is not ready. )
Jul 27, 2020 8:13:02 PM - Error bpbrm (pid=295316) from client cvksvs491.xxx.xxx.com: ERR - Snapshot Error while reading file: GLOBALROOT\Device\HarddiskVolumeShadowCopy112\LOGFILES\DSM_log.ldf
Jul 27, 2020 8:13:03 PM - Critical bpbrm (pid=295316) from client cvksvs491.xxx.xxx.com: FTL - Backup operation aborted!
Jul 27, 2020 8:13:04 PM - Info bpbkar32 (pid=8012) accelerator sent 16393147904 bytes out of 971424292864 bytes to server, optimization 98.3%
Jul 27, 2020 8:13:06 PM - Info bptm (pid=165352) waited for full buffer 3016 times, delayed 199298 times
Jul 27, 2020 8:13:09 PM - Info bptm (pid=165352) EXITING with status 0 <----------
Jul 27, 2020 8:13:10 PM - Info cvscvk10-b.xxxx.xxxx.com (pid=165352) StorageServer=hp-StoreOnceCatalyst:172.16.96.216; Report=scanned: 948656436 KB, CR sent: 1938003 KB, dedup: 99.80%
Jul 27, 2020 8:13:11 PM - end writing; write time: 1:10:24
Jul 27, 2020 8:13:12 PM - Info bpbkar32 (pid=8012) done. status: 156: snapshot error encountered
Jul 27, 2020 8:13:26 PM - Using resilient connections
Jul 27, 2020 8:13:27 PM - Info bpbrm (pid=311160) Starting delete snapshot processing
Jul 27, 2020 8:13:33 PM - Info bpfis (pid=7752) Backup started
Jul 27, 2020 8:13:33 PM - Warning bpbrm (pid=311160) from client cvksvs491.xxx.xxx.com: cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.cvksvs491.xxx.xxx.com_1595856632.1.0
Jul 27, 2020 8:13:34 PM - Info bpfis (pid=7752) done. status: 4207
Jul 27, 2020 8:13:34 PM - Info bpfis (pid=7752) done. status: 4207: Could not fetch snapshot metadata or state files
Snapshot error encountered  (156)

1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

If you are backing up multiple drive letters in a single stream, then a failure of one drive letter will cause the ENTIRE backup of ALL drive letters to fail. NOTHING will be marked as successful.

The entire policy for all drive letters need to be restarted. 

If you do NOT want this to happen, you need to enable multiple data streams.
This feature will start a separate backup job for each item in the Backup Selection. 
If snapshot for drive letter L (or any other drive letter) fails, then backups for other drive letters will not be affected. 

See this extract from NBU Admin Guide: 
https://www.veritas.com/support/en_US/doc/18716246-126559472-0/v41274943-126559472

Because the backup streams run independently, the use of multiple data streams can shorten the retry time in the event of a backup failure. A single failure only terminates a single stream. NetBackup can restart the failed stream without restarting the others.

For example, assume the backup for a 10-gigabyte partition is split into five streams, each containing 2 gigabytes. If the last stream fails after it writes 1.9 gigabytes (a total of 9.9 gigabytes is backed up), NetBackup retries only the last gigabyte stream. If the 10-gigabyte partition is backed up without multiple data streams and a failure occurs, the entire 10-gigabyte backup must be retried.

The Schedule backup attempts property in the Global Attributes properties, applies to each stream. For example, if the Schedule backup attempts property is set to 3, NetBackup retries each stream a maximum of three times.

The Activity Monitor displays each stream as a separate job. Use the job details view to determine the files that are backed up by each of these jobs.

View solution in original post

5 REPLIES 5

Lowell_Palecek
Level 6
Employee

Get the bpbkar and bpfis logs on the client and look at them. Bpbkar32 drives the MS-Windows backup. It calls bpfis. Bpfis makes the snapshots and generates the 156 error code.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@TJN06 

Can we assume that you have ALL_LOCAL_DRIVES in the Backup Selection? 
From what I can see, there is only a single bpbkar process [PID 8012] running on the client, i.e. single stream backup.

Best to have 'Allow multiple data streams' selected in the Policy Attributes. 
This way, NBU will generate a separate job for each drive letter. 
A failure of one drive letter will not cause the entire backup for the client to fail. 

If you want to change the policy attribute, please only do so before a Full schedule is due. 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Oh! I have just noticed the following:

ERR - failure reading file: L:\LOGFILES\DSM_log.ldf 

If this is a database server, you need to exclude database files from regular file-level backups.
Databases should be backed up with the appropriate database agent policy. 

we have c,d,f,h,l drives are backing up with out enabling the multiple data stream option. my doubt is how can we identifiy which drive got success and how can we check those.

if backup got failed for L drive with the partificlar folder or path and then what about c,d,f,j,h drivers is it backups completed successfully if yes how to check those. if L drive got failed for next time we need to retrigger the whole policy ?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

If you are backing up multiple drive letters in a single stream, then a failure of one drive letter will cause the ENTIRE backup of ALL drive letters to fail. NOTHING will be marked as successful.

The entire policy for all drive letters need to be restarted. 

If you do NOT want this to happen, you need to enable multiple data streams.
This feature will start a separate backup job for each item in the Backup Selection. 
If snapshot for drive letter L (or any other drive letter) fails, then backups for other drive letters will not be affected. 

See this extract from NBU Admin Guide: 
https://www.veritas.com/support/en_US/doc/18716246-126559472-0/v41274943-126559472

Because the backup streams run independently, the use of multiple data streams can shorten the retry time in the event of a backup failure. A single failure only terminates a single stream. NetBackup can restart the failed stream without restarting the others.

For example, assume the backup for a 10-gigabyte partition is split into five streams, each containing 2 gigabytes. If the last stream fails after it writes 1.9 gigabytes (a total of 9.9 gigabytes is backed up), NetBackup retries only the last gigabyte stream. If the 10-gigabyte partition is backed up without multiple data streams and a failure occurs, the entire 10-gigabyte backup must be retried.

The Schedule backup attempts property in the Global Attributes properties, applies to each stream. For example, if the Schedule backup attempts property is set to 3, NetBackup retries each stream a maximum of three times.

The Activity Monitor displays each stream as a separate job. Use the job details view to determine the files that are backed up by each of these jobs.