snapshot error 156 of file system policy
File system backups are failing with 156 error in file system having multiple drivers backups are running in one point of time backups are failing with 156 error then how can we identifiy which drive got failed.
Note : Allow multie streaming option is already disabled. and we have clear instructions should not enable the multie streaming option to FS policy kindly help us how can we identifiy which drive is failing and where we need to check.
Job details
Jul 27, 2020 7:00:01 PM - Info nbjm (pid=15428) starting backup job (jobid=377261) for client cvksvs491.xxx.xxx.com, policy MSFS_SCCVPBLRDBS_OS_FULL_006, schedule DAILY_INCR
Jul 27, 2020 7:00:01 PM - Info nbjm (pid=15428) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=377261, request id:{F91DD5F8-D23F-45A2-86B1-AAE76423D304})
Jul 27, 2020 7:00:01 PM - requesting resource DCD2D004_MSFS001_BKP_STU
Jul 27, 2020 7:00:01 PM - requesting resource cvssvni01.xxxx.xxx.com.NBU_CLIENT.MAXJOBS.cvksvs491.xxx.xxx.com
Jul 27, 2020 7:00:27 PM - granted resource cvssvni01.xxxxx.xxx.com.NBU_CLIENT.MAXJOBS.cvksvs491.xxx.xxx.com
Jul 27, 2020 7:00:28 PM - granted resource MediaID=@aaaaB;DiskVolume=DCD2D004_NBU_MSFS001;DiskPool=DCD2D004_MSFS001_DP;Path=DCD2D004_NBU_MSFS001;StorageServer=172.16.96.216;MediaServer=cvsvks10-b.xxxxx.xxxxx.com
Jul 27, 2020 7:00:28 PM - granted resource DCD2D004_MSFS001_BKP_STU
Jul 27, 2020 7:02:19 PM - estimated 1412999503 kbytes needed
Jul 27, 2020 7:02:19 PM - Info nbjm (pid=15428) started backup (backupid=cvksvs491.xxx.xxx.com_1595856632) job for client cvksvs491.xxx.xxx.com, policy MSFS_SCCVPBLRDBS_OS_FULL_006, schedule DAILY_INCR on storage unit DCD2D004_MSFS001_BKP_STU
Jul 27, 2020 7:02:25 PM - started process bpbrm (pid=295316)
Jul 27, 2020 7:02:28 PM - Using resilient connections
Jul 27, 2020 7:02:30 PM - Info bpbrm (pid=295316) cvksvs491.xxx.xxx.com is the host to backup data from
Jul 27, 2020 7:02:30 PM - Info bpbrm (pid=295316) reading file list for client
Jul 27, 2020 7:02:33 PM - Info bpbrm (pid=295316) accelerator enabled
Jul 27, 2020 7:02:37 PM - connecting
Jul 27, 2020 7:02:40 PM - Info bpbrm (pid=295316) starting bpbkar32 on client
Jul 27, 2020 7:02:41 PM - connected; connect time: 0:00:00
Jul 27, 2020 7:02:42 PM - Info bpbkar32 (pid=8012) Backup started
Jul 27, 2020 7:02:42 PM - Info bpbkar32 (pid=8012) change time comparison:<enabled>
Jul 27, 2020 7:02:42 PM - Info bpbkar32 (pid=8012) accelerator enabled backup, archive bit processing:<disabled>
Jul 27, 2020 7:02:43 PM - Info bptm (pid=165352) start
Jul 27, 2020 7:02:44 PM - Info bptm (pid=165352) using 262144 data buffer size
Jul 27, 2020 7:02:44 PM - Info bptm (pid=165352) using 30 data buffers
Jul 27, 2020 7:02:46 PM - Info bptm (pid=165352) start backup
Jul 27, 2020 7:02:47 PM - Info bptm (pid=165352) backup child process is pid 119140.15492
Jul 27, 2020 7:02:47 PM - Info bptm (pid=119140) start
Jul 27, 2020 7:02:47 PM - begin writing
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <C:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <D:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <F:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <H:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <L:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <M:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) will attempt to use change journal data for <T:\>
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) not using change journal data for <System State:\>: not supported for non-local volumes / file systems
Jul 27, 2020 7:02:50 PM - Info bpbkar32 (pid=8012) not using change journal data for <_BACKUP_SPECIAL_OBJECTS AFTER System State:>: not supported for non-local volumes / file systems
Jul 27, 2020 7:03:38 PM - Info bpbkar32 (pid=8012) not using change journal data for <C:\>: unable to locate change journal data
Jul 27, 2020 7:11:28 PM - Info bpbkar32 (pid=8012) not using change journal data for <D:\>: unable to locate change journal data
Jul 27, 2020 7:11:39 PM - Info bpbkar32 (pid=8012) not using change journal data for <F:\>: unable to locate change journal data
Jul 27, 2020 7:14:05 PM - Info bpbkar32 (pid=8012) not using change journal data for <H:\>: unable to locate change journal data
Jul 27, 2020 8:13:00 PM - Info bpbkar32 (pid=8012) not using change journal data for <L:\>: unable to locate change journal data
Jul 27, 2020 8:13:02 PM - Error bpbrm (pid=295316) from client cvksvs491.xxx.xxx.com: ERR - failure reading file: L:\LOGFILES\DSM_log.ldf (WIN32 21: The device is not ready. )
Jul 27, 2020 8:13:02 PM - Error bpbrm (pid=295316) from client cvksvs491.xxx.xxx.com: ERR - Snapshot Error while reading file: GLOBALROOT\Device\HarddiskVolumeShadowCopy112\LOGFILES\DSM_log.ldf
Jul 27, 2020 8:13:03 PM - Critical bpbrm (pid=295316) from client cvksvs491.xxx.xxx.com: FTL - Backup operation aborted!
Jul 27, 2020 8:13:04 PM - Info bpbkar32 (pid=8012) accelerator sent 16393147904 bytes out of 971424292864 bytes to server, optimization 98.3%
Jul 27, 2020 8:13:06 PM - Info bptm (pid=165352) waited for full buffer 3016 times, delayed 199298 times
Jul 27, 2020 8:13:09 PM - Info bptm (pid=165352) EXITING with status 0 <----------
Jul 27, 2020 8:13:10 PM - Info cvscvk10-b.xxxx.xxxx.com (pid=165352) StorageServer=hp-StoreOnceCatalyst:172.16.96.216; Report=scanned: 948656436 KB, CR sent: 1938003 KB, dedup: 99.80%
Jul 27, 2020 8:13:11 PM - end writing; write time: 1:10:24
Jul 27, 2020 8:13:12 PM - Info bpbkar32 (pid=8012) done. status: 156: snapshot error encountered
Jul 27, 2020 8:13:26 PM - Using resilient connections
Jul 27, 2020 8:13:27 PM - Info bpbrm (pid=311160) Starting delete snapshot processing
Jul 27, 2020 8:13:33 PM - Info bpfis (pid=7752) Backup started
Jul 27, 2020 8:13:33 PM - Warning bpbrm (pid=311160) from client cvksvs491.xxx.xxx.com: cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.cvksvs491.xxx.xxx.com_1595856632.1.0
Jul 27, 2020 8:13:34 PM - Info bpfis (pid=7752) done. status: 4207
Jul 27, 2020 8:13:34 PM - Info bpfis (pid=7752) done. status: 4207: Could not fetch snapshot metadata or state files
Snapshot error encountered (156)
If you are backing up multiple drive letters in a single stream, then a failure of one drive letter will cause the ENTIRE backup of ALL drive letters to fail. NOTHING will be marked as successful.
The entire policy for all drive letters need to be restarted.
If you do NOT want this to happen, you need to enable multiple data streams.
This feature will start a separate backup job for each item in the Backup Selection.
If snapshot for drive letter L (or any other drive letter) fails, then backups for other drive letters will not be affected.See this extract from NBU Admin Guide:
https://www.veritas.com/support/en_US/doc/18716246-126559472-0/v41274943-126559472Because the backup streams run independently, the use of multiple data streams can shorten the retry time in the event of a backup failure. A single failure only terminates a single stream. NetBackup can restart the failed stream without restarting the others.
For example, assume the backup for a 10-gigabyte partition is split into five streams, each containing 2 gigabytes. If the last stream fails after it writes 1.9 gigabytes (a total of 9.9 gigabytes is backed up), NetBackup retries only the last gigabyte stream. If the 10-gigabyte partition is backed up without multiple data streams and a failure occurs, the entire 10-gigabyte backup must be retried.
The Schedule backup attempts property in the Global Attributes properties, applies to each stream. For example, if the Schedule backup attempts property is set to 3, NetBackup retries each stream a maximum of three times.
The Activity Monitor displays each stream as a separate job. Use the job details view to determine the files that are backed up by each of these jobs.