05-25-2015 08:08 AM
Hello,
We have Windows client that is constantly getting the dreaded 156 error when I click on the details I see these type errors.
WRN – Snapshot Error while opening directory: GLOBALROOT\Device\HarddiskVolumeShadowCopy34\Program Files\Common File\System
WRN -Snapshot Error while opening directory: GLOBALROOT\Device\HarddiskVolumeShadowCopy42\Program Files\Common File\System
I checked the vssadmin list writers and they look ok. I check the viewer logs and I see nothing to indicate it’s a shadow copy issue
C:\>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'Task Scheduler Writer'
Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
State: [1] Stable
Last error: No error
Writer name: 'VSS Metadata Store Writer'
Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
State: [1] Stable
Last error: No error
Writer name: 'Performance Counters Writer'
Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {4d51c367-77d7-418c-adf6-256aac9c25a3}
State: [1] Stable
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {62430e04-d624-40c6-bc5a-2826aed1f08d}
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {90a6807d-f573-4600-aaa5-742a4dff80e7}
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {3377fc33-6642-437a-95a8-4ac6ea60e2f3}
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {6aeeb7c2-03ee-4b64-86ed-c3d948935757}
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {dd2a0f44-38ab-4bde-b8d6-47c7acefcb81}
State: [1] Stable
Last error: No error
Also, I changed the client so that it is using VSS and NOT VSP for open files however we are still getting the 156 error.
I attached the bpcd log however is doesn't show much and the bpfis and bpbkar logs are empty and the remain empty for each backup.
NOTE: This is a relatively new backup and it has been getting the 156 error since we started backing it up, so it has never got a completely successful backup.
Client info
Windows 2008 64bit server
We have 7.6.0.3 client running on it.
WRN – Snapshot Error while opening directory: GLOBALROOT\Device\HarddiskVolumeShadowCopy34\Program Files\Common File\System
WRN -Snapshot Error while opening directory: GLOBALROOT\Device\HarddiskVolumeShadowCopy42\Program Files\Common File\System
Solved! Go to Solution.
05-26-2015 12:15 AM - edited 08-02-2018 01:06 AM
It sounds as if backup is actually writing but fails later on with status 156?
If that is the case, then you need to check comms between client and the master server.
The client needs to update snapshot info that is located on the master server.
This info seems to be updated at the end of the backup, not the beginning.
bpfis log will confirm.
2 possible solutions:
1. Check that Client has correct Server entry for Master (not just a server) and that connectivity works in both directions (forward and reverse name lookup as well as port connectivity).
2. In Host Properties -> Master -> Client Attributes, add/select client name.
In Windows Open File Backup tab, for Snapshot error control, select Disable snapshot and continue.
The default is Abort.
My own experience with status 156 :
https://vox.veritas.com/t5/NetBackup/Failing-backups-status-156/m-p/710476#M190301
05-26-2015 06:54 AM
In addition to the very good advice above there is also this blog http://blogs.technet.com/b/askcore/archive/2010/06/18/reasons-why-the-error-enumeration-of-the-files...
Which I have used a lot to troubleshoot different VSS errors.
Have often found the problem to be a too small shadow area, unfortunately this does not always gives event log entries.
Another problem I have seen is big open files like a running SQL server dump
Hope this helps
05-27-2015 04:33 PM
05-27-2015 09:45 PM
05-25-2015 09:50 AM
bpfis log should be generated...
plese show us the detail status of the failed job, and also put the verbose as 5 and make sure bpfis log directory is created in correct log location and try backup again.
if you see the bpfis log with verbose 5, attach the log to this post
05-25-2015 05:44 PM
Thanks Nagalla for your response,
I have created a bpfis log before however not with verbose 5. Do I just create like this >>>bpfis=verbose 5? I tried to find the correct syntax but no luck.
05-25-2015 08:50 PM
Any volsnap errors in Event Viewer?
05-25-2015 10:27 PM
Please check the space allocated for snapshot creation. e.g. If u have 100GB filled drive ; u need to have at 10Gb allocated for snapshot
05-26-2015 12:15 AM - edited 08-02-2018 01:06 AM
It sounds as if backup is actually writing but fails later on with status 156?
If that is the case, then you need to check comms between client and the master server.
The client needs to update snapshot info that is located on the master server.
This info seems to be updated at the end of the backup, not the beginning.
bpfis log will confirm.
2 possible solutions:
1. Check that Client has correct Server entry for Master (not just a server) and that connectivity works in both directions (forward and reverse name lookup as well as port connectivity).
2. In Host Properties -> Master -> Client Attributes, add/select client name.
In Windows Open File Backup tab, for Snapshot error control, select Disable snapshot and continue.
The default is Abort.
My own experience with status 156 :
https://vox.veritas.com/t5/NetBackup/Failing-backups-status-156/m-p/710476#M190301
05-26-2015 06:54 AM
In addition to the very good advice above there is also this blog http://blogs.technet.com/b/askcore/archive/2010/06/18/reasons-why-the-error-enumeration-of-the-files...
Which I have used a lot to troubleshoot different VSS errors.
Have often found the problem to be a too small shadow area, unfortunately this does not always gives event log entries.
Another problem I have seen is big open files like a running SQL server dump
Hope this helps
05-26-2015 11:29 AM
Riaan, easy and Marianne thanks for your replies,
Any volsnap errors in Event viewer? NO there are no volsnap errors that I see infact I don’t see any errors that could be related to the backup.
Possible,
There looks to be enough space for snapshot
Marianne,
Yes , the backup is actually doing some writing before failing.
I checked the client has the correct master server entry.
I did notice while checking connectivity that the client host name does not match when I entered the following command.
./bptestbpcd -client moxxywinthorpe.mycompany.com-verbose -debug
The connectivity seems fine still checking though however the name still comes up with the name of the server that this server replaced.
[root@nbu10w0 admincmd]# ./bptestbpcd -client Moxxywinthorpe.mycompany.com -verbose -debug
11:59:59.875 [67763] <2> bpcr_get_version_rqst: bpcd version: 07600002
PEER_NAME = nbu10w0
HOST_NAME = number55.mycompany.com
CLIENT_NAME = number55.mycompany.com
VERSION = 0x07600002
PLATFORM = win_x86
PATCH_VERSION = 7.6.0.3
SERVER_PATCH_VERSION = 7.6.0.3
MASTER_SERVER = nbu10w0
EMM_SERVER = nbu10w0
NB_MACHINE_TYPE = CLIENT
11:59:59.907 [67763] <2> vnet_pbxConnect: pbxConnectEx Succeeded
11:59:59.935 [67763] <8> do_pbx_service: [vnet_connect.c:2152] via PBX VNETD CONNECT FROM xx.xx.xxx.xx.63286 TO xx.xxx.xx.1556 fd = 5
xx.xx.xxx.xx:63286 -> xx.xx.xxx.xx:1556
<2>bptestbpcd: EXIT status = 0
12:00:00.170 [67763] <2> bptestbpcd: EXIT status = 0
[root@nbu10w0 admincmd]#
2 ) The correct name is in >>Host Properties >>Master>>>Client Attributes.
I did have to update the Windows Open file Backup tab for snapshot error control. I selected Disable snapshot and continue as you suggested. So we will see if that helps with the backup.
As far as the bpfis log I did create one and it continues to NOT generate anything. How do you create the bpfis verbose 5 log in windows? When create the log I normally just create bpfis in the netbackup logs folder and normally something is generated after a backup. However so far nothing appears in the bpfis or bpbkar logs.
Thanks again for your responses I will update soon.
05-26-2015 11:33 AM
Thanks Micheal G for your response as well. I will take a look at the blog.
05-26-2015 11:57 AM
Hi All,
update
the >>./bptestbpcd -client Moxxywinthorpe.mycompany.com -verbose -debug command generated something in the bpcd log see attached log
Marianne, once again you are on to something I see this process_requests: Server access denied
connectivity issue in the log. I have to find out what is causing it because the connection to our backup server seems fine
thanks again for all your responses
05-26-2015 01:33 PM
Wow, that log looks a lot like one of our SQL servers that went awry last week.
I'd be curious what you find to fix this.
05-27-2015 08:40 AM
Hello,
The backup completed with a partial success. (NO 156 error) however not sure if that is good either.
Hi Marianne,
The below is what I did that I believe got us a partial success backup.
In Windows Open File Backup tab, for Snapshot error control, select Disable snapshot and continue.
Should I leave it as is? Just checking because in the details of the errors I see the following.
WRN - can't open object System State:\System Files\System Files (Win32 3758161617: Unknown error)
WRN - can't open object System State:\Registry\Registry (Win32 3758161617: Unknown error)
WRN - can't open object System State:\COM + CLASS REGISTRATION DATABASE\COM+REGDB (win32 3758161617 : Unknown error)
WRN - can't open object System State:\Windows Management Instrumental\WMI(WIN3213758161617: Unknown error)
WRN - can't open object System State:\Event logs (WIN32 3758161617: Unknown error)
Best Regards
05-27-2015 04:33 PM
05-27-2015 09:45 PM