06-18-2012 03:06 AM
Hi,
I have some trouble to backup Shadow Copy Components on Windows 2008R2 Netbackup 7.5.0.1 Client.
I have Netbackup 7.5.0.1 Master-Media Server on Win2003R2 x32. Client - Win2008R2, Netbackup Client - 7.5.0.1. Client is passive node of Microsoft Failover Cluster.
If i attempts to backup clients Shadow Copy Component - backup fail:
6/18/2012 12:31:19 PM - Error bpbrm(pid=1768) from client cls05-node-02: ERR - Volume resolution error for (System State:\System Files\System Files)
6/18/2012 12:31:21 PM - Error bpbrm(pid=1768) could not send server status message
6/18/2012 12:31:23 PM - Info bpbkar32(pid=3020) done. status: 156: snapshot error encountered
In bpbkar log:
12:31:18.992 PM: [3020.3016] <16> dos_backup::V_InitializeSystemState: ERR - FS_AddToSnapshotSet() Failed! (System State:\System Files\System Files) $s
12:31:18.992 PM: [3020.3016] <2> tar_base::V_vTarMsgW: ERR - Volume resolution error for (System State:\System Files\System Files)
12:31:19.023 PM: [3020.3016] <2> tar_base::V_vTarMsgW: INF - EXIT STATUS 156: snapshot error encountered
(All bpbkar log in attach)
Also I had the same problem with this client, but NBU version is 7.1. I use w2koption to exclude shared cluster disks: w2koption -backup -ignore_unresolved_volumes E:F:Q: (http://www.symantec.com/business/support/index?page=content&id=TECH128805) and problem is solved.
But in this case (NBU version 7.5.0.1) this solution doesn't works. Help me please to find solution.
06-18-2012 04:19 AM
156 is a snapshot error. Probably nothing to do with NBU.
What is in the bpfis log ?
Usual cause of this is that there is not enough diskspace tavailable to create the snapshot.
You need the bpfis log to investigate.
martin
06-18-2012 06:39 AM
On client I set additional logging level to maximum and in bpbkar log I find some interesting strings:
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF - Status E_FS_VOLUME_NOT_FOUND (0xE0000352) for object 'System Files'. Faild to resolve volume path name for 'e:\liga\' in SHADOW::SetSelectedForBackup
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF - Status E_FS_VOLUME_NOT_FOUND (0xE0000352) returned setting selected for backup for object System Files in SHADOW::AddToSnapshotSet
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF - AD:Trouble adding to set - Status E_FS_VOLUME_NOT_FOUND (0xE0000352) in SystemState::AddToSnapshotSet:566
3:30:29.983 PM: [3972.3536] <16> dos_backup::V_InitializeSystemState: ERR - FS_AddToSnapshotSet() Failed! (System State:\System Files\System Files) $s
3:30:29.983 PM: [3972.3536] <4> tar_base::V_vTarMsgW: INF - tar message received from dos_backup::V_InitializeSystemState
3:30:29.983 PM: [3972.3536] <2> tar_base::V_vTarMsgW: ERR - Volume resolution error for (System State:\System Files\System Files)
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF - v_beds::V_FindNext() EXIT Name: bRC:false LastError:0xFFFFFE07:0xa000fe07 (2684419591)
3:30:29.983 PM: [3972.3536] <2> tar_base::V_vTarMsgW: INF - EXIT STATUS 156: snapshot error encountered
Disk E:\ is cluster shared disk. In normal state disk E:\ is mounted on active node. I retrieve backup Shadow Copy Components of passive cluster node.
In directory e:\liga\ is installed some very !@#$ application and it is a lot of headache for administration :)
How to exclude e:\liga from Shadow Copy Components?
w2koption -backup -ignore_unresolved_volumes doesn't work in this case.
Detailed bpbkar log in attachment. I think bpfis log is no longer relevant.
UDP: system have enough disk space on C:\ :
used:11,2Gb
free: 18,6Gb
06-18-2012 12:19 PM
bpfis is the log required for a 156 status. There may be errors in other logs as seen, but the starting point is bpfis.
Martin
06-19-2012 04:12 AM
Also I manually play with bpfis on client:
C:\Program Files\Veritas\NetBackup\bin>bpfis create -fim VSS -id test1 C:\
C:\Program Files\Veritas\NetBackup\bin>bpfis delete -id test1 -copy 1
After this commands bpfis create log in /veritas/netbackup/logs/bpfis. This log in attachement.
06-19-2012 04:12 AM
Please tell, how to get bpfis log.
I setup global logging level to 5 in Master server properties->Logging.
In Client properties->Logging I setup all logging levels to 5. In Client properties->Windows Client->Client Settings I setup additional logging levels to maximum.
On master-media server and client system in folder .../veritas/netbackup/logs I create folder bpfis.
But after failed backup job directory .../veritas/netbackup/logs/bpfis on master-media and client is empty.
What am I missing?
06-19-2012 04:43 AM
go to client ... & post o/p of below commands (start CMD by run as administartor)
vssadmin list writers
vssadmin list providers
***************************
to test snapshot on OS level run below on Client
vssadmin create shadow /For=D: /AutoRetry=2 (D: for D drive & test this on all drive ).
also check do u hav enough free space on each drive....i hope this help you :)
06-19-2012 05:09 AM
Output for vssadmin command:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\Administrator>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7
------------------------------------------
C:\Users\Administrator>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'Task Scheduler Writer'
Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
State: [1] Stable
Last error: No error
Writer name: 'VSS Metadata Store Writer'
Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
State: [1] Stable
Last error: No error
Writer name: 'Performance Counters Writer'
Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
State: [1] Stable
Last error: No error
Writer name: 'FSRM Writer'
Writer Id: {12ce4370-5bb7-4c58-a76a-e5d5097e3674}
Writer Instance Id: {a719bbf9-82a4-4d4b-9365-a6a87f7d855b}
State: [1] Stable
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {b608c1ed-8e90-4406-a044-5ba761f5947a}
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {c6c87ac7-ca32-4175-9686-ff9c3d18a2f6}
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {77d6e5bf-c8e8-49db-af73-8116a81c657b}
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {2c2db6f4-e6d5-461d-865d-27a824c20efb}
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {42f96162-ca09-4e78-b7cd-275d3fd1a311}
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {7bd29c1f-664e-46b4-b593-0d975021e073}
State: [1] Stable
Last error: No error
Writer name: 'Cluster Database'
Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
Writer Instance Id: {103bdd7f-0146-40e1-87e4-8c7b8a643802}
State: [1] Stable
Last error: No error
------------------------------------------
C:\Users\Administrator>vssadmin create shadow /For=C: /AutoRetry=2
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Successfully created shadow copy for 'C:\'
Shadow Copy ID: {0078a8e0-5e0a-4ca4-9347-32a37d1f405e}
Shadow Copy Volume Name: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4
C:\Users\Administrator>
------------------------------------------
C:\Users\Administrator>vssadmin delete shadows /all
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Do you really want to delete 1 shadow copies (Y/N): [N]? y
Successfully deleted 1 shadow copies.
C:\Users\Administrator>
------------------------------------------
I tried to backup C:\ and Shadow Copy Components on passive cluster node cls05-node-02. This client have mounted only disk C:\. All other disks is shared and mounted on active node - cls05-node-01.
I think, application in directory E:\liga on shared cluster disk E:\ during the installation process recorded some system data in (System State:\System Files\System Files), and this data placed on unreachable (for passive node) disk E:\liga.When I try backup Shadow Copy Components (include System State:\System Files) client can't find some application data on E:\liga and fail.
PS: Application on E:\liga require custom HASP and HASP-driver to be run.
PSS: Disk C:\ have enought free space for shap:
used:11,2Gb
free: 18,6G
06-19-2012 05:37 AM
What is backup selection in NBU policy ??
06-19-2012 05:56 AM
Backup selection is
Shadow Copy Components:\
C:\
Policy type: MS-Windows
06-19-2012 11:56 AM
same error, different version:
https://www-secure.symantec.com/connect/forums/backup-virtual-machine-agent-error-156
I would open a support case.
06-22-2012 01:37 AM
.
06-22-2012 02:35 AM
The following 2 TNs may help:
http://www.symantec.com/docs/TECH128805
http://www.symantec.com/docs/TECH76141
The problem is caused by W2008 cluster, regardless of NBU version. See extract from TECH128805 :
Newly in Windows 2008 and Windows 2008 R2, every "Path to Executable" for every Windows Service is cataloged into the VSS System Writer and is therefore backed up by Shadow Copy Components. When a Shadow Copy Components backup starts, all of the VSS Writers are interrogated to compile a list of Volumes on which files classified into Shadow Copy Components reside - then Microsoft takes a Snapshot of those volumes.The Microsoft Volume Shadow Copy service is unaware that the node is Passive and the drive letter for the Service is not present. It attempts to snap the absent volume and an unexpected error occurs, which causes bpbkar32.exe to produce an Application Fault.
06-22-2012 04:16 AM
TN, described in http://www.symantec.com/docs/TECH128805 no work on Netbackup 7.5.0.1 client.
TN http://www.symantec.com/docs/TECH76141 also didn't work. My passive cluster node don't have
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SystemWriter\ExcludedBinaryPaths dir.
06-22-2012 04:30 AM
I tired...back to the last working configuration:
Uninstall Netbackup 7.5.0.1 client on passive cluster node and install Netbackup 7.1 client.
Apply hotfix NB_7.1_ET2256525_2 on client.
Exclude shared cluster disks: w2koption -backup -ignore_unresolved_volumes E:
Now all works fine, thanks for help, during the investigation I learned a lot of new information (for me).
06-22-2012 04:53 AM
My gut-feel is that 'w2koption -backup -ignore_unresolved_volumes ...' would also work with NBU 7.5 client installation.
I have not downloaded the hotfix to see what files/binaries is included, but my assumption is that it contains a new version of win2koption that includes '-ignore_unresolved_volumes [list of volumes]'.
I have just checked command parameters on my laptop with 7.5 installed:
C:\>w2koption /?
w2koption: -backup | -restore [-display] [-server server_name]
For -backup the optional parameters are:
-system_state <0=disallow legacy backup, 1=allow legacy backup>
-system_state setting only applies to
Windows 2003 (and later) servers
-ignore_unresolved_volumes [list of volumes]
Ex. -ignore_unresolved_volumes G:l:
....... .....
06-22-2012 06:15 AM
Yes, w2koption works in NBU7.5.0.1 client. With w2koption I exlude all shared cluster disks:
w2koption -backup -ignore_unresolved_volumes E:F:Q:
but Shadow Copy Components backup still fails (NBU client 7.5.0.1)
I hope this bug, which has already been fixed in 7.5.0.3. But I have not time to test/upgrade NBU infrastructure to 7.5.0.3 now. I'll do it after 5 july, but now still works on NBU 7.1 client :)
07-06-2012 05:11 AM
I update my Netbackup environment to 7.5.0.3 (master-media and clients), but have same issue with backing up Shadow Copy Components on cls05-node-02.
10-28-2012 09:17 AM
Hi!
Also on 7.5.0.3 and same error on the passive node did anyone find a sulotion to this?
//Mats
11-05-2012 05:53 AM
Hi,
I had the same problem with NB 7.5.0.4 on a passive node of a Windows 2008 R2 cluster (with SAP installed) and I've solved adding this registry key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SystemWriter\ExcludedBinaryPaths
Containing the executable path of the Windows service that was in charge of the active node and thus was on one of the cluster disk.
/Gildo
http://www.symantec.com/docs/TECH76141