cancel
Showing results for 
Search instead for 
Did you mean: 

Windows 2008R2 Shadow copy components backup failed, Volume resolution error for

vakulenko_sergi
Level 3
Partner Accredited

Hi,

I have some trouble to backup Shadow Copy Components on Windows 2008R2 Netbackup 7.5.0.1 Client.

I have Netbackup 7.5.0.1 Master-Media Server on Win2003R2 x32. Client - Win2008R2, Netbackup Client - 7.5.0.1. Client is passive node of Microsoft Failover Cluster.

If i attempts to backup clients Shadow Copy Component - backup fail:

6/18/2012 12:31:19 PM - Error bpbrm(pid=1768) from client cls05-node-02: ERR - Volume resolution error for (System State:\System Files\System Files)
6/18/2012 12:31:21 PM - Error bpbrm(pid=1768) could not send server status message      
6/18/2012 12:31:23 PM - Info bpbkar32(pid=3020) done. status: 156: snapshot error encountered      

In bpbkar log:
12:31:18.992 PM: [3020.3016] <16> dos_backup::V_InitializeSystemState: ERR - FS_AddToSnapshotSet() Failed! (System State:\System Files\System Files) $s
12:31:18.992 PM: [3020.3016] <2> tar_base::V_vTarMsgW: ERR - Volume resolution error for (System State:\System Files\System Files)
12:31:19.023 PM: [3020.3016] <2> tar_base::V_vTarMsgW: INF - EXIT STATUS 156: snapshot error encountered

(All bpbkar log in attach)

Also I had the same problem with this client, but NBU version is 7.1. I use w2koption to exclude shared cluster disks: w2koption -backup -ignore_unresolved_volumes E:F:Q: (http://www.symantec.com/business/support/index?page=content&id=TECH128805) and problem is solved.

But in this case (NBU version 7.5.0.1) this solution doesn't works. Help me please to find solution.

19 REPLIES 19

mph999
Level 6
Employee Accredited

156 is a snapshot error.  Probably nothing to do with NBU.

What is in the bpfis log ?

Usual cause of this is that there is not enough diskspace tavailable to create the snapshot.

You need the bpfis log to investigate.

martin

vakulenko_sergi
Level 3
Partner Accredited

On client I set additional logging level to maximum and in bpbkar log I find some interesting strings:

3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF - Status E_FS_VOLUME_NOT_FOUND (0xE0000352) for object 'System Files'.  Faild to resolve volume path name for 'e:\liga\' in SHADOW::SetSelectedForBackup
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF - Status E_FS_VOLUME_NOT_FOUND (0xE0000352) returned setting selected for backup for object System Files in SHADOW::AddToSnapshotSet
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF -   AD:Trouble adding to set - Status E_FS_VOLUME_NOT_FOUND (0xE0000352) in SystemState::AddToSnapshotSet:566
3:30:29.983 PM: [3972.3536] <16> dos_backup::V_InitializeSystemState: ERR - FS_AddToSnapshotSet() Failed! (System State:\System Files\System Files) $s
3:30:29.983 PM: [3972.3536] <4> tar_base::V_vTarMsgW: INF - tar message received from dos_backup::V_InitializeSystemState
3:30:29.983 PM: [3972.3536] <2> tar_base::V_vTarMsgW: ERR - Volume resolution error for (System State:\System Files\System Files)
3:30:29.983 PM: [3972.3536] <2> ov_log::V_GlobalLog: INF -  v_beds::V_FindNext() EXIT Name: bRC:false LastError:0xFFFFFE07:0xa000fe07 (2684419591)

3:30:29.983 PM: [3972.3536] <2> tar_base::V_vTarMsgW: INF - EXIT STATUS 156: snapshot error encountered

Disk E:\ is cluster shared disk. In normal state disk E:\ is mounted on active node. I retrieve backup Shadow Copy Components of passive cluster node.

In directory e:\liga\ is installed some very !@#$ application and it is a lot of headache for administration :)

How to exclude e:\liga from Shadow Copy Components?

w2koption -backup -ignore_unresolved_volumes doesn't work in this case.

Detailed bpbkar log in attachment. I think bpfis log is no longer relevant.

 

UDP: system have enough disk space on C:\ :

used:11,2Gb

free: 18,6Gb

mph999
Level 6
Employee Accredited

bpfis is the log required for a 156 status.  There may be errors in other logs as seen, but the starting point is bpfis.

Martin

vakulenko_sergi
Level 3
Partner Accredited

Also I manually play with bpfis on client:

C:\Program Files\Veritas\NetBackup\bin>bpfis create -fim VSS -id test1 C:\
C:\Program Files\Veritas\NetBackup\bin>bpfis delete -id test1 -copy 1

After this commands bpfis create log in /veritas/netbackup/logs/bpfis. This log in attachement.

vakulenko_sergi
Level 3
Partner Accredited

Please tell, how to get bpfis log.

I setup global logging level to 5 in Master server properties->Logging.

In Client properties->Logging I setup all logging levels to 5. In Client properties->Windows Client->Client Settings I setup additional logging levels to maximum.

On master-media server and client system in folder .../veritas/netbackup/logs I create folder bpfis.

But after failed backup job directory .../veritas/netbackup/logs/bpfis on master-media and client is empty.

What am I missing?

Yogesh9881
Level 6
Accredited

go to client ... & post o/p of below commands (start CMD by run as administartor)

vssadmin list writers

vssadmin list providers

***************************

to test snapshot on OS level run below on Client

vssadmin create shadow /For=D: /AutoRetry=2   (D: for D drive & test this on all drive ).

also check do u hav enough free space on each drive....i hope this help you :)

vakulenko_sergi
Level 3
Partner Accredited

Output for vssadmin command:

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Administrator>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Provider name: 'Microsoft Software Shadow Copy provider 1.0'
   Provider type: System
   Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   Version: 1.0.0.7

------------------------------------------

C:\Users\Administrator>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Writer name: 'Task Scheduler Writer'
   Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
   Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
   State: [1] Stable
   Last error: No error

Writer name: 'VSS Metadata Store Writer'
   Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
   Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
   State: [1] Stable
   Last error: No error

Writer name: 'Performance Counters Writer'
   Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
   Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
   State: [1] Stable
   Last error: No error

Writer name: 'FSRM Writer'
   Writer Id: {12ce4370-5bb7-4c58-a76a-e5d5097e3674}
   Writer Instance Id: {a719bbf9-82a4-4d4b-9365-a6a87f7d855b}
   State: [1] Stable
   Last error: No error

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {b608c1ed-8e90-4406-a044-5ba761f5947a}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {c6c87ac7-ca32-4175-9686-ff9c3d18a2f6}
   State: [1] Stable
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {77d6e5bf-c8e8-49db-af73-8116a81c657b}
   State: [1] Stable
   Last error: No error

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {2c2db6f4-e6d5-461d-865d-27a824c20efb}
   State: [1] Stable
   Last error: No error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {42f96162-ca09-4e78-b7cd-275d3fd1a311}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {7bd29c1f-664e-46b4-b593-0d975021e073}
   State: [1] Stable
   Last error: No error

Writer name: 'Cluster Database'
   Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
   Writer Instance Id: {103bdd7f-0146-40e1-87e4-8c7b8a643802}
   State: [1] Stable
   Last error: No error

------------------------------------------

C:\Users\Administrator>vssadmin create shadow /For=C: /AutoRetry=2
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Successfully created shadow copy for 'C:\'
    Shadow Copy ID: {0078a8e0-5e0a-4ca4-9347-32a37d1f405e}
    Shadow Copy Volume Name: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4

C:\Users\Administrator>

------------------------------------------

C:\Users\Administrator>vssadmin delete shadows /all
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Do you really want to delete 1 shadow copies (Y/N): [N]? y

Successfully deleted 1 shadow copies.

C:\Users\Administrator>

------------------------------------------

I tried to backup C:\ and Shadow Copy Components on passive cluster node cls05-node-02. This client have mounted only disk C:\. All other disks is shared and mounted on active node - cls05-node-01.

I think, application in directory E:\liga on shared cluster disk E:\ during the installation process recorded some system data in (System State:\System Files\System Files), and this data placed on unreachable (for passive node) disk E:\liga.When I try backup Shadow Copy Components (include System State:\System Files) client can't find some application data on E:\liga and fail.

PS: Application on E:\liga require custom HASP and HASP-driver to be run.

PSS: Disk C:\ have enought free space for shap:

used:11,2Gb

free: 18,6G

 

 

 

Yogesh9881
Level 6
Accredited

What is backup selection in NBU policy ??

vakulenko_sergi
Level 3
Partner Accredited

Backup selection is

Shadow Copy Components:\

C:\

Policy type: MS-Windows

Will_Restore
Level 6

same error, different version:

https://www-secure.symantec.com/connect/forums/backup-virtual-machine-agent-error-156

 

I would open a support case.

mph999
Level 6
Employee Accredited

.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The following 2 TNs may help:

http://www.symantec.com/docs/TECH128805

http://www.symantec.com/docs/TECH76141

The problem is caused by W2008 cluster, regardless of NBU version. See extract from TECH128805 :

Newly in Windows 2008 and Windows 2008 R2, every "Path to Executable" for every Windows Service is cataloged into the VSS System Writer and is therefore backed up by Shadow Copy Components.  When a Shadow Copy Components backup starts, all of the VSS Writers are interrogated to compile a list of Volumes on which files classified into Shadow Copy Components reside - then Microsoft takes a Snapshot of those volumes.
 
The Microsoft Volume Shadow Copy service is unaware that the node is Passive and the drive letter for the Service is not present.  It attempts to snap the absent volume and an unexpected error occurs, which causes bpbkar32.exe to produce an Application Fault.
 

vakulenko_sergi
Level 3
Partner Accredited

TN, described in http://www.symantec.com/docs/TECH128805 no work on Netbackup 7.5.0.1 client.

TN http://www.symantec.com/docs/TECH76141 also didn't work. My passive cluster node don't have

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SystemWriter\ExcludedBinaryPaths dir. 

vakulenko_sergi
Level 3
Partner Accredited

I tired...back to the last working configuration:

Uninstall Netbackup 7.5.0.1 client on passive cluster node and install Netbackup 7.1 client.

Apply hotfix NB_7.1_ET2256525_2 on client.

Exclude shared cluster disks: w2koption -backup -ignore_unresolved_volumes E:

Now all works fine, thanks for help, during the investigation I learned a lot of new information (for me).

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

My gut-feel is that  'w2koption -backup -ignore_unresolved_volumes ...'  would also work with NBU 7.5 client installation.

I have not downloaded the hotfix to see what files/binaries is included, but my assumption is that it contains a new version of win2koption that includes '-ignore_unresolved_volumes [list of volumes]'.

I have just checked command parameters on my laptop with 7.5 installed:

C:\>w2koption /?


w2koption: -backup | -restore [-display] [-server server_name]
       For -backup the optional parameters are:
           -system_state <0=disallow legacy backup, 1=allow legacy backup>

           -system_state setting only applies to
           Windows 2003 (and later) servers

           -ignore_unresolved_volumes [list of volumes]
               Ex. -ignore_unresolved_volumes G:l:
....... .....

vakulenko_sergi
Level 3
Partner Accredited

Yes, w2koption works in NBU7.5.0.1 client. With w2koption I exlude all shared cluster disks:

w2koption -backup -ignore_unresolved_volumes E:F:Q:

but Shadow Copy Components backup still fails (NBU client 7.5.0.1)

I hope this bug, which has already been fixed in 7.5.0.3. But I have not time to test/upgrade NBU infrastructure to 7.5.0.3 now. I'll do it after 5 july, but now still works on NBU 7.1 client :)

vakulenko_sergi
Level 3
Partner Accredited

I update my Netbackup environment to 7.5.0.3 (master-media and clients), but have same issue with backing up Shadow Copy Components on cls05-node-02.

MatsHolm
Level 4
Partner Accredited

Hi!

Also on 7.5.0.3 and same error on the passive node did anyone find a sulotion to this?

//Mats

gildoc
Level 2

Hi,

I had the same problem with NB 7.5.0.4 on a passive node of a Windows 2008 R2 cluster (with SAP installed) and I've solved adding this registry key

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SystemWriter\ExcludedBinaryPaths

Containing the executable path of the Windows service that was in charge of the active node and thus was on one of the cluster disk.

/Gildo

http://www.symantec.com/docs/TECH76141