Exchange 2003 Backup fails due to VSS

[Tech_Support_4]

hi,

Its the last week, i am facing issue to my Exchagne 2003 backup (differential and full).

The VSS crashes and fails the exchange backup.

the following are the error I have received:

1st Error, Had to cancle the job.

EXCHANGEAOFO: Initialization failure on: "\\EXCHANGE\Microsoft Information Store\Second Storage Group". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
Snapshot provider error (0xE000FED1): A failure occurred querying the Writer status.

Check the Windows Event Viewer for details.
Writer Name: Exchange Server, Writer ID: {76FE1AC4-15F7-4BCD-987E-8E1ACB462FB7}, Last error: The VSS Writer failed, but the operation can be retried (0x800423f3), State: Failed during freeze operation (9).  Consult your Microsoft documentation for details. Check the Windows Event Viewer for corresponding application or system errors.
The following volumes are dependent on resource: "J:" .

2nd Error

AOFO: Initialization failure on: "\\EXCHANGE\Microsoft Information Store\Second Storage Group". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
Snapshot provider error (0xE000FED1): A failure occurred querying the Writer status.
Check the Windows Event Viewer for details.
Writer Name: Exchange Server, Writer ID: {76FE1AC4-15F7-4BCD-987E-8E1ACB462FB7}, Last error: The VSS Writer failed, but the operation can be retried (0x800423f3), State: Failed during freeze operation (9).  Consult your Microsoft documentation for details. Check the Windows Event Viewer for corresponding application or system errors.

3rd Error, For Full Backup (weekly)

AOFO: Initialization failure on: "\\EXCHANGE\Microsoft Information Store\Second Storage Group". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
Snapshot provider error (0xE0008527): VERITAS Storage Foundation for Windows (VSFW) unexpectedly failed. Check that the VSFW service is running, and check the status of existing disk groups and volumes.
Check the Windows Event Viewer for details.
AOFO: Initialization failure on: "\\EXCHANGE\Microsoft Information Store\First Storage Group". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
Snapshot provider error (0xE0008527): VERITAS Storage Foundation for Windows (VSFW) unexpectedly failed. Check that the VSFW service is running, and check the status of existing disk groups and volumes.
Check the Windows Event Viewer for details.

During the 2nd Error, AS most articles i had read, were mentioning that it will resolve during the Full Backup. But it failed with the 3rd Error.

Now am not sure what steps to take.

either register the VSS providers or create a new Backup Selection for my Exchange 2003.

Please guide me to resolve this issue.

Thanks

[CraigV]

Hi,

A new selection list won't do anything...this isn't a corruption issue, but rather lies with Windows.

I suspect you're using Windows Server 2003, so do a search on Google for Windows Server 2003 VSS Rollup Patch and download that.

Try the troubleshooting steps in the following order:

1. Reregister the VSS *.dlls (doesn't require a server restart).

2. Restart the server - this can normalise the VSS writers again...

3. Run the VSS rollup patch which will update VSS on Win2K3. <-- I'd recommend loading this anyway.

Thanks!

[Tech_Support_4]

I have restarted the server on the 2nd Error. But it fail on the Full Backup.

After 2nd Error, i did restart the server and check the VSS writers, it gave all [1].Stable with No Errors

Now as this is my production server, running exchange, I dont want to do any more trial & errors.

So Should I register the VSS dlls or just update with rollup patch.

Where will the registeration of dll be done? on the exchagne server or the backup server?

As well the patch to be on the exchange server or the backup server?

Does it require restart of the server.

Sorry, in confused state, or please support me.

[CraigV]

The reregistration is done on the server with the VSS errors only.

The VSS Rollup patch will require a restart, but it does address VSS errors. If it is urgent, and another restart is a couple of days off (outside of emergency changes!), then do the rollup patch first.

[pkh]

It appears that you have Veritas Storage Foundation.  You might want to specify a snapshot provider rather than let AOFO choose one automatically.  Go to AOFO and choose either VSS or Veritas Storage Foundation explicitly.

[Tech_Support_4]

pkh: Which provider should i use Microsoft Volume Shadow Copy or Veritas Storage Foundation.

How to i test my backup sucessfully (justing running the system state, but i guess the system state is not related to exchange and its drive J). Should i have to run a full backup for exchange, as the last full failed.

[pkh]

I think you should try VSS first.

[CraigV]

...always advisable to run a full backup if your last 1 failed

[Tech_Support_4]

How do I test it?

[pkh]

Just go to the job properties, select AOFO and then select VSS (third option).  Don't select the first option.

[Tech_Support_4]

The 3rd Option is Veritas Storage Foundation. So Select Veritas Storage Foundation or, Select from the Microsoft Volume Shadwo Copy, and from the dropdown, select System - Use Microdoft Software Shadow Copy Provider.

I current setup is automatic as attached image screenshot.

[Tech_Support_4]

So I have to wait for my weekend, as currently we are running on a single tape drive, for doing our backups.

[CraigV]

...this is entirely up to you. You can conceivably run a full backup of Exchange during the day if getting a full backup is critical to business continuity. If it was up to me (and I do full backups on my sites), I would do that. I personally off-set having no backups and something failing vs. a bit of slow-down for users for the hour or 2 it takes to run a backup. Plus, someone is on site to change tapes on those sites with single-slot drives.

My 2c...

[pkh]

What I am saying is: Don't let AOFO choose i.e. use the first option.  Either use VSS or Storage Foundation.  If one does not work, use the other technology.

[Tech_Support_4]

My Full exchange backup takes 6hrs for a suceesffull backup.

[CraigV]

...again, that decision lies with you. Weigh up not having a full backup with having 1...

[Tech_Support_4]

The following are the state of VSS after re-registering the vss dll:

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {ec9325f6-691f-43d2-bd69-063053146bab}
   State: [1] Stable
   Last error: No error

Writer name: 'FRS Writer'
   Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
   Writer Instance Id: {9db020d6-e6b6-4ea2-8e87-dc1ad53e8bdd}
   State: [1] Stable
   Last error: No error

Writer name: 'Microsoft Exchange Writer'
   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {d6f53ee2-7ee8-437e-9b9d-04826cf7926f}
   State: [1] Stable
   Last error: No error

Writer name: 'MSDEWriter'
   Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}
   Writer Instance Id: {a5ebda10-42b0-4f2d-bf14-387e0d644b9f}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {7030263a-270a-49ca-8007-0c01b3c73e5d}
   State: [1] Stable
   Last error: No error

Writer name: 'NTDS'
   Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
   Writer Instance Id: {eb69a47e-a539-4679-b93b-603456e6372a}
   State: [1] Stable
   Last error: No error

Writer name: 'Event Log Writer'
   Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}
   Writer Instance Id: {8ac6a3d6-eed0-4b1d-8513-2d4582e30124}
   State: [1] Stable
   Last error: No error

Writer name: 'Cluster Service Writer'
   Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
   Writer Instance Id: {f55d9a18-70f9-41b1-82b0-946b416646a0}
   State: [1] Stable
   Last error: No error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {cc3af773-fd4f-494a-b883-d5ca974b238c}
   State: [1] Stable
   Last error: No error

Writer name: 'IIS Metabase Writer'
   Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
   Writer Instance Id: {bef0124e-a27b-448a-a1ae-6ee6e98a1ae1}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {c637ae4b-7f9d-451f-9417-c0e956463fda}
   State: [1] Stable
   Last error: No error

I have also applied the VSS Rollup patch, and restarted the server.

In BackupExec I have Selected System- Use Microsoft Software Shadow Copy Provider, in the AOFO options.

Before running the Backup (Full) i have started the VSS and Ms Software Shadow Copy services in the exchagne server; they were not started, and were Manual.

Now I have to wait for my excahnge full backup to complete successfully.