Exchange 2010 DAG and VSS Writer failures (0x800423f3)

[StrongmanTech]

Recently, my Backup Exec 2012 SP1a install has been receiving failures from 1 of my 2 Exchange 2010 databases under DAG.

The VSS errors state: The VSS Writer failed, but the operation can be retried (0x800423f3)

The Backup Exec Job fails with: V-79-57344-862 - The Backup Exec Agent for Microsoft Exchange was not used to create the last full backup of this database.

Symantec Tech Support's initial fix was to email me this technote: http://www.symantec.com/business/support/index?page=content&id=TECH141370 which states the solution is to backup only the Active Database Copy.

This is not an option, as the passive database copy and the backup server are in the same data center, with the active copy being in the client main office. I'm not backing up the active db over a WAN link. 50Mbps MetroE connectivity or not, it'll still be slow.

With the Passive Copies of the two DB's on the same exchange server, why is one failing and the other not. And, why is it reporting that Backup Exec did not create this backup?

Anyone else see this yet? If so, did you get it resolved?

[EliasA]

Two troubleshooting suggestions:

1. Are you receiving this error during incremental backups?  If so, perform a full backup on the problem database followed by an incremental.

2. Have you tried to restart the MSExchangeRepl service on the passive copy Exchange server?

[StrongmanTech]

Elias,

I am receiving this issue on Full and Incremental backups.

I have not tried restarting the particular service by itself, but the active and passive servers go through reboots every Sunday morning for maintenance, patching, etc.

Mailbox Database 1's last successful backup was on 6/30. Currently flying by the seat of my pants with this client. Hope they don't ask for a restore any time soon.

[EliasA]

What is the 'Copy Status' of both passive database copies?

[StrongmanTech]

Healthy on both.

After running backups, this is what VSS lists:

Writer name: 'Microsoft Exchange Replica Writer'
   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {79b1dc1a-06d5-4219-a136-c334bf0a685c}
   State: [7] Failed
   Last error: Retryable error

Writer name: 'Microsoft Exchange Writer'
   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {74f2b714-318f-4ff0-a2e4-aee05e6e683d}
   State: [1] Stable
   Last error: No error

[EliasA]

Thanks.  I'm taking this information and checking with engineering....I'll get back to you as soon as I hear anything.

[EliasA]

Thanks for your patience.  Here is the information I found on this topic:

You should check the Event Log to see if there are any errors listed there.  The "retryable error" is not an easy one to diagnose.

Also suggest to enable verbose Exchange VSS Writer Event logging via PowerShell. 
To do this from a PowerShell commend:

C:\Windows\system32>get-eventloglevel *vss* | Set-EventLogLevel -Level expert
C:\Windows\system32>get-eventloglevel *vss*

Identity /   EventLevel
-----------------------------
MSExchange Repl\Exchange VSS Writer     /   Expert

To set this back to normal when it has captured the log, again from a PowerShell command:

get-eventloglevel *vss* | Set-EventLogLevel -Level lowest

In addition, it would be helpful to also collect the corresponding RAWS debug log with the event logs from the Exchange server for the failing backup job.  information on how to enable this level of logging can be found here:  http://www.symantec.com/business/support/index?page=content&id=TECH83408