Backup Exec 12.5 sp2 Hyper-V backups

[Matty_M]

Hi,

Not sure what could be causing this one but I have Backup Exec 12.5 SP2 with latest hotfixes running on a Windows 2003 R2 Server. It remotely backs up a Windows 2008 SP2 Hyper-V server. The Hyper-V server has about 5 VMs running. When the backup runs, sporadically after about 2 or 3 weeks it says Snapshot Processing, meanwhile the Windows 2008 Server has completely hung/becomes unresponsive. The only way to get it working again is to power off the system and restart which is not very good.

There is nothing in the event viewer of each system to indicate a fault of any sort, the last entry logged on each system is -

Log Name:      Application
Source:        VSS
Date:          07/09/2009 22:54:15
Event ID:      12322
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      vicky
Description:
Revert operation for volume \\?\Volume{cfa93f87-08eb-11de-8a8f-001ec9f11b61}\ has begun.  Volume is being reverted to the shadow copy with id {bac21b01-f103-4fa2-88df-e6388f873729}.

Operation:
   Revert a Shadow Copy

Context:
   Execution Context: Coordinator
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="VSS" />
    <EventID Qualifiers="0">12322</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2009-09-07T21:54:15.000Z" />
    <EventRecordID>4154</EventRecordID>
    <Channel>Application</Channel>
    <Computer>vicky</Computer>
    <Security />
  </System>
  <EventData>
    <Data>\\?\Volume{cfa93f87-08eb-11de-8a8f-001ec9f11b61}\</Data>
    <Data>{bac21b01-f103-4fa2-88df-e6388f873729}</Data>
    <Data>

Operation:
   Revert a Shadow Copy

Context:
   Execution Context: Coordinator</Data>
    <Binary>2D20436F64653A20434F52434F4F524330303030313531332D2043616C6C3A20434F52434F4F524330303030313436322D205049443A202030303030333935322D205449443A202030303030353139362D20434D443A2020433A5C57696E646F77735C73797374656D33325C76737376632E6578652020202D20557365723A204E5420415554484F524954595C53595354454D20202020202D205369643A2020532D312D352D3138</Binary>
  </EventData>
</Event>

being different for each VM. I also see a couple of Load Perf counters loading and thats it, nothing else until I power the whole windows 2008 server off and start the backup again, because the job wont cancel I also have to reboot the Windows 2003 R2 server. The server running Windows 2008 is a Dell Poweredge 2950 III. All drivers/firmware are up to date on all systems.

Anyone else had this? I think I may have to move back to VMWare :(

Many thanks

[Fan]

We just had this happen last night for the first time after 2 weeks of successful backups.  Basically it was catastrophic since the hyper-v server was hosting our network services servers.  Signing into the backup server this morning the hyper-v job was hung at snapshot processing.   We had the same issues all the way around.  Had to reboot the backup server to free the job. 

Both servers are 2008 64bit fully patched and BE 12.5 is fully patched as well.  I'm still trying to hunt down any information as to the cause of this problem but have been completely unsuccessful.  The higher ups are pressuring me for answers as to why our entire network was down this morning and specific answers are hard to come by.  Any help would be greatly appreciated.

[Matty_M]

Out of interest are you using a Dell Poweredge?

[Fan]

The Backupserver is a poweredge 1950 and the Hyper-V host is a Dell R900.  One of my colleagues was testing V-locity on the server that day.  We are hoping that was part of the problem and removed it.  Are you running any similar software on the host?

[Matty_M]

Nope not running anything like that. I thought it might have been the DRAC card virtual drives causing it on the Hyper-V server, because it does some weird mounting and unmounting of the shadow copies I think, I know the DRAC card used to cause havoc periodically with Linux ES4 some years back and it was never fixed properly. I did get an error last night saying failed to query the writer status which makes no sense as the backup has run fine since I last posted. It didnt cause the server to lock up though.