Forum Discussion

lege's avatar
lege
Level 3
15 years ago

Backup Exec 2010 and Dell MD3000i VSS provider issues

Hello,

We are trying to use BE2010 to set up backups for a Hyper-V cluster on Windows Server 2008 R2. Our SAN is an iSCSI Dell MD3000i. We have installed the hardware provider on all the Hyper-V hosts, and it shows up properly in vssadmin list providers:


vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool

(C) Copyright 2001-2005 Microsoft Corp.
 
Provider name: 'SmVssProvider'
   Provider type: Hardware
   Provider Id: {24e4e8a3-69cb-4370-8b7b-c5276cd49765}
   Version: 10.60.01
 
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
   Provider type: System
   Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   Version: 1.0.0.7


If we enable Advanced Disk-Based backup in Backup Exec to move the backup snapshot to the backup server, it does not find the provider and comes up with this error:

The job failed with the following error: Backup Exec could not locate a Microsoft Volume Shadow Copy Services (VSS) software or hardware snapshot provider for this job. Select a valid VSS snapshot provider for the target computer.
If we disable the Advanced Disk-Based backup option, that error doesn't show up, however, we get an error saying The job failed

There's this additional information logged by Backup Exec:


 - AOFO: Initialization failure on: "VRTSRV::\\Hyper-V.domain.int\Hyper-V?HA?VM\VM1". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
 
V-79-10000-11226 - VSS Snapshot error. The Microsoft Volume Shadow Copy Service (VSS) snapshot provider selected returned: "Unexpected provider error". Ensure that all provider services are enabled and can be started. Check the Windows Event Viewer for details.
 
The Event Log on the target computer shows:


Volume Shadow Copy Service error: Unexpected error calling routine DeviceIoControl(IOCTL_VOLUME_GET_GPT_ATTRIBUTES).  hr = 0x80070002, The system cannot find the file specified.
. 
 
Operation:
   Processing PreFinalCommitSnapshots
   Executing Asynchronous Operation
 
Context:
   Provider Name: SmVssProvider
   Provider Version: 10.60.4
   Provider ID: {24e4e8a3-69cb-4370-8b7b-c5276cd49765}
   Current State: DoSnapshotSet

Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {24e4e8a3-69cb-4370-8b7b-c5276cd49765}. Routine details PreFinalCommitSnapshots({c9faf664-b4bb-4ce7-b975-9b423a6554d0}, 2) [hr = 0x80042302, A Volume Shadow Copy Service component encountered an unexpected error.
Check the Application event log for more information.
]. 
 
Operation:
   Executing Asynchronous Operation
 
Context:
   Current State: DoSnapshotSet

We have contacted Dell assuming it was a problem with their Hardware VSS provider software, but they insisted the problem was with Backup Exec and that hardware VSS provider is working as designed.

The BE2010 documentation states that SmVssProvider for MD3000i is supported. 

On a related note, we have also tried DPM 2010 with the provider and the same error occurs in the VSS provider: http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.dataprotectionmanager&tid=b608a39e-b0cc-4f9c-b126-98ceb723476e&mid=b608a39e-b0cc-4f9c-b126-98ceb723476e&cat=&lang=&cr=&sloc=&p=1


Again, Dell claims it's not their problem, so any help in getting this to work for us would be appreciated. We're interested in Backup Exec but can't license if this doesn't work.

If the problem turns out to be indeed in the VSS provider, please let us know so we can refer Dell to the official Symantec response.

Thank you very much.

2010
Advanced Disk-Based Backup Option
Agent for Microsoft Hyper-V
Backup and Recovery
Backup Exec
Windows

16 Replies

Sort By
Show More
  • lege's avatar
    lege
    Level 3
    14 years ago

    I would have to respectfully disagree on the offhost backup part. :) 

    Here's Dell's official technical paper on this: http://www.dell.com/downloads/global/products/pvaul/en/powervault-md3200-series-hyper-v-protection.pdf

    Quote:

     

    A more convenient way to back up VMs with Backup Exec 12.5 is to use the Microsoft VSS Service with
    the MD3200 or MD3200i storage array VSS hardware provider that is included on the MD3200 / MD3200i
    series of storage arrays series of storage arrays Resource CD. This method lets you create hardwarebased
    snapshots on the MD3200 or MD3200i storage array and map them automatically to your
    dedicated backup server without performing any scripting.
     
    Note: Their documentation mentions Backup Exec 12.5 which is an older version and didn't support CSVs, but Backup Exec 2010 R2 does support CSVs as mentioned here: http://www.symantec.com/redirects/backup_exec/hypervfaq/ (New! Support for 2008 Server Hyper-V R2 Cluster Shared Volumes (CSVs).)
     
    Hardware VSS is a supported scenario for Hyper-V CSV backups. Even Microsoft DPM 2010 can do it (and yet it fails with the same error)

    You can even take a VSS snapshot from the command line using hardware VSS over CSVs.

    A VSS provider backup is not an offhost backup unless you choose to transport the snapshot volume over to the media server, which is optional. Otherwise the snapshot volume is mounted on the host and the Backup Exec agent just streams the data over to the media server.

    Also, since Microsoft DPM 2010 can also do the offhost backup part, I really doubt Backup Exec couldn't, but I'm not sure. However, this thread is not really about offhost backups.

     

    Backup Exec is supposed to be much more mature, and it is why we're looking into it as our first option of licensing.

    Again. I have created a screencast of all the steps I'm going through, from setting up the backup to when the error appears in the event log, it's only a few minutes long, so if anyone wants to help and take a look at it, please let me know and I'll PM you the link.

    Thanks!

  • kpalmer954's avatar
    kpalmer954
    Level 2
    14 years ago

    lege,

    It sounds like you have gotten as far as I have with this and it isn't working. Have you installed the latest version of the Hyper-V integration components in all of the guests you are attempting to back up?

    Have you verified the following hotfix is installed correctly?

    http://xtravirt.com/dpm-2010-rc-and-hyper-v-r2-vm%E2%80%99s-csv%E2%80%99s-order-patches

    Hotfix uninstall on Server 2008 Core?

    http://arstechnica.com/civis/viewtopic.php?f=17&t=61524

     

    Hi

    Its not a very smooth operation, but i works 

     

    -Uninstalling a hotfix

    • Open a command-prompt and execute Expand /f:* filename.msu c:\Patch

    • Navigate to c:\Patch\ and open filename.xml in a text editor.

    • In filename.xml, replace Install with Remove and save the file

    In the command prompt, type:

    Pkgmgr /n: filename.xml

    Checking Integration Components version on Hyper-V RTM

    http://blogs.technet.com/b/cedward/archive/2009/06/12/checking-integration-components-version-on-hyper-v-rtm.aspx

     

    Right click / properties on the c:\windows\system32\drivers\vmbus.sys file in all of the hosts. Product version should be 6.1.7600.20542 or newer.

     

    - Reinstall both hotfixes in the correct order as described in the article.

    - Install the latest integration components on all of the guests.

    - Verify the integration component version is correct on all of the guests.

     

    I have an MD3000i that I was never able to get to work with the Hardware VSS provider. I'm getting ready to give it one more try before moving to Equalogic. I worked with Microsoft Paid Product Support and the engineer was unable to get the hardware VSS provider to work with or without CSV support. I'm trying to use the MD3000i hardware VSS provider with DPM 2010. Microsoft and I both tested with the Vshadow tool from the VSS SDK.

    Kevin

  • diz's avatar
    diz
    Level 2
    14 years ago

    Are there errors in windows application log of backup server from smprovider?

  • Pascal-FR's avatar
    Pascal-FR
    Level 2
    14 years ago

    Hello, I found this quite interesting thread while looking at "Hyper-V off host backup".

    I'm preparing the same hardware infrastructure as Lege did:

    A Hyper-V cluster based on Windows Server 2008 R2 (2 nodes) attached to a DELL MD3200i iSCSI SAN.

    I've asked DELL Consultant this morning about Guest VM backup stored on the CSV at the SAN. He told me this can be done with the "Enterprise Licence" for the Storage array:

    This is an optional licence key you enter in your MD3200i that unlocks the Snapshot feature within the SAN firmware.

    Currently I do not have this Key, and I fail to do a hardware snapshot using vssadmin from the Server where a LUN from the SAN is attached.

    Two white papers describe what we all want to do:

    PowerVault MD32x0 and MD32x0i Data Protection Whitepaper (explain what the features keys are for, and what VSS and snapshot)

    http://www.dell.com/downloads/global/products/pvaul/en/powervault-md32x0-md32x0i-data-protection-whitepaper.pdf

     and the one pointed by Lege

    http://www.dell.com/downloads/global/products/pvaul/en/powervault-md3200-series-hyper-v-protection.pdf

     

    Lege, if your screenCam is still available could you PM me the URL. It seems we're facing the same issue, and i'm currently leaving BE 12.5 moving to DPM2010.

    Regards,

    pascal.

    My current MD3200i is not at the latest Firmware issued 18 january 2011.

    C:\Program Files (x86)\Dell\MD Storage Manager\util>vssadmin list providers
    vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
    (C) Copyright 2001-2005 Microsoft Corp.

    Provider name: 'SmVssProvider'
       Provider type: Hardware
       Provider Id: {24e4e8a3-69cb-4370-8b7b-c5276cd49765}
       Version: 10.70.08    <----- This could be updated in the new DELL MD3200i CD

    Provider name: 'Microsoft Software Shadow Copy provider 1.0'
       Provider type: System
       Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
       Version: 1.0.0.7

     

     

  • lege's avatar
    lege
    Level 3
    14 years ago

    We opened a support ticket with Microsoft, and the Microsoft engineer verified the obvious, and confirmed the problem is with Dell's VSS provider and we then confronted Dell about it, but Dell completely ignored us.

    Apparently, they know that CSV backups are not supported (buggy) in their VSS providers, and because these SANs are not even produced by DELL, and they just resell from LSI, they don't care and won't do anything to fix it.

    Needless to say, we won't be buying Dell again.

  • Pascal-FR's avatar
    Pascal-FR
    Level 2
    14 years ago

    I will call back my DELL Rep and see what he think about this (CSV not working) anf no fix proposed.

    My Rep. clearly admited the other point you mention:

    The DELL PowerVault MD3200i and the HP StorageWorks P2000 are basically the same hardware; they’re both rebadged LSI Engenio enclosures. Even IBM resel it in its portfolio.

    My Rep was having a Sales Tool, PDF Table compraison of the 3 offers, with features, differences and so...

    It's a pitty you didn't get any fix. The main reson to puchase this Storage Array was to build an Hyper-V cluster (with CSV of course).

    I will post the DELL update in this thread.

    Pascal.