cancel
Showing results for 
Search instead for 
Did you mean: 

Differential VM Backups with BackupExec 2010 R2 and AVVI

Jim_S
Level 4

Background:  I have BE 2010 R2 installed on a physical Windows 2008 R2 server that has access to the iSCSI SAN and my vCenter 4.1 server.  I'm trying to do a Full AVVI backup of my vSphere infrastructure over the weekend using a Disk-to-Disk-to-Tape policy I created and then have Differential Disk-to-Disk-toTape backups happen during the weekdays.  I'm using the SAN method and have GRT enabled for SQL.  I have the Remote Agent for Windows servers pushed out to all Windows servers on the network, and I've checked to make sure the VMware VSS provider and synch driver are uninstalled.

 

After some tweaking, my Full backups are running OK  (793GB in ~ 13 hours for the toDisk portion, and then another 9 hours to duplicate that to LTO3 via an HP Autoloader).  It's a little slower than I expected, but I'll work on tweaking it later.

 

The primary issue I'm having now is that my daily Differential backups are taking almost as long (11 hours and 6 hours respectively for the disk and tape portions) and have nearly as much data in them.  After poring over the logs and troubleshooting this myself I've found that it is primarily due to a rather large virtual compatibility RDM that always gets backed up with the Full method.  When I look at the DataStore for this VM I can see the .chk files related to the virtual RDM, so vSphere is change tracking the blocks on it, however BackupExec refuses to perform a differential backup on it. 

 

I had a heck of a time finding anything about this in the documentation.  I couldn't see anywhere in the Administrator Guide anything about RDMs breaking differential or incremental backups.  I had to search google for quite a while before I found a AVVI agent FAQ that stated that RDMs (both virtual and physical) were not supported for differential or incremental backups using the AVVI.  I can totally understand the physical RDMs not working, hell they don't work at all since you can't snapshot a VM witha physical RDM, but I can't understand why a virtual RDM causes any issues, especialy when vSphere 4.1 clearly is tracking the changed blocks.

 

My current plan of action is to create a new datastore and then use Storage vMotion to change the virtual RDM into a vmdk on a VMFS datastore, but I don't really want to do this as it will cause some replication issues on my NetApp SAN due to all the changed blocks at the SAN level this will cause.  My current workaround is to disable differential backups and use standard network agent backups to just backup data, but this is not an ideal solution long-term.

 

Does anyone have any ideas for a different solution for this virtual RDM issue?  Can anyone from Symantec please chime in as to why virtual RDMs are handled in this way?

 

Thanks,

 

Jim S

 

10 REPLIES 10

Colin_Weaver
Moderator
Moderator
Employee Accredited Certified

Hmm - I am wondering if you might have just helped pinpoint why some customers have reported AVVI Differential backups running as fulls.

I will try and setup a test of this (can't promise a timescale) just to see if I get the same results against a Virtual RDM

 

Jim_S
Level 4

Check page 14 of this FAQ.  In there it states that Diffs won't work if there is a virtual RDM.

 

However, the Admin guide says nothing about this...

teiva-boy
Level 6

You know much of this could be the fault of VMware's vStorage API, which has a pretty severe set of limitations that backup vendors must integrate with...   The whole physical RDM thing is 100% vmware, and not Symantec's fault.  For a virtual RDM, I wouldnt be surprised if it were the same.

Jim_S
Level 4

I've done a little more digging around and I noticed that Veeam 4.0 didn't properly support virtual-mode RDMs, but in the Veeam 4.1 release they now support change-block tracking in virtual-mode RDMs.  I posted on the VMware Community Forums asking for a clarification on whether vSphere 4.1 fully supports CBT on vRDMs: http://communities.vmware.com/thread/289166

Jim_S
Level 4

Sure, I fully understand the physical RDM issue since VMware can't/won't snapshot a physical RDM.  However, it looks like the virtual-mode RDM appears to be tracking changed blocks. 

On the attached picture you can see the datastore for one of the VMs in question.  DirDB_2.vmdk is the RDM mapping for the vRDM and you can plainly see the associated CTK file for that VMDK.

Jim_S
Level 4

I compared some log files from a full and the subsequent differential backup.  My issue does directly relate to the VM with a vRDM:

 

Server Full Backup from Policy - 10/13/2010 6:19AM Differential Backup from Policy - 10/13/2010 7:00 PM Notes
Full bytes Full time Diff bytes Diff time
DirDB 286,934,302,987 3:33:45 560,905,966,583 7:38:40 Windows 2003 w/ SQL 2005 and File storage.  File Storage on vRDM

 

As you can see the Full backup size is actually smaller than the differential backup size.  The full backup is just backing up the used blocks, whereas the differential backup is backing up the entire VMDK series with used and free space accounted for. 

 

So basically not only is it not using CBT, it's doing a FULL size backup of this VM not even taking into account zeroed blocks.

Jim_S
Level 4

Per the Designing Backup Solutions for VMware vSphere document (attached):

 

Limitations on Changed Block Tracking

Changed Block Tracking does not work in any of the following cases:

Virtual hardware version is 6 or earlier.

The virtual disk is a “physical compatibility RDM.”

The virtual disk is attached to a shared virtual SCSI bus.

Changed Block Tracking can be enabled on virtual machines that have disks like these, but when queried for

their change ID, these disks always return an empty string. So if you have a virtual machine with a regular

“O/S disk” and a passthrough RDM as a “data disk” you can track changes on the “O/S disk” only.

  

So according to VMware's documentation Change Block Tracking does work on virtual-mode RDMs, just not physical mode.  Also VMware's vDR appliance supports differential backups using changed block tracking and does work on virtual-mode RDMs.ware's  

Jim_S
Level 4

I storage vMotioned the RDM to a VMDK on a regular VMFS datastore.  Will let the differential job run again tonight and see what the results are.

Colin_Weaver
Moderator
Moderator
Employee Accredited Certified

Just for completeness I have tested this with Backup Exec 2010 R2 and VMware ESX 4.1.0,260247

Differential backup is still currently running but it looks like not only is it doing a full backup of the vRDM, BUT perhaps more crucially it is NOT ignoring the white space and is therefore trying to backup the complete size of the vRDM (making it very very slow to complete)

I guess for the time being Symantec stand by the information in the FAQ which states that vRDM is not supported by AVVI for Differential and Incremental Backups.

As such please visit the Ideas section at the bottom of the Forums menu at the top of this web page and submit an enhancement request.

 

mikdivad
Not applicable

Do you have any update? How do your Full and Differential jobs run after moving the data to a VMDK?