Windows backups take a lot longer when specifying ALL_LOCAL_DRIVES directive.

[LT2013]

Hello,

I am running NBU 7.5.0.6, RH Linux Master and media servers.

we are migrating to Exchange 2010, and while we do plan on using the Exchange agent, currently I am simply trying to do an OS backup of the servers (we will be migrating to DAG env). So I added the new exchange servers to a regular Windows policy, with the ALL_LOCAL_DRIVES directive, and setup the appropriate excludes for Exchange (though exchange has yet to be installed), and it is taking over 2.5 hours to backup 14GB of data. If I explicitly list the local drives (5 in total), the backup takes less than 15 minutes.

From the bpbkar log, it appears the snapshots are taking almost exactly 30 minutes for each drive (sample output below)...

ALL_LOCAL_DRIVES:

9:48:38.118 AM: [41832.34124] <4> V_DetermineMountInfo: INF -   Adding NTFS Volume C:\ ==> \\?\Volume{b94e3504-45a4-11e4-9530-806e6f6e6963}\
9:48:38.118 AM: [41832.34124] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - ======================================================================
9:48:38.118 AM: [41832.34124] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - Attempting to create snapshots for 'C:'
9:48:38.118 AM: [41832.34124] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - CREATE request: D:\VERITAS\NetBackup\bin\bpfis create -owner NBU -fso -WOFB -fim VSS -id mmxpcy701_1419432482 C:\
10:18:59.321 AM: [41832.34124] <4> V_Snapshot::V_Snapshot_ParseBpfisOutput: INF - Snapshot creation, FIS_ID: mmxpcy701_1419432482
10:18:59.321 AM: [41832.34124] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - Snapshot creation was not successful
10:18:59.321 AM: [41832.34124] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - ======================================================================
10:18:59.321 AM: [41832.34124] <2> tar_base::V_vTarMsgW: INF - Volume snapshots not enabled
10:18:59.321 AM: [41832.34124] <2> tar_base::V_vTarMsgW: JBD - change journal NOT enabled for <C:\>
10:18:59.368 AM: [41832.34124] <4> folder_access::V_OpenForRead: INF - obtained disk quota data: C:
10:18:59.383 AM: [41832.34124] <4> dos_backup::tfs_scannext: parent of <C:\$Recycle.Bin\S-1-5-21-1312041997-2487223646-369738992-1014\desktop.ini> has been forced, forcing backup

Specifying each drive....

10:45:19.735 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - ======================================================================
10:45:19.735 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - Attempting to create snapshots for 'C:'
10:45:19.735 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - CREATE request: D:\VERITAS\NetBackup\bin\bpfis create -owner NBU -fso -WOFB -fim VSS -id mmxpcy701_1419608384 C:\
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_AddMappings: INF - input parameters:
    source path:<C:\>
      snap path:<\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy37\>
     mount path:<(null)>
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_GetSourceVolume: INF - input parameters:
      source:<C:\>
    snapshot:<\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy37\>
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_AddMappings: INF - Volume Snapshot Mapping: \\?\Volume{b94e3504-45a4-11e4-9530-806e6f6e6963}\ --> GLOBALROOT\Device\HarddiskVolumeShadowCopy37
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_ParseBpfisOutput: INF - Snapshot creation, FIS_ID: mmxpcy701_1419608384
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - Snapshot creation was successful
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - Snapshot provider: VSS
10:45:50.156 AM: [11304.26688] <4> V_Snapshot::V_Snapshot_CreateSnapshot: INF - ======================================================================
10:45:50.156 AM: [11304.26688] <2> tar_base::V_vTarMsgW: INF - Volume snapshots enabled
10:45:50.156 AM: [11304.26688] <2> tar_base::V_vTarMsgW: JBD - change journal NOT enabled for <C:\>

[LT2013]

So I did some more digging around, and after getting access to the Windows server, I saw some errors in the event viewer:

The Data ONTAP VSS Hardware Provider service failed to start due to the following error:
The service did not start due to a logon failure.

(the additional drives for Exchange use are on NetApp hardware).

I found the following technote: http://www.symantec.com/docs/TECH158127

and after issuing the w2koption command on the NBU client, and forcing the "System Provider" the "ALL_LOCAL_DRIVES" backup ran fine.

So for my own edification, I'm wondering why this affected the policy using ALL_LOCAL_DRIVES and not the one where I listed each drive? Also, since this server will eventually be Exchange, should I change the option to use "Software Provider"?

[Marianne]

For ALL_LOCAL_DRIVES, the VSS provider must be the OS VSS (Microsoft / System Provider).

We cannot tell you why it affected ALL_LOCAL_DRIVES only.

On Exchange Server, please remember to exclude all Exch data files and/or folders from OS backups.

You need to use NBU for Exchange agent to backup Exchange.

[Nicolai]

Are the exchange server virtual servers ?

I may indicate a I/O problem on the underlying Lun's.

[LT2013]

Nicolai,

The servers are physical.

After reading the technote, and applying the change to force use of the "system" provider, the backups ran fine. What was interestng is that while there was an issue with the Data OnTap VSS writer (the service login account has a bad password), the backup still completed, albeit slow...exactly 30 mintues slower per drive.

That, and the fact that if I explicitly listed all the drives, it ran fine.

[Nicolai]

Thanks for the update