Failure reading files, and partial backups.

[Toddman214]

Windows 2008r2 Master and media servers running 7.5.0.4

Hi all,

I've been reading through the forums here and web searching some solutions, but thought I'd reach out while I'm researching on my end. Not having much luck so far.  Our main company file share is sitting out on a NetApp filer, and of course contains many millions of files. The full backup of this data is over 9tb, and I've been using NDMP to back this data up. While the backups using NDMP are fairly stable, the restores can be rediculously slow (I've given up on that fight). Plus, NDMP restores are proprietary to the platform that the data was originally backed up on, so if/when we eliminate all of our Netapp appliances, I'll have yet another bridge to cross for restores.

But for now, I'm trying to run a cifs backup of this data using the MS-Windows policy type. I'm running a Full backup, then will run incrementals, and then will start playing with Synthetic backups for the fulls. The data is writing as I type this, and has completed about 4tb, so the backup is working, sorta. I'm jumping the gun a bit by reaching out, since the backup has not actually failed yet, but the best I can expect is a status 1, which *may* interfere with the Synthetic backup when I'm ready to try that.  I'm seeing a ton of these messages in the job details...................

6/3/2014 3:09:46 PM - Warning bpbrm(pid=9468) from client pdc00nbua804w.ohlogistics.com: WRN - can't open file: \\DDC-FS-01\XXX.XXXX (WIN32 32: The process cannot access the file because it is being used by another process. )

6/3/2014 2:14:04 PM - Error bpbrm(pid=9468) from client pdc00nbua804w.ohlogistics.com: ERR - failure reading file: \\DDC-FS-01\XXX.XXX.XXXXXX (WIN32 33: The process cannot access the file because another process has locked a portion of the file. )

6/3/2014 1:11:32 PM - Error bpbrm(pid=9468) from client pdc00nbua804w.ohlogistics.com: ERR - failure reading file: \\DDC-FS-01\XXX.XXX.XXXXXXX(WIN32 58: The specified server cannot perform the requested operation. )

There are so many, the job details tuncate the message after 10 occurences.

I'll gladly provided any information needed. Hoping some of you have experienced similar issues and can shed some light.

Thank you!

Todd

[Mark_Solutions]

The problem you have with backing up files in that way is that the VSS writers will have a hard time locking the open files and so they will get skipped.

Assuming you dont have anything nice like de-duplication which allows you to use Accelerator then hopefully you have an Enterprise Client License Spare.

If so then rather than use the CIFS in that way mount the file systems to your media server as mount points so that they get backed up as d:\mount\filesystem1\ etc.

That way VSS can go to work and do its best with the open files.

Better still use the Enterprise Client to do a Flashbackup-Windows policy against these mount points using the format \\.\D:\mount\filesystem1\

This allows for much faster backups, a lock on open files but the ability to restore individual files.

You can even multi stream the jobs for better performance

NEW_STREAM

\\.\D:\mount\filesystem1\folder1\

NEW_STREAM

\\.\D:\mount\filesystem1\folder2\

Hope this helps and gives some ideas

[SymTerry · Accepted solution]

NetBackup is not able to access these files because they are in use and therefore locked or it doesn't have permissions to those files. I would check that you can access those files with the permissions the NetBackup client runs under

Because your running backups through a windows share and not NDMP, you will face issues like this. The true solution is to backup via NDMP, but if your looking to get rid of your filers, that is not a viable solution.

[Toddman214]

Unfortunately, I cannot use the Accelerator features as yet. We do have de-dup as we use DataDomain, but my current units are older DD670's, which have just about reached capacity, and are running an older OS. I have two brand new DD4200's, one at each data center, just waiting to be deployed, and will be upgrading to NBU 7.6..XX soon. I should be able to begin playing with Accelerator then.  But, with Brocade installs, UCS, and other projects, my DD4200 installs are not considered a priority, so I have to work with what's in place for now. :( Sorry for all the background.

Currently, I have my policy pointing to one of my primary media servers as the Client.

The backup selection looks like below.

\\ddc-fs-01\XXX\Global
NEW_STREAM
\\ddc-fs-01\XXX\Apps
NEW_STREAM
\\ddc-fs-01\XXX\H_User
NEW_STREAM
\\ddc-fs-01\XXXF_User
 

It sounds like you are recommending additional configuration above what I currently have in place. I could potentially use a flashbackup-windows policy?

[Mark_Solutions]

VSS will never get a hold of open files on a UNC path so if you are on a capacity licensing model or have a spare Enterprise Client License then set those shares as mount points on the media server you currently use.

The policy type set to be flashbackup-windows (assuming it is a window media server) and your file list based on where you have set the mount point. You can still use multiple streams.

So if you set the share as a mount point of D:\DDCFS01\ that maps to \\ddc-fs-01\XXXX then you could have a policy list like:

NEW_STREAM
D:\DDCFS01\Global
NEW_STREAM
D:\DDCFS01\Apps
NEW_STREAM
D:\DDCFS01\H_User
NEW_STREAM
D:\DDCFS01\F_User

Worth giving it a try