Recommendations for Removable Backup-to-Disk Folder Configuration

[m3ei]

We are backing up a single SBS 2003 R2 server to (directly attached) external USB hard drive, using a defined Removable Backup-to-Disk Folder. We have 4 of these hard drives (320GB each). Full backup size is currently about 75GB, with BE software compression enabled they're probably 45-50GB on disk. We're using BE 11d for Small Business Server.

Everything is working fine except we are increasingly encountering problems with jobs failing to complete, or not starting at all, because of "low disk space" on the USB hard drive. I suspect this is because of incorrect settings of "overwrite protection period" and/or "append" and/or other configuration options available for the Removable Backup-to-Disk folder. I admit I haven't encountered these concepts in other backup software packages I've used and my understanding of their use is probably incomplete. However, rather than detailing its current configuration I'd prefer to focus on what I'd like the BE installation to achieve.

At the most basic level, full backups are scheduled every evening. The following morning, the backup operator changes the USB hard drive and puts last night's drive in her bag to take with her when she goes home at the end of the day (i.e. off-site backup).

- For simplicity, we want to do complete backups every evening. We'd like to continue to do full backups unless there's a compelling reason otherwise

- Each USB hard drive can hold (at the moment) 5 or 6 complete backups. We recognize that backup size will grow over time and we may purchase additional external USB hard drives to increase our available archive window

- Top priority is that the current job must complete to the currently attached external USB hard drive. BE should overwrite any older backups present on the currently attached USB hard drive, starting with the oldest, as required to free up enough disk space for the current job to complete.

- As a second priority, BE should maintain as many backups as possible on each external USB hard drive.

- We cannot accept any configuration that demands that a certain USB hard drive be attached for a given night's backup job. Sometimes drives may not be switched or it's possible a drive may not be available because the backup operator is sick at home with the required drive. However, we can guarantee that a USB hard drive will be attached to the server for each nightly backup.

- At the moment, we have full backups scheduled for Saturday and Sunday evenings as well as the weekdays. Although there isn't much activity on weekends, Exchange is still receiving e-mails and there are some remote users working with server files and we'd like to back up these changes. The USB hard drive is not changed on the weekend. So, with a 4-day long weekend there could be as many as 5 sequential backups on a single USB hard drive. If this is a significant issue or stumbling block we can revisit weekend backups.

Can someone please recommend how to configure the Removable Backup-to-Disk Folder to support the above backup plan?

I don't know if any Symantec tech monitors these messages but if so I have a recommendation:

- Assuming that the above backup plan can be supported by BE 11d, could this be set up as an easy-to-find configuration item or template for a Removable Backup-to-Disk Folder? I've spent considerable time spelunking in the docs and help and could not find out how to do this. I'm sure there must be thousands, if not tens of thousands of other small businesses who want to implement this type of backup plan.

- I imagine that the "overwrite protection period" and other configuration options are very powerful for large/complex environments but it's hard to understand for the typical small business with a single server.

A related question:

- Is there a report available in BE that shows which backups remain available for restore jobs, that have not been overwritten in a Removable Backup-to-Disk folder? Something that can tell us, at a glance, how far back our current backups go? This would be very useful when deciding to buy more external USB hard drives, potentially using 2 or more at a time etc.

[David_Deakin]

Hi m3ei

Have your tried restoring data from your USB drives using a new installation of Backup Exec (i.e. testing a disaster scenario when your backup server is not available)? When using a backup scheme similar to yours I would get 'inconsistent data' failures when restoring, I believe due to swapping external drives used as a single removable backup-to-disk device.

I would love to hear from someone who has a successful method of rotating USB disks with backup-to-disk folders. From the docs it appears that you must have each USB disk assigned to a different drive letter for it to work properly. I have managed to get a rotation scheme working fairly reliably by mounting the disks on different folders (I don't have enough drive letters free) and creating separate backup-to-disk devices for each, but it seems to go wrong if Windows remounts the disk on the wrong folder (it seems unable to successfully identify which disk is which reliably) or the server is rebooted resulting in the backup-to-disk devices going offline.

Thanks

David Deakin

[m3ei]

>Have your tried restoring data from your USB drives using a new installation of Backup Exec (i.e. testing a disaster scenario when your backup server is not available)?

No - not yet. Something to keep in mind - thanks.

>I would love to hear from someone who has a successful method of rotating USB disks with backup-to-disk folders.

Me too. I'm a bit surprised that no-one has yet been able to offer suggestions on how to do this.

This issue with me is getting worse. It looks like BE is failing to recycle old media sets. I've set the global overwrite protection completely off (which BE highly recommends AGAINST) and I'm STILL getting backup failures due to "insufficient disk space".

Browsing briefly over other messages in this forum I've seen hints that other people are not getting BE to recycle old "media" as expected. I'm going to do a manual check for BE updates and apply any that may be available, then see what happens.

[Ken_Putnam]

What media set(s) are  your job(s) writing to?   Are the jobs Overwrite or Append?

from the Media Tab, right click that/those media sets\Properties

What is the OPP (Overwrite Protection Period)?

if the jobs are all Append, NONE of the BKF files will ever expire because they are all considered part of the same "media", and the overwrite protection is reset each time the "media" is closed

As far as rotation, Veritas and now Symantec are adamant that you must mount each USB/Firewire drive at the same drive letter each time that you connect it (That is that drive1 must always be mounted at N:, drive2 at O: etc).  I've maintained since B2D was introduced in v8.6 that this is a SERIOUS design flaw, but since I'm only a user, I apparently don't know how things should work.

To get BackupExec top write to whichever drive is mounted, you should create a drive pool containing all the B2D folders, and point your job(s) to the pool or to "all Pools", rather than a specific drive

[m3ei]

Thanks for your help:

>What media set(s) are  your job(s) writing to?   Are the jobs Overwrite or Append?

I've set up a new Media Set. OPP is currently 1 day. Append period is "None" ("Infinite - Allow Append" ). Media Vault under the Vault Rules tab is "None".

In Job Properties...Destination...Device and Media:

- "Device" is set to the Removable B2D folder I've set up
- "Media Set" is the new Media Set I've created
- "When this job begins" is set to "Append to media, overwrite if no appendable media is available"

The way I read it, the final option above should do what I want, but I could certainly change it to "Overwrite media" if that would be more correct or more reliable for my scenario.

I've set the global Overwrite Protection Level back to "Partial" (the default).

Re: drive mounting - the drives always mount as drive F: so this should not be an issue. Thanks for the advice should drive/folder pooling ever become necessary.

Message Edited by m3ei on 06-19-200712:07 PM

[Ken_Putnam]

- "When this job begins" is set to "Append to media, overwrite if no appendable media is available"

This will mean that none of the BKF files will ever be overwritten.  As I said before, when you append, BackupExec considers each  BKF file as part of one big "backup file".  Each time the "file " is closed, the overwrite period is extended - for every BKF file associated with it

If all your external drives mount on the same drive letter, you will probably run into problems, as Symantec explicity says not to do this. I'll dig around and see if I can find a reference for that

Generally, you would have two jobs - a weekly defined as Overwrite, and a daily defined as Append.  target the same media set ( you cannot append to a volume created by a different media set)

You can also use one job defined as "append, else overwrite", with the global media management option set to "use overwritable media in the target media set before scratch media.    On monday, it SHOULD look for overwritable media, find none, and then overwrite a scratch tape.   But for a single server, my way would only be two jobs, so no headaches keeping things straight

Set the OPP to 26  days and the APP to 6 days  (allow append for six days from the time that the first backup set is written, and protect for 22  days - three weeks plus a day - after the last set is closed )

[m3ei]

Thanks for your continued help:

>- "When this job begins" is set to "Append to media, overwrite if no appendable media is available"

>This will mean that none of the BKF files will ever be overwritten.  As I said before, when you append, BackupExec considers each  BKF file as part of one big "backup file".  Each time the "file " is closed, the overwrite period is extended - for every BKF file associated with it.

Thanks for clearing that up - I'll set that option to "overwrite".

Once overwriting is working reliably, we'll experiment with different/multiple job types and overwrite vs. append.

>If all your external drives mount on the same drive letter, you will probably run into problems, as Symantec explicity says not to do this. I'll dig around and see if I can find a reference for that.

Hmm, I thought in your earlier post you said this should be OK? We're only ever attaching 1 drive at a time to the server, it always appears as drive F: (after disconnecting the old and connecting the new). My understanding is this is the simplest possible scenario for a removable B2D folder - if any configuration should work it should be this one. Conceptually the same as removing one tape from a tape drive and putting in a different one.

[David_Deakin]

Hello again

I found this reference to USB drives on the Symantec website:

http://seer.entsupport.symantec.com/docs/276904.htm

It refers to 10d but I presume 11d is the same. As I understand it from the docs, removable backup-to-disk devices are things like zip drives, whereas USB disks are considered no differently from any other hard disk which is perhaps why if you have more than one then it must always be mounted on the same drive letter. I agree with Ken that this is a major limitation.

I only found problems with rotating the disks when I tried restoring to a new installation of Backup Exec on a new server to simulate recovery from a disaster. Using Maxtor One Touch USB drives I had two disks for Friday full backups which were alternated, and one each for differential backups on Monday to Thursday. Each job was set to overwrite the existing media on the disk. I had no problems with the backups, or restoring data using the server that had made the original backups, or so I thought. However when I attempted to restore using the new installation, I would get restore failures with 'inconsistent data', I think possibly due to very large files spanning more than one bkf file. I never got entirely to the bottom of what was going on as I changed my strategy when I found out that swapping USB drives sharing a drive letter was not supported, and that my backups were consequently not reliable.

I then tried a different method where I mounted the USB drives on the local file system (rather than as different drive letters) and created separate backup-to-disk devices for each one. I then setup individual backup jobs, with each backing up to the respective USB disk. Obviously this was quite a bit more complicated than the previous system but seemed to work ok. However the server would often mix up the disks and mount them on the wrong folder, which I would have to correct in Disk Management. I found though that after rebooting the backup server that it would take the backup-to-disk devices offline, except that for some reason and somewhat randomly Windows would allow access to one or two of the folders where the USB disks would be mounted and Backup Exec would then write new changer.cfg and folder.cfg files. This appeared to mess up the system when the USB drives were re-mounted and the backup-to-disk devices brought back online.

So I have now tried yet another method where I keep two of the USB disks permanently mounted on Y and Z drives, and always backup to those. I then use a batch file to simply copy their contents to the remaining disks which I rotate take off site. I haven't been able to test yet whether this works when restoring to a new installation of Backup Exec.

Sorry for the long posting but I hope my findings might help others like me struggling with backup-to-disk folders on USB drives. I had originally hoped that using them would be a good way of having reliable backups (having had first hand experience of problems with tapes) that would be quick to access, but now Im not so sure!

Regards

David Deakin

[David_Deakin]

I originally posted in the 10d forum about my setup and problems, but did not receive any replies. It might put my above posting into a bit more context though:

https://forums.symantec.com/syment/board/message?board.id=103&message.id=55493

I would agree that using removable backup-to-disk folders on USB disks (rather than those with removable media such as zip drives) should conceptually be the same as rotating tapes, but it doesn't seem that Backup Exec works that way.

Regards

David Deakin

[Ken_Putnam]

Looks like Symantec finally wised up (or else someone just dropped the ball - nah, that couldn't have happened   )

http://support.veritas.com/docs/276904  which says

Drives used as backup-to-disk targets can be removed and plugged back in at a later time, but each Firewire or USB drive must be assigned a static drive letter and must have its own backup-to-disk device created within Backup Exec.

was not updated for v11d

[m3ei]

Hi Ken,

Not sure what you meant in your last message:

>Drives used as backup-to-disk targets can be removed and plugged back in at a later time, but each Firewire or USB drive must be assigned a static drive letter and must have its own backup-to-disk device created within Backup Exec.

>was not updated for v11d

 - are you saying that the support doc should be updated to indicate it applies to 11d as well? Or, that it does NOT apply to 11d because something has changed?

Also, does the doc seems to refer to non-removable B2D folders. Would it apply to Removable ones as well?

Thanks.

[m3ei]

Hi Ken,

Ignore my last message. Looking more closely at my recent backup failures it looks like they're being caused by having GRT enabled for Exchange: http://seer.entsupport.symantec.com/docs/288387.htm In other words, a known 11d bug.

It looks as though BE has been recycling media properly (notwithstanding the above), and I was just thrown off by the e-mail failure alerts and the final log error messages ("insufficient disk space" ).

Rather than Symantec's recommended workaround of "backup to tape" (ha-ha) I've disabled GRT for Exchange. This is quite annoying because it's the major reason we bought BE rather than just using the built-in SBS 2003 backup. Hopefully Symantec will fix this problem soon rather than just acknowledging it to be a problem.

Sorry about leading you on something of a wild goose chase, but it's been an interesting learning experience for me, at least.

Message Edited by m3ei on 06-20-200711:19 AM

[Ken_Putnam]

What I was trying to say was the the TechNote says for v8.6, v9x and v10x, you must assign each physical USB or Firewire disk it's own Drive Letter, and it must ALWAYS be mounted on that drive letter.

The TechNote was never updated to say that it applies to v11d as well, so maybe Symantec finally fixed a MAJOR (IMNSHO) design flaw

As for the GRT problems

I thought that build 7170 with HotFixes cured the GRT problems  Your technote says

Products Applied:
 Backup Exec for Windows Servers 11d (11.0) 6235

[JVKAdmin]

Hi,

I recently just configured a removable backup to disk solution. I had some difficulties. I am using BE 9.1 however.

1. Purchased 5 Removable HD's (Sandisk Firelite 80 GB). To my suprise, HD's in the removables: 3 ended up being one drive manufacuturer and 2 another. (same brand and model however)

2. Tried to configure removable drives as separate regular Back to Disk folders (non-removable) each with its own drive letter. This did not work. the 3 similar drives each tried to maintain the same assigned static drive letter that I assigned (M:\ as example). The other two drives with another drive letter I assigned (N:\ as example.

I could not get a separate drive letter for each drive. Note that this server was running windows 2000 Server SP4 with only a single USB 1.1 port available using the same cable and swapping drives. I did use the proper "eject/unplug device" before swapping each drive.

3. I swapped each at first and ran an inventory so that Backup exec would create its file on the drives. It created media folders which I was then able to move into a Backup set which I called DAILY BACKUP.

4. I tested the drive mapping to make sure that it consistently was M:\ for monday to wedesdnay and N:\ for thursday and friday. It was. Strange how windows does not keep track of each drive's serial number or other means to disinguish itself from another drive... but I digress..

5. Set up backup jobs 1 and 2 for m - w and t - f respectively.

6. Backups all ran fine and verified fine.

I have heard users complaining that the backups don't work when trying to restore them ? 

Does anyone have any more information on this as to why it doesn work ? Perhaps users are not ejecting properly using safely remove (under 2003/XP) or eject/unplug (Win2K) ?

Also, has anyone else had drive mapping problems for each drive like i originally did ?

Any comments would be appreciated

[RoninV]

Okay, let me see if I have summarized all the threads regarding using external hard drives (USB, firewire) for backup-to-disk setup in a WIndows Server environment.

All BE versions support the use of tape backup, where the user can simply swap tapes from a tape drive. A number of BE documents indicate that external hard drive swapping IS NOT supported by BE versions earlier than v11. Several BE documents also indicate that v11 does not support swapping as well. I've read where some users have created a 'workaround' for this by creating removable backup-to-disk folders.

Currently, to use multiple external hard drives, in a backup-to-disk configuration, a drive letter must be assigned to that hard drive when it is attached to the server (a path to the hard drive will NOT work). This drive letter must be exclusively used by that particular hard drive. For example, if a user wanted to make nightly backups, using five separate hard drives, in a backup-to-disk configuration, the user must assign each drive its own, exclusive, drive letter (1-F, 2-G, 3-H, 4-I, 5-J). To restore a file from hard drive #3, drive #3 has to be reattached to the server and given the drive letter H.

Some uniformity/clarity would be appreciated. Currently, we use tapes to do incremental backups during the week, and a full backup on the weekend. With the installation of a new server, I would like to move to firewire hard drives, do one full backup the last business day of each month, incremental backups M-F (which are taken offsite the next business day), and one full backup on the weekend (which is taken offsite, along with the nightly incremental backup during the week).

[T__William]

***Has anyone any suggestions on how to use device pools with removable drives?

Has anyone tried using Device Pools to run to removable backup-to-disk devices? We tried this past weekend, and BackupExec decided to insist on trying to write to a drive that wasn't attached to the server at the time. I got it to go to the correct drive by acknowledging the alert several times, until it finally chose the drive that was actually plugged in. I'm guessing Symantec has left out an existence check when using backup-to-disk devices in a device pool? We were hoping to use the pool so that we didn't have to change the target device on the weekly jobs every week, but rather could just physically attach and detach the drives (like what the routine was when using tapes!)

Currently, we are doing exactly what was described in paragraph 3 of the last message, using Lacie firewire drives. So far, it is working fine for us. The server "remembers" the drive letter that was last used for each removable drive, so once it is set up correctly, there is not much to worry with as far as setting up the devices themselves each time.

We also discovered that we needed to change the hardware policy for those drives in order get much better performance - instead of "optimize for quick removal" we choose "optimize for performance" and click the first checkmark, "enable write caching" but NOT the second option to "enable advanced performance" (it makes performance much worse.) We also found that using file sizes of 10GB works MUCH better than either 1GB or 1TB sizes, there may be a more optimum size for performance, but satisfied with what it is doing now. We are getting rates of 2,200 to 2,800 MB/min according the job logs, with a total of backup of ~430GB in 4 1/2 hours and a local copy of that job completing in 2 1/2 hours.

I just thought I would share all that in case anyone else is going through some frustration with the speed of backup-to-disk.

Lastly - I noticed earlier in the thread that someone was reporting that their MS Exchange backup was not working properly, because they had enabled GRT and were using backup-to-disk devices and that was documented as not being supported (I think that's what was in the message.)

I wanted to report that we ARE successfully using GRT on our drives, with the caveat that incremental backups using GRT do not work. So we have allocated space to make a FULL backup of MS Exchange *daily*. It's a high priority for us to use GRT, so we committed ourselves to the cost of having enough space to do that.

[RoninV]

TW,

Currently, I have a client that does incrementals M-Th and a full backup on the weekend. I would like to add an additional full backup, from the prior month, to their backup scheme. For example, based on today, with my new scheme, they would currently have the full backup from October 2007, full backup from this past weekend, and the incremental from Thursday night. Since they will be installing a new server, I think the switch to firewire hard drives is the way to go. I'll definitely will check out those Lacie firewire drives you mentioned.

Are you doing full backups of everything daily, or just with Exchange? It appears that a backup-to-disk configuration requires each drive to have its own exclusive drive letter (forever). From what you wrote about attempting to use drive pools, you have to go in and point BE to the attached drive prior to each backup performance. Ahh tapes!! 

It would seem relativily easy, from a server point of view, to make exclusive a block of drive letters for use in a backup-to-disk configuration. But it appears that this drive letter memory doesn't apply to BE.