Constant Failures of Jobs with numerous reasons

[wedmonds]

Hi People,

I am attempting to get regular and consistently succesful backups using Backup Exec 2012 sitting on a 2008r2 server, running Exchange 2007.

I am relatively new to backup exec 2012 and have not really had problems with previous versions of the software so I may be missing critical elements that everyone else assumes and does by default.

Originally, the server was set to tape backup and this regularly failed with inconsistent tape error due to errors on the tape.
This I am happy to put down to a case of age of tape, condtions they were kept in etc etc.

In order to get away from tape we began testing to Backup-to-folders located on another machine on the network.
We split the original job into a number of smaller elements, specifcally an 'Exchange' backup and a daily 'Work' backup.  We then pointed each one of these backups to its own folder on the other machine.

We then created a brand new user account and followed the instructions as advised by Symantec in order to create an admin account.
This was then tested and all works great.

We then tested the backups manually and each one completed succesfully.
Finally we checked the times they took to complete and scheduled them to run automatically each evening ensuring that they did not overlap (although we did not see this being a major problem if they did as our tests showed they still ran fine)

----------

Some evening these backups work, some they do not (in fact most of the time they fail)
If we run the job manually, sometimes they work, sometimes they do not.
Nothing is being changed by us so its proving very difficult to find the exact cause.

Work Backup
This job fails with the message:

Job ended: 19 June 2013 at 00:53:47

Completed status: Failed

Final error: 0xe000fe09 - The directory is invalid.

Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-65033

Backup- E: V-79-57344-65033 - Directory not found. Cannot backup directory \Example\Example1 and its subdirectories.

Note: Click 'Edit Selection List...' to display the View Selection Details tab of the job's selection list. Select the entry with the directory that no longer exists and click Delete to remove it.

Questions

If I uncheck the folder above click submit and then re-check the folder and submit the job appears to work.

Could this be a simple case of the user deleting a folder and Backup Exec no longer being able to find it?
Files and folder are obviously going to be deleted regularly by users so what can i try to stop these failures?

Exchange Backup

This job fails with the message:

Completed status: Failed

Final error: 0xe000ff02 - One or more of the databases for this storage group are currently dismounted.  This backup is incomplete.  Make sure that all databases are mounted, and then resubmit the backup job.

Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-65282

Questions

I want to ensure that I backup all elements of exchange so that I can perform a restore correctly.

At the moment I have selected the location that Exchange is installed into and the information store.

I was also selecting 'System State' and 'Shadow Copy Components'.

All of these elements i was able to backup with success for a period of time until recently when the above message has started coming up.

My testing has revealed that its directly related to 'System State' and 'Shadow Copy Components' but I am not sure how you go about making sure that all databases are mounted???

Sorry to waffle but I think it sets the scene nicely!

Help appreciated by you gurus!

[Jaydeep_S]

Ideally in a file system backup, BE should automaticlly be able to figure out what data is present or missing in the Files and folders. However, do you run any kind of archiving or run some utilities to move the data from a location to another. If the job has already started and a 3rd party archiving software or a batch file to move the data is executed then you might frequently get these errors about files or directlry being invalid.

For Exchange components, you only need to backup the Information store. There is no need to backup the actual location where it is installed and the folders for database and logs.

Along with the System state and Shadow Copy components, you also need to backup the System Volume (C:\) completly.

For the resto of the drive letters, you could backup depending upon the data on them, just ensure that you exclude the Exchange database and logs folder paths.

For the Exchange database being not mounted error, confirm frm the Exchange Management console, if you have any database (created for testing or other reasons) which is not being mounted. This would cause the job error.

[wedmonds]

Hi Jaydeep S,

Thanks for coming back so quickly.

To my knowledge there is no software moving files/folders around but i cannot say for certain so I will go away and check this as it does relate to a finanace area so its possible the staff are doing something extra or quirky in order to ensure they have everything.

The plan was ultimately to have two backups.

One for work which includes all general folders (and I will now add the information store to this).

And

One for recovery in which I wanted to be able to restore  everything that is required to restore Exchange from scratch in the event we loose the server.

When I attempt to backup System state and Shadow Copy components the job fails with the unmounted database error which only suddenly started happening a few days ago.  It was working fine.

I checked to see what databases are mounted and followed these steps and there is only one information store and this was mounted already:

1. Click on Start -> All Programs -> Microsoft Exchange Server 2007 -> Exchange Management Console.
2. Open the branch Server Configuration -> Mailbox.
3. In the center panel click on First Storage Group -> Mailbox Database so it is highlighted. If there is a gray down-arrow superimposed on the icon, you need to mount the mailbox database. 
4. Right-click on Mailbox Database and select Mount Database. This will mount the private mailbox database.

Can I also check that when you say the System Volume do you mean the entire root of (C:)? 

[Jaydeep_S]

By System Volume I meant the entire root of the volume on which OS is installed. So if it is on C:\ then yes Entire C:\

Also, could you confirm if there are any Rollups applied on the Exchange Server recently. If so, was it rebooted after that. Also, try to check the event viewer on the Exchange Server for any errors or warnings corresponding to the time when the job failed stating that the database was not mounted. See if you could find any events related to store mount dismount. Typically, Event IDs 4001, 9518, 9519, and 9546 are Logged when an Exchange 2007 database is dismounted.

Edit: Check these documents as they might help you fix the Exchange errors -

http://www.symantec.com/docs/TECH52124 

http://www.symantec.com/docs/TECH56078

[wedmonds]

OK some feedback on this which is the result of comments you made  So thank you!.

The work backup failing producing errors along the lines of:

Backup- E: V-79-57344-65033 - Directory not found. Cannot backup directory \Example\Example1 and its subdirectories.

This turned out to be down to security permissions on specific folders that the last support company had setup and were nested deep.  I should have checked this sooner but as there was (and still is not) any reason for specific permissions on the folders in question, it did not cross my mind.  Giving the correct symantec backup account permissions to these folders resolved the problems.

It does not fully explain why the backups failed sometimes and not others but I was trying a number of things so its possible I used a different account when the backups did occasionally work.

The Exchange backup is stil causing me some issues though and from what I see the database is mounted but there is a reboot pending which may be causing the issue and I have read that some people have experienced problems with the Shadow Copy Service getting stuck with a reboot being the only real fix.  Could this my cause?

I also tried the registry fix but this did not resolve the issue.