LCK file in-use during post job command on backup to disk folder?

[Kremlar]

One of our typical backup configurations is Backup Exec 2010 running to a Backup To Disk Folder.  Once the job completes, we use Robocopy to copy the contents of the folder to external media for archiving or off-site storage.  This has worked very well for us.

With our latest install of Backup Exec 2010 I'm noticing that our Robocopy logs are showing a LCK file in-use during the copy process (post job command).  This has not happened with any of our previous installs.

If I understand correctly, the LCK file is placed in the Backup To Disk Folder until the job completes, at which point it is removed.

My questions:

 - Has something changed recently with Backup Exec 2010, perhaps an update, that changes the LCK file behavior?  Perhaps causing the LCK file to remain until the post job commands are complete?

 - Can the LCK file be safely skipped during my post job copy?

Thanks in advance!
 

[Colin_Weaver]

Flat file copies of BKF or IMG data is very strongly NOT recommended as you are only moving part of a very large relational database. As such if at a later date you try to bring those files and folders back to the same media server you could introduce inconsistencies between those files and folders and the catalog and inventory data on the server that is from a newer date.

As such we recommend you always use duplicate jobs within Backup Exec to move data to secondary storage and your robocopy solution is not a good idea and may cause problems during restore operations.

Obviously because you are going against our recommended practices Symantec cannot give you an official answer to your actual question about the LCK files.

[Kremlar]

Problem is we run into numerous issues trying to backup or duplicate to USB storage media.  We have spent countless hours on this with Symantec and have found no reliable solution.

We have done testing and have found no issues when restoring from copied Backup To Disk folders in the scenario where we would need to (disaster recovery).  If Backup Exec has no existing catalogs, we are able to catalog and restore these copies without issue.

Am I missing something?

Can you provide any infromation on changes related to LCK files being cleared before or after post job commands?

[Colin_Weaver]

I suspect your tests are not based on what happens as the BKF media in question gets re-used and then you try to restore form older versions of the files so to explain a bit more about why not recommended.

Every piece of media (BKF, IMG, tape) has a unique guid in our database (the inventory)

Every piece of media either contains a complete backup set or could be part of a spanned set involving other pieces of media. Lose any part of a backup set and you won't be able to restore that set.

Every piece of media also belongs to a media family which again it might be a family only containing 1 media or it might be part of a bigger family. Families are a bit more complicated as they don't actually need to be complete however can't be missing a member from the middle of a sequence as this can cause catalog problems.

Now imagine the scenario where you run a job (JOB1) that uses BKF25, BKF26 and BKF27. Then another job (JOB2) runs that uses BKF28, BKF29 and BKF30 and you copy all those BKF files away with Robocopy. With all of these BKF files given unique IDs

Then 6 months down the line those 6 media become available for overwrite and JOB3 uses BKF25, BKF26, BKF27 and BKF28 followed by Job 4 that uses BKF29, BKF30 and BKF31. The overwrite operation of a b2d however creates a new GUID for the media as it does a delete and create operation, instead of a re-use.

Now you need to restore from job 2 so you recover the older version of BKF28, 29 and 30

1) This breaks the backup sets for Job 3 and Job 4 in one go. So obviously you run another copy operation to somewhere new before recovering the older BKF28, 29 and 30 to protect the newer data.

2) You then realize the restore selections show the newer job data only so you attempt to run catalog jobs to correct this and get errors because the guids of the media in the bkf folder don't match the inventory, so you retire and delete the media from Backup Exec and Inventory the media and then catalog it, which should work (although you might have to empty the catalog folder as well)

3) After completing the restore you then realize you want to put back the job 3 and job 4 data sets as being the more recent , but because you have reset the inventory and catalog to do your restore for Job 2, you now have to repeat the process again to get back Job 3 and Job 4s media.

The above is just a very simple scenario, as your server gets busier and the media handling gets more involved it gets much more complicated and you may even have to start with an Empty database in order to inventory and catalog your recovered media. The operation therefore uses more time to achieve and if you have somehow managed to generate a sequence that has broken the media family then worse case scenario might be not being able to restore some of your data, (admitedly this is rare usually it is the amount of work required and time taken to achive the restore that is the main problem.)

As such if you must flat file copy b2d and img data the recommendation is to dump the bedb just prior to your copy and copy the bedb dump file and the catalog files as well. This should at least provide you with a consistent set of data. Although to use it you would still have to protect the current bedb, catalog and bkf files, do the recovery of the older information, do your restore. then put back the newer information. (so still a little time consuning)

The other option is to always have a spare media server handy with an empty database so you can use that server for your recovery in which case it would be just put back the b2d and img data, run an inventory run a catlog. (remembering to put back an empty database when your restore is finished for future operations)

With regards your USB comments I don't know what the problem is but does the following help: http://www.symantec.com/docs/HOWTO55855

[Kremlar]

I do understand your concern and appreciate your thorough explanation.  I want to reiterate again, though, that these flat copies are strictly for extreme disaster recovery scenarios.  Normal restores would never utilize these flat copies.  I do understand it is not recommended practice.

Do you have any information on my actual question though, regarding the LCK files?  I believe this may have started occurring with SP1 for 2010 R3.  From what I can see LCK files are now cleared AFTER the Post Job Command runs rather than before.

Is this change in behavior intentional, or is it a bug of some sort I can expect to be corrected?

I have now tested restores from these flat copies excluding the lock files and they do seem to restore without issue.

Thanks
 

[Colin_Weaver]

We have had issues (even before R3) where customers tried to do utility jobs as either pre or post commands and found they could not as the device remains locked. I suspect you are experiencing a similar problem.

EDIT:

For DR purposes or migrations flat file copies are accpetable as in that situation you would either start with an empty database or be restoring the time matched database as well. You wouidl also be recovering BKF and IMG folders that would be time matched and not from compkletely separated points in the history of the server.

[Kremlar · Accepted solution]

My bad on this - I had not changed the post job command to run after AFTER the verify.  With that set there is no LCK file during the post job command.  The LCK file must clear after the verify.