Backup Exec 2014 mistakenly backs up E: drive when asked to backup C: drive

[today]

This is a very strange one.  I have Backup Exec 2014 running on Server 2012 as the media server.  This media server also has 7 Hyper-V VMs on it that need backing up.

I made a job to backup the entire media server, including the VMs.  I included everything.  I can see from the logs that when the job says it is backing up the C: drive that it is actually backing up the E: drive.  Later, when it says it is backing up the E: drive, it is indeed backing up the E: drive... again.  Then when I go to do a file restore on the C: drive, the browsing contents for C: are actaully the contents for the E: drive.

To do a better job of debugging, I made another job, and this time I told it to only backup the C: drive.  Again, I got a backup of the E: drive, but Backup Exec is calling it the C: drive.

Here is a snapshot of the restore pane that proves that the contents showing up under C: drive are not typical C: drive contents, but the E: drive contents instead.

I have installed Hotfix 217591

I know this is a strange one, but any hints are appreciated.

[lmosla]

Hi,

perform a native Windows 2012 server backup.  does it backup the drives correctly?

[today]

The built-in windows server backup program has no problem backing up the C: drive
 

[today]

Here's the very strange thing about this.  I had installed BE 2014 without the Hotfix and it backed up the C: drive just fine.  The problem is that I couldn't get incrementals to work with the hyper-v clients.  So I completely removed BE 2014, remove all the left-behind directories, then completely reinstalled BE 2014 including the Hotfix.  The incrementals on the hyper-v clients now appear to be working.  However, I now have the problem described above.  Ugh.

[today]

More info: System State backups also fail because it looks at the E: drive instead of the C: drive.

I have dumped the volume lables of each logical drive.  I've looked through the metadata that Backup Exec stores in various files.  The logical drives all match the volume labels, wherever I can find both mentioned.  I've even gone so far as to browse the SQL database and use the ResourceIDs to track the usage of that hosts C: and E: drives in various tables.  I cannot find a mistaken crosslink anywhere.

Remember, I had this working without the hotfix.  Then reinstalled the software and added the hotfix, and this problem appeared.

This is possibly a serious issue in the software.  I would appreciate it if you are reading this and have any connections if you could forward it to the software team.

[Artegic]

Open a support case so that a Symantec support person can have a look at your system. It's very time consuming, but if it helps correcting a serious problem like that you would be doing everyone a service.

[today]

I would love to open a support case, but I cannot, because I am only running the Trial version of BE 2014.

My problem is I can't buy this software if it doesn't do the job, but I can't file a bug report about it not doing the job unless I buy it.

That's why I'm begging any Symantec employee who might be reading this to assist me in kicking this case a bit further up the ladder.  I need help making this work or else I can't spend $4k on licenses - I'll have to move on to some other solution.

I've been struggling with all sorts of problems the past three weeks since BE 2014 first came out, and I still don't have it working.  I've uninstalled and reinstalled it, and hotfixed it.  I don't think my setup is very complicated to back up, and yet, this software just won't work for me.

[Artegic]

In that case you should start evaluating alternatives. I don't think you'll get a response from Symantec quick enough to finish your evaluation of Backup Exec 2014 successfully. Alternatively, if you have a good reason why you want to buy Backup Exec, try getting support from the distribution partner from whom you'll eventually buy the product.

[lmosla]

Hi today,

I will dm you for contact information.

[Laurie_Downey]

Today,

I have created a support case for you at the request of Lmosla. I will private message you the case details. Please let me know if you need anything further and I am happy to help.

[today · Accepted solution]

So I got a free support ticket, which I appreciate.  However, despite the best efforts of the person trying to assist me, we couldn't come up with a solution.  It appeared that doing a repair install solved the problem, but after a reboot, the problem was back.

However, I appear to have solved the problem another way, and I might have found a bug in Backup Exec 2014.  The story is a little long, though.

So the C: drive on my Server 2012 Hyper-V Host Server that I'm also using as a BE 2014 Media Server is only a 60GB partition.  That's not nearly enough space to house a default install of BE 2014, because BE 2014 violates Windows programming specs and WRITES APPLICATION DATA INTO THE PROGRAM FILES DIRECTORY BY DEFAULT!  Argh, that is so damn evil.  And it is bad for me because my C: drive is only 60GB.  Well, if you read this post, you know I've already had problems with using the BEUtility and registry editing to move the various data folders around due to hard-coded paths in BE 2014.  So this time, after fully uninstalling BE 2014, I pre-made the "C:\Program Files\Symantec\Backup Exec" directory.  Inside this directory, I placed JUNCTIONS for the Data, Catalogs, and Logs directories that lead to Data, Catlogs, and Logs directory on my E: drive.  I then installed BE 2014.

This appeared to work.  All the data normally stored in the Data, Catalogs, and Logs directory were showing up on the E: drive.  And for the most part, everything backed up just fine.  However, when it came time to backup the C: drive, it would grab all the files and directories from the E: drive and store them on tape as the C: drive contents, as in the main post above.

I believe the problem was my use of the JUNCTIONS somehow confused BE 2014.  It shouldn't - what I did with these JUNCTIONS is perfectly legal.  I've used JUNCTIONS many times in the past to get around drive space problems and never before have I had a single issue.

My solution was to uninstall BE 2014, move around a bunch of resources so I could expand the C: drive partition from 60GB to 1TB, then reinstall BE 2014 taking all of the path defaults for data storage and not using any JUNCTION points.  Such an outrageously large (for a server) system drive insures that Backup Exec will not cause the main operating system drive to run out of space.

My advice to you is to make sure you install BE 2014 onto a very large drive and do not change any of its paths for storing data, either by using the BEUtility, or registry editing, or by JUNCTION points.  BE 2014 naturally has not been tested under these other conditions, so best to just use everything at default to avoid strange bugs.

[Colin_Weaver]

You can install BE program and the SQL database part of BE to another drive. I more or less do this every time I install. You can also customize the catalog and any debug and job log locations later if needed.

I do not recomend using Junction points in the way you are describing as this will cause strange VSS behaviour and may run into other Operating system issues - at one point Microsoft had a bug on server restarts (with one of their old operating systems) where mount points did not get accessed properly or in a timely manner after a reboot and we had someone put the catalogs folder in such a mount. They used to log a support case after almost every reboot until their helpdesk understood that the cause was outside of Veritas's (Symantec's) control.

[today]

Customizing the catalog and debug job log locations is not enough.  There is now a huge thumbprint database generated for GRT of virtual machines.  On my relatively small deployment, this database grew to 20GB.  BEUtility does not seem to allow you to move that independently.  I tried to move it by editing BE 2014's registry entry for Data Path, but even that was not enough, as BE 2014 ignored the Data Path registry entry when writing temporary thumbprint databases for Hyper-V.  This is a bug.

Due to this bug, the only way to safely solve the space problem without expanding the C: partition is to install the full installation to another drive.  But that is inconvenient for other reasons.  I am using the built-in Windows Server Backup to build recovery images which are stored to disk instead of tape, just to make sure I can bring my media server with minimal fuss in a disaster.  However, a "bare metal restore" image requires that all drives with Service executables be fully backed up in the image.  So this will drag in the ENTIRE drive that you installed BE 2014 software on, which could result in a massive restore image if that drive is used for more than just BE 2014.

In general, BE 2014 should not be storing data in the same directory where the program files reside.  Not only is it against the rules for modern Windows programming, but by splitting the program from the data, it allows more flexibility for the end user.

(BTW, why are job logs stored by default in the Data subdirectory instead of Data\Jobs or simply Jobs?  It seems very sloppy to me to store these in the same folder where the main database and all the certificates are stored.)

I ultimately went back to the brute force of expanding my C: drive to a ridiculous size because I thought to myself, what is the configuration which is likely to be the most tested by Symatec and the beta sites?  Well, the full default install, of course, with defaults for all paths.  So even though moving some directories with the standard BEUtility tool is supported, it was likely less tested, therefore, I didn't even do that.  I just got tired of running into all of these strange bugs.

Junction points (even those that point to different drives) should not present any problem for VSS.  I use them all the time for various reasons and have never before had a problem with any other backup software, or with VSS volume snapping.  Mount points are completely different from Junction points.

Junction points should not be a problem for Backup Exec.  I still think what I've experienced indicates a deep bug in the software.  I read through the admin manual about junction points, and it seems that BE 2014 should not care about them if you leave unchecked the options to follow them during backup.