06-09-2014 02:25 AM
Hi all,
We have a 2003 server that is running SSR2011 that we have had issues with for a while. What we see is that from a fresh server restart or stop and restart of the services, SSR is able to do a base backup and then several days worth of incrementals (3 per day) and then stop working with Unknown errors followed by Out of Memory errors.
The steps I have taken so far involve patching, removing and reinstall (via add/remove) - removing completely with the RemoveSSR batch file (see https://www-secure.symantec.com/connect/forums/trying-re-install-ssr-2011-after-manual-removal-windows-2003-server). Removing all the backup jobs including the hidden files stored on the source disk, using a completly new folder with no backups in it as a destination completely creating from scratch, all to no avail.
To be honest when we have had this before with other systems - a removal and reinstallation (plus updates) normally fixes this. Typically, in this instance, it hasnt worked.
Thanks
Wayne.
Spec of the system.
C: 33.9gb with 15gb free
D: 203gb with 39.6gb free
2gb Ram - Windows 2003 sp2 + Windows Updates.
Error messages in SSR Event Log - this is the first of the failues untill restart
Error EC8F17B7: Cannot create recovery points for job: Drive Backup of System (C:\), Data (D:\).
Error EC8F040B: Cannot create incremental recovery point of D:\ drive.
Error E0BB0083: Unable to create incremental recovery point.
Error EBAB001A: An unknown exception has occurred at .\DiskGroup.cpp 1977. (UMI:V-281-3215-6071)Details:
Source: Symantec System Recovery
Followed by repeating
Error EC8F17B7: Cannot create recovery points for job: Drive Backup of System (C:\), Data (D:\).
Error EC8F03EA: Cannot create a virtual volume image of the selected drive.
Error EBAB00CC: Application unable to continue due to lack of memory. (UMI:V-281-3215-6071)Details:
Source: Symantec System Recovery
Solved! Go to Solution.
06-30-2014 02:57 AM
Hi Chris,
Thanks for the reply, unfortunatly the customer has let lapse the support on this so am having to work with 2011.
My 'fix' at the moment is to have a script file run every day that stop starts the symantec services. So far, 6 days without issue which includes a base weekend backup. Officially I guess not an official fix but the best I've got so far.
Wayne.
06-09-2014 10:45 PM
Hm, maybe some checkdisk-ing?
06-10-2014 09:09 AM
Mmm, would that not also cause a problem when a base backup is ran also rather than it failing after it has managed to do a few days backups?
06-10-2014 09:21 PM
Out of my experice checkdisk has never had such an impact.
06-13-2014 09:06 PM
The fact this is happening after a few successful runs (see logfile contents) tends to imply a problem where the VSS is gradually failing - in a manner where the VSS is eventually unable to create a lock where the system can be backed up during operation.
Things to look at:
1. I am currently investigating an issue with SSR 2013 - where the SymTrackService (the new VSS manager for SSR 2013) crashes at startup - leaving an entry in the Reliability History index. This seems to occur because the SymTrackService deadlocks with other services during the "startup melee" when the OS is initially loading.
2. For SSR 2013, the situation can be mitigated by changing the Symantec System Recovery Service Startup type from "Automatic" (default) to "Automatic - Delayed Startup" - along with changing the SymTrackService Startup type from "Manual" (default) to "Automatic - Delayed Startup" as well.
3. If I remember correctly, the version of the VSS manager used in older versions of Ghost/SSR was called "StorageCraft Shadow Copy provider". It is possible that a similar workaround using this VSS manager may be effective as well.
4. If the above is not helpful, see Technote: http://www.symantec.com/docs/TECH203082 for details on ways to deal with "StorageCraft Shadow Copy provider" issues.
Hope this helps.
06-23-2014 07:54 AM
Hi Twixt,
Thanks for replying.
Still working with the same symptoms, but have checked over the VSS providers as you suggested. There is only the two, MS and Symantec.
Provider name: 'Symantec Software VSS Provider'
Provider type: Software
Provider Id: {262b716e-bb23-41b5-aaef-e2c15e767167}
Version: 1.0Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7
Also had an issue with it not running the scheduled job and would completely miss it - no errors - nothing. My fix to that one though was to edit the job and move the start times on 1 min. Let it re-write the job back was my thought. That seems to have stopped that re-occurring. Don’t know what the problem was with that one.
Still a puzzle
Wayne.
06-25-2014 02:59 AM
Are you able to upgrade to SSR 2013 to see if this helps?
If the problem persists with 2013, I would recommend opening a support case so this can be investigated further.
06-30-2014 02:57 AM
Hi Chris,
Thanks for the reply, unfortunatly the customer has let lapse the support on this so am having to work with 2011.
My 'fix' at the moment is to have a script file run every day that stop starts the symantec services. So far, 6 days without issue which includes a base weekend backup. Officially I guess not an official fix but the best I've got so far.
Wayne.
07-03-2014 07:15 AM
Any updates here?
07-09-2014 03:44 AM
Not from my point - my 'fix' for the min is to restart the services every morning via a batch file. Bit of a fudge, but its the only way I've been able to get things reliable at the min. If anyone has any other ideas...?
Wayne.
07-09-2014 03:55 AM
OK, so please mark your post as solution.
07-30-2014 01:16 AM
For completetion, and no further feedback on this - the site I implemented the batch file on hasnt had any errors in 31 days. Interestingly - it did miss two days activity where nothing had attempted to run. The only entries in the System Recovery event log on those days were for the restart of the services. Still no idea why it should just forget to run.
The site is running 1 base backup on Sunday with 3 incrementals per day for the other 6 days, so in the grand scheme of things, missing 4 backups (it missed the base once) I guess isnt too bad.