Forum Discussion

RichST71's avatar
RichST71
Level 2
13 years ago

Database Management job brings disk folders offline

I have just upgraded from Backup Exec from 2010 to 2012 and have been finding my backup to disk folders have been going offline for no apparent reason.  On investigation, I am seeing that there are event logs that show the Virtual Disk service restarting at 4AM, which coincides with time the disks go offline.

It looks like the Backup Exec Database Maintenance routine runs at 4AM - I am guessing that this is somehow causing the problem.  I cannot see how to modify the Maintenance routine, or why it would stop the Virtual Disk service.

Has anyone come across this before and if so, is there a resolution to it?

 

Thanks

Richard

2012
Backup and Recovery
Backup Exec
Backup Exec (media server only)
  • Cullen's avatar
    Cullen
    12 years ago

    It turns out it is not the database management job after all. Running the database job at any other time does not cause the drives to go offline, but the drives still go offline at 4AM. It took me about 5 hours to figure this one out. First, some info about my setup:

    2 identical Dell 2950 servers, Server 2008R2 (64-Bit), Backup Exec 2012 SP1a, 2 Identical Dell MD300 arrays, directly attached. The drives go offline on one server, but not the other

    The drives go offline 5 seconds after the database job ends at 4AM (04:00:11). It is usually the same drive that fails.

    The log files in C:\Program Files\Symantec\Backup Exec\Logs that are created when the management job runs or the services restart have a time stamp that matches the failure. The only OFFLINE error message is in the InitVDS log file, so it is the Virtual Disk Service Initialization job that causes the failure.

    The only difference between the two arrays I have is that the "Access" Virtual disk that gives access to the host group was using LUN 0, and not LUN 31 as specified by Dell, and was using the same LUN as one of the virtual disks. The backup servers are not live yet so I removed all of the virtual disks, hosts and host groups from the array and configured it again using the correct LUN for the "Access" disk. The drive no longer goes offline when the services are restarted. I believe the InitVDS job saw the access disk at LUN 0 and tried to initialize it. Now that it is not the first LUN, the job does not cause an error.

    Symantec needs to patch that discovery procedure to deal with the possibility that the access partition will be scanned and not crash if it is.

    I hope this helps you all and that this truly is the answer. It worked for me. I have a Case open with Symantec and I will pass this info on to them and hopefully they will put out a KB article on it.

    Maybe they will send me a check for the time I spent troubleshooting their product. Now they don't have to spend the time doing it! :)

    We will see if it goes offline at 4AM tomorrow morning. Cross your fingers....

    Good Luck!

18 Replies

Sort By
  • CraigV's avatar
    CraigV
    Moderator
    13 years ago

    Hi Richard,

     

    Open up BE and go to Configuration & Settings (hit the BE button within BE) --> Backup Exec Settings.

    Click Database Maintenance --> Perform database maintenance at...

    Change it to a time more suitable to allow your folders to stay online.

    Thanks!

  • RichST71's avatar
    RichST71
    Level 2
    13 years ago

    Thanks Craig

    I will investigate this.  I was to be able to monitor the disks for going offline and briung back online as and when it detects this.  Looking at the powershell cmdlets, I don't see a way to bring storage back online.  Do ytou know if this is possible?

    Ultimately it would be better to find out why it keeps going offline and fix this, but right now I am after a workaround to bring the disks back online.

    Thanks again for your response.

    Regards

    Rich

  • MDCnet's avatar
    MDCnet
    Level 3
    13 years ago

    I am experiencing the exact same issue with iSCSI volumes.    Even disabling the database maintenance makes no difference.   Symantec when will a hotfix be released to resolve this problem?   Note I have applied BE 2012 SP1 already

  • Colin_Weaver's avatar
    Colin_Weaver
    Moderator
    13 years ago

     

    We are aware of an issue where B2D Locations on SAN volumes (particularly where multipath might be involved) go offline during DB maintenance or service restart. I suspect your condition might be related. 

    With regards the B2D issue, for some reason the TECH article for this issue is not yet public and it is currently still early in the investigations. If anyone does contact tech support formally about this issue then please quote internal TECH article TECH189931. Current workaround does appear to be change the DB maintenenace to a time when no jobs are running.

  • MDCnet's avatar
    MDCnet
    Level 3
    13 years ago

    Thanks Colin, if I log a support call will they be able to provide a hotfix yet or is it still being developed?

  • JamesBarber's avatar
    JamesBarber
    13 years ago

    Colin,

    Do you have any kind of timescales as to when a fix for this may be available? Both SP1 and 1a made no difference to this issue for us and when I logged a call through technical support they were not aware of any issues and went through a whole range of things to attempt to fix it blaming backup jobs etc

    We are going to have to revert back to using 2010 as we have been unable to perform backups using 2012 reliably using 2012.

    Thanks

    James

  • U645893's avatar
    U645893
    Level 3
    13 years ago

    I am experiencing this same problem (and many others) with Backup Exec 2012 SP1a. My disk based storage, a Data Domain DD140, goes off-line about once or twice a day. This only started happening after I upgraded the DD140 to the latest OS version, as I was instructed to do by Symantec in an attempt to solve other major problems. When this happens, all I have to do is go set it to on-line, no other measures are required.  However, any jobs directed to this disk will obviously fail unless I intervene.  I also have a Backup Exec 2010 R3 media server, using the same DD140, and it never experiences this issue.  I have even had a management session open to the DD140, and had Backup Exec 2012 open at the very instant that it went off-line, and there was nothing else going on, no database maintenance, no backup jobs running, nothing, and it just mysteriously goes off-line as far as Backup Exec 2012 is concerned, but is functioning properly in ever other way.  I have deleted the Changer.cfg and Folder.cfg files and ran an inventory, no problem.  Then at any random time, it will just go off-line.

    Any suggestions from anyone is most appreciated.

    Thanks,

    Todd

  • HNeumann's avatar
    HNeumann
    Level 4
    13 years ago

    Hello at all,

    plaese open a call with Symantec. My Case was fixed by an orphan fix which was sent from Symantec. After applying the hotfix the error is gone. Refer to my case number. I think Symantec can help now. The case numerb is Case 418-444-078.

    Regards HNeumann

  • U645893's avatar
    U645893
    Level 3
    13 years ago

    I did open a case.  It took me almost an hour to persuade the technician to review Case 418-444-078, as recommended above.  Once he did, he said I had to be transferred to an advance level technician, and I would get a call back.  Later that day, I got a call back, after another long conversation, I was told that my case had to be handled by an advance level technician, and I was right back where I started.  I waited for two days for a call back, nothing.  I called back in this morning, was put on hold for 30 minutes, then the call was disconnected with no call back.  Very frustrating, and this is happening with regularity when dealing with Backup Exec 2012 support.