No concurrent jobs after upgrade to 2010

[Seabee27]

I had a banger of a problem with 12.5 after SP2 came out and had to go to Advanced Support.  I cannot locate the solution to this problem.

The hardware:

Quantum Scalar 50
2 HP LTO-3 Tape drives
LSI U320 SCSI controller

I ran the upgrade to 2010 the other night and now I am back to square one on being able to run concurrent jobs.  Nothing has changed on the hardware side.  The tape drives are on the same SCSI chain along with the changer.  I was getting great throughput on 12.5.  My drives and library are running the latest firmware and the SCSI ID's are the same.

So far I have tried HP drivers, Symantec drivers and I have added the BusyRetryCount key for each tape drive set to 250 decimal in the registry.

All these changes and I still cannot get 2 jobs to run at the same time, I receive the general "Hardware Error" in the job log, however no SCSI Event ID's are logged in the Event Viewer.

I appreciate any help I can get on this.

[Ken_Putnam]

Silly question first:


Do you have an LEO serial installed on the media server ?

[ih8usrnames]

Finally somebody else who is having the same problem I am - sorry for my celebration.

I had been running 12.5 with zero issues - all my jobs were going great, then, I upgraded to 2010.

I did my upgrade while I had a tech on the phone observing via Webex - everything went exactly as it should

I did not change anything:
- Same hardware
- Same selections
- Same Jobs
- Same policies

The only thing that change was the installation of 2010, since then it has been a disaster.

All my jobs fail every night.  I spent this weekend running every job one at a time so I would at least have a full backup in place.

I am running 2003 sp2 and backing up to four 8 drive RAID arrays connected to Areca RAID cards.

Hopefully We can get some traction on this because I have had an open ticket for a week as of today.

Syslog had the following:

Date:              5/3/2010

Time:             9:31:35 PM

Source:         Service Control Manager

Type:             Error

Catagory:      None

Event:            7034

User:              N/A

Computer:     NAS1

The Backup Exec Job Engine service terminated unexpectedly.  It has done this 3 time(s).

All my jobs fail with the following error:

The Backup Exec job engine system service is not responding.

[ih8usrnames]

Thought I would add some more information.

I just rebooted the server, error pops up when I log in.:

szAppName : bengine.exe     szAppVer : 13.0.2896.109     

szModName : kernel32.dll     szModVer : 5.2.3790.4480     offset : 0000bef7

All jobs have failed with the following error in BE 2010:
 

e00081d9 - The Backup Exec job engine system service is not responding.

[Seabee27]

No changes to my situation.  Still working with Symantec Advanced Support.  I have tried upgrading my LSI driver for VMWare and checked my hardware for any errors.

No one has an answer yet.  I have used HP drivers, Symantec drivers both old and new and have created the BusyRetryCount registry key.

I have gone back to VMWare and Symantec both for assistance.  I'll post an update when I have it.

[Seabee27]

Yes.  LEO license is installed and verified.

[teiva-boy]

Have you virtualized the BackupExec Media server?  You mentioned that you have " tried upgrading my LSI driver for VMWare."

[Seabee27]

I did virtualize the Backup Exec media server.  I have an Adaptec 29320ALP card if I'm not mistaken in the physical server.  The virtualization part in all reality, doesn't really matter.  I've run the Microsoft LSI driver and it ran perfect before.  I was doing full backups with verify of 300GB in just under 3 hours.  I vaguely remember using the HP drivers from 2007.  I found them on the net, and from what I remember they did work.

My whole configuration was running smoothly before the upgrade to 2010.  here is an update as to where I am at:

1.  Concurrent jobs are failing only on HP2.  Which is Target 2 in my SCSI chain.

2.  Failed concurrent jobs on HP2  are across all data types; i.e. doesn't matter if I am backing up Exchange or my File Server, either way they fail.

3.  After running putty on the VMWare host, I have found that Target one negotiates its connection speed at 160 mb/s and 80Mhz.  However Target 2 (HP2), auto negotiates at 320 mb/s and 160Mhz.  really doesn't make sense that the drive that is auto netgoiating at a higher speed is the one that is failing. 

4.  Backup Exec is reporting 17 hard write errors on HP2.

At this point my only options are the following:

1.  Run one job at a time.

2.  Have HP2 replaced.  Strangely coincidental that I would have a tape drive problem at the same time.

3.  Force VMWare to hard set the SCSI speed instead of auto negotiating.  Maybe try downing it to 160mb/s and 80Mhz to see if that resolves the issue.

As of right now I am no where better than I was before.  This issue is going on about 2 weeks now. 

[teiva-boy]

You may have more luck also trying the VMware community forums.  I say this as Symantec does not support virtualizing the BackupExec server at all.  You are on your own from a support standpoint.

I ran into one company in AZ that had does turn key networks, selling an HP DL380 series server, with ESXi on a flash card.  They have a BE Media server, a file server, Exchange, and an AD server.  Well they had 4 to deploy to new customers, and while all identical and purchased at the same time, 1 refused to work at all  with backupexec not seeing the tape drive for no rhyme or reason.  It was returned and replaced with a G6 from a G5 or something like that.  
They shrugged it off, but it goes to show that sometimes there is no reason to the insanity of quirky weirdness when you virtualize SCSI cards and the like in a media server.

[Seabee27]

I've gone through the VMWare support world.  I've had Quantum, VMWare and Symantec on the phone at the same time.  What is interesting is that when I was on 12.5 and went to Advanced Support, they resolved my issue.

The SCSI card is a separate purchase that was agreed up on by Symantec, Quantum and VMWare and it worked for well over a year, until I upgraded to 2010.

Quantum has sent me a replacement drive to test, because drive 2 (HP2) is the drive that continues to fail it's backups.  We'll see what happens from there.

I am pretty suprised that virtualizing a media server is such a big problem.  In my mind it makes perfect sense, and my throughput has been awesome.

[Seabee27]

I tried the new tape drive and now the error on HP 1.  After looking further I am seeing this error on each object that is backing up:

e0008445 - The media operation was terminated by the user

This is leading me to believe that I should try reseating HP1 and then running the backups again and see what happens.  If successful then i will try upgrading to the latest Symantec drivers.

If not successful then I may have to replace HP1 as well. 

The difference I am finding in the drives is that the one I recieved is revision A12 vs. the A11 that i have in the library now.