Backup Exec 10d extremely slow to Dell 124T LTO3 Library(NTBackup is fine)

[Grant_Coffey_2]

Dear Forum members,

I have a situation at one customer that I believe to be a Backup Exec issue and am having trouble resolving it.

6 months ago my customer bought a Dell 124T autoloader based on an IBM LTO3 drive and two 8-tape carts. At first we were getting the peak performance (averaging over 800MB/MIN ) from the combination of the 124T LTO3 and Backup Exec 10d for SBS. A few months ago performance dropped dramatically to about 220MB/MIN; akthough strange at the time, this was easily solved by uninstalling and reinstalling Backup Exec.

Recently the slow performance has returned, only this time we cannot correct it by reinstalling BEX. We have spent the past three weekends trying various combinations to no avail. Today after 12 hours and dozens of reboots I have decided to post our details to the forums to see if anyone has solved this in the past.

Server is a Dell 2950, Dual Xeons with 4GB RAM and RAID5 SATA. Disks and CPUS run about 10-15%, with 1.5GB physical RAM usually on tap. OS is SBS 2003.

Heres the tests I have performed:

1: BEX to the 124T library runs about 6 to 13GB/hr (120 to 220MB/min) - this is at least 4 or 5 times slower than it should be running
2: BEX to a local B2D runs about 90GB/hr (the disk R/W throughput ceiling)
3: BEX to a network B2D runs to max LAN speed, slightly quicker than #2 above
3: NTBACKUP to the 124T runs about 46gB/hr (780MB/min) - near the 124T speed ceiling
4: We have installed both BEX 10.0 and 10.1 and have used combinations of latest 10.1 and 10.0 Symantec tape/loader drivers and the OEM drivers, still with slow throughput under BEX
5: We have the following settings for the LTO3 drive under BEX Drive Properties dialog: BLOCK:64k; BUFFER: 64K (tried 128K too); BUFFER CT: 10: HIGH WATER:0 and all 4 checkboxes for "Block Mode/PassThrough" are OFF as per forum post guidelines elsewhere.

I think our tests have proved that is an issue somewhere with Backup Exec writing to this SCSI LTO3 drive -- what I cannot work out is why it has suddenly started when all was working well for a few months. And why does NTBACKUP still work perfectly to the same tape device?

I believe we have proved that the machine, disks and tape system incl scsi card are capable of full speed by the NTBACKUP being at full speed on the same data sets that BEX is going so slowly on.

If any Symantec specialists have any ideas as to what settings we may have misconfigured we would appreciate any help.

Regards

Grant Coffey

[Philip_Pearson]

There are a couple of things I would check, are you fully patched with the latest drivers? Dis the slowdown coincide with an upgrade?

Do you have the latest firmwares on your drive and SCSI card?

[Grant_Coffey_2]

Hi Philip

The machine in question is fully patched Windows-wise via our WSUS server, and I have tried BEX 10.0 bare, 10.0 w/SP3, and 10.1 bare, 10.1 w/SP1, 10.1 w/SP2 and 10.1 w/SP3. In each iteration of my 10.1 tests I used both native/OEM drivers *and* the latest Veritas drivers. As the 124T library is not detected by the 10.0 bare release I could only test under 10.0 using the latest 10.0 device driver kit.

The drive and the library have the latest firmware available loaded. We have already proved that the 124T+Drive can do full speed backups, but in any case we had the autoloader replaced by Dell two weeks ago 'just in case' - they were very obliging doing this for us, replacing the loader and cables and SCSI card, but alas it made no difference.

I am not sure how to update the firmware on the SCSI card (Adaptec 39320A) although as it operates fine with NTBACKUP and once worked fine with BEX, and has not been reflashed/down-flashed I don't think this will be the issue. To further rule out the SCSI card we did replace the 39320 card with another one (same model/firmware revision), and tried both ports on both cards with same results. We also had the cables replaced and the terminator.

The slowdown did not coincide with any software patches that we can account for on the day - we checked our logs and the slow-down started a couple of days either side of a major WSUS approval run.

Besides WSUS the only other automated updates are the Symantec SAV Corp updates; although I have tried running the backups without SAV loaded and no dirfferences are noticable.

Grant
Brisbane Q AU

[Philip_Pearson]

Dare I suggest you try a trial version of 11

[Grant]

I was able to install Backup Exec 11d build 7170 this weekend; unfomrtunately it's made no difference - still only getting about 20MBytes/minute

The last month or so I have been using the NTBACKUP program and it's been working nice and fast however it's a real dog to manage. I like BEX but it's just too slow to be usable.

Grant

[Jeremy_Budd]

I have a Dell 124t LTO3 library as well and here is what performance I am getting.

When I use Bex 11D 7170 I backup at about 500MB/Min on the local server, then the funny thing happens. When I backup over the network I backup at over 1000MB/min sometimes up to 2000MB/min. This is on a cisco gigabit switch. I am wondering why the local server backups slower than the network server. Does the network server compress the files before they get to the server the backup drive is connected?

I use the latest symantec drivers for Bex and the latest adaptec driver for the standalone scsi 320 card. All at the latest firmwares.

[Grant]

I installed 11d - same problem.

I then changed the SCSI ID of the 124T to 4 instead of 3 and it went back to full speed again -- ~1400GB/hr

I can't fathom why a SCSI ID change from 3 to 4 would have any effect, but it did.  Pretty flaky for what you pay for it this Backup Exec.

Now I have to spend another few hours next weekend uninstalling 11d and reinstalling the customers licenced v10 ....

Interested to see if this tip helps anyone else whos having speed issues to the PowerVault LTO3 drives on Dell POwerEdge machines with Adaptec U320 cards,

Grant,

Brisbane AU

[Rich_Lichvar]

Interesting, a SCSI ID change simply made that big a difference? Mmmmm. Our's is sitting at 6. I'm wondering ......

[mag1guru]

We had similar issues after buying a new file server solution

Dell 2950 Server

Dell MD1000 Array

Dell PowerVault 124T LTO3

Symantec Backup Exec. 11d 6235 (Now upgraded to 7170)

Symantec Systems Recovery Server

All current drivers and firmware

Backups would start out fast and quickly slow down to around 4MB min.

I opened a case with Symantec but recieved no resolution.

I found that by disabling the Write SCSI pass-through mode option and using Dells driver for the tape drive my speeds stayed consistant @ around 1600 MB Min.

Hope this may help others

[JoesTech]

I have the exact same problem!  v 10.1 BEX, Dell 124T LTO3 autoloader, runs at 4MB/sec while backing up its own HDD!!  It is a newly purchased autoloader and I've yet to get it to work at normal speed.

Changing SCSI ID doesn't work for me.  Has Symantec released a fix for this bug yet?

[Ken_Putnam]

From the Devices Tab, right click the drive\properties

What are the settings for Block Sizer, Buffer Size, number of Buffers?

Also both Single Block settings and both writie through settings?

Message Edited by Ken Putnam on 07-12-200711:35 AM

[JoesTech]

Block Size: 64K
Buffer Size: 64K
Buffer Count: 10
High Water Count: 0

Only "Write" checkboxes.  "Read" are both unchecked.

[Ken_Putnam]

Deselect the Write Single Block setting. This totally bypasses buffering  (read one block from disk write one block to tape, read one block etc.) and should almost never be used unless directed by Symantec Support

The Write Through options should only be selected if the device is NOT locally attached, but on a SAN, but should not affect throughput, but may keep errors from appearing in the event logs

Your block and buffer settings should work OK.  You may want to try setting the buffer to 128 or 256 if you have plenty of RAM in the media server., but probably won't make a large difference.

[JoesTech]

Sorry for the delayed reply.  Was on vacation.  I tried the change you recommended and that actually made it slower.  I fiddled around with the settings in question and tried a variety of combinations.  The speed varied, but never got more than 74MB/min.  Way slower than what LTO3 is capable of.

I don't know what else to try other than call Symantec Support and pay out the nose for them to fix their own bug.

[Jesus_I_Belong_]

Hello, I am experiencing the same issue.
We are using Dell PowerVault LTO-060 and Backup Exec 11d here.

Does the issue solved?