Sol 10 + T2000 + Nb 6.5.1 slows down after 2 days

[Peter_Jakobs]

Hi,

I had a pretty stable system before I added some extra clients to my backup system.

I start getting random backup failures as described in http://seer.entsupport.symantec.com/docs/285057.htm

Following the doc I installed the latest Solrais patches.

During the weekend that followed my backups went slower and slower and finally did not respond any more.

Sun support advised me to add set zfs:zfs_arc_max=0x140000000 to the /etc/system file.

During the week the system runok (slower then before but it run), but this weekend was terrible again.

the strange thing is that some streams run fast, but other slow down to 50 KB/sec.

Anyone else implemented the solution mentioned the in veritas doc?

Anybody else if having these kinds of problem with Solaris 10? on a T2000?

I have a lot of ram 32GB, seperate HBA's for disks and tapes. Backing  up on 6 LTO4 drives.

Sun support is also working on it but hoping to hear something...

Peter

[marc_thibodeau]

I might be seeing the same issues as you. I'm running NBU 6.5.1a and I've seen in a 12 hour windows my Write speed in MB/s go from around 12 MB/s to 4MB/s. It's a very small decrease, but over time it's easy to see that it's getting slower. I have a MS server 2003 SP2 and a MLS2024 TLO3. (but I'm running this backup onto disks) SCSI Ultra 320 10K arrays (RAID5).

Plus I started an new backup job. The same backup job yesterday had an average speed of 12-11 MB/s, now it's 4-4 MB/s.

[Peter_Jakobs]

Just got the official response from Sun and I thought to share that...

> the problem has been identified as a bug introduced in kernel patch
> 127111-03.
>
> Customer has applied 127111-06 and is now hitting this bug.
> >This will cause bad network performance at the reader (backup server)
> and the
> >> memory consumption is just the result of this large mblk chain
>
>
> Workaround would be to back-out the kernel patch (use 127111 in rev < 03
> or use 125100-10 which was the previous kernel patch before the
> 127111-xx series).
>
> The bug has been escalated to sustaining to get a fix.

[Stumpr2]

thank you for following up and sharing the resoltion with us.

[Stafford]

Peter - just wondering if there was an official fix to this.  Think I'm running into the same problem with a T5220.

Latest version is -10, I'm looking through the fixes and don't see anything that appears to match up with the described problem.


Message Edited by Stafford on 03-18-2008 07:40 AM

[Peter_Jakobs]

Not yet, bug id is 6666472

http://bugs.opensolaris.org/view_bug.do;jsessionid=6e920dd2519155df1e19f4c02d7?bug_id=6666472

The only workaround is not going further then kernel patch 127111-02.

Peter

[Stafford]

Thanks for the quick response, I'll keep checking on it but if you hear anything please update the thread.  The solution so far has been weekly reboots of the media server which is technically still in pre-prod so it's not a big deal but a PITA nonetheless.