Solaris 10 tuning

[Stumpr2]

In Solaris 10 the facilities are either automatically configured or can be controlled by resource controls. Facilities that can be shared are memory, message queues, and semaphores.

I checked the Backup Planning and Performance Tuning
Guide

this is where it suggests researching
For further assistance with Solaris parameters, refer to the Solaris Tunable Parameters Reference Manual, available at:
http://docs.sun.com/app/docs/doc/819-2724?q=Solaris+Tunable+Parameters
The following sections of the Solaris Tunable Parameters Reference Manual may be of
particular interest:
◆ What?s New in Solaris System Tuning in the Solaris 10 Release?
◆ System V Message Queues
◆ System V Semaphores
◆ System V Shared Memory

I am hoping someone who has already explored the above links could give some pointers for how they configured NetBackup master and media servers.Message was edited by:
Bob Stump

[Gale_De_Los_San]

I've referred to this document on occasion since we've upgraded to Solaris 10 and Sun T-2000 servers. It's been a tremendous help. We're continuing to improve NetBackup 6.0's performance with a few tweaks here and there.

Currently we're in the midst of trying to find what's eating up our RAM and using up CPU cycles. After just a week of heavy NetBackup use ... swapping occurs (less than 1 GB of 16 GB free RAM... and 14 GB of 20GB swap space get's used.

Even if all NetBackup daemons are halted... Only 4 GB of RAM get's free'd up. So around 11GB is being held by some unknown process(es).

I'll definitely post up what our findings are once we find the culprits.

[Stumpr2]

Gale,
looking forward to your post as well as others that may contribute to this post.Bob

[Dennis_Strom]

I have not looked at this in quite a while but I went mostly with the defaults that were used with Sol 9. I did some tweaking and I suppose I should look at it more. I have had other fires. Here is what I have on my Sol 10 box. I am getting good throughput. I can probably do better.
* Veritas

* Message queues
set msgsys:msginfo_msgmax=8192
set msgsys:msginfo_msgmnb=65536
set msgsys:msginfo_msgmni=1024
set msgsys:msginfo_msgtql=1024

* Semaphores
set semsys:seminfo_semmni=2048
set semsys:seminfo_semmns=2048
set semsys:seminfo_semmnu=1024
set semsys:seminfo_semmsl=300
set semsys:seminfo_semopm=128
set semsys:seminfo_semume=64

* Shared memory
set shmsys:shminfo_shmmax=33554432
set shmsys:shminfo_shmmni=512

# VERITAS recommends adding the following forceload statements to the
# /etc/system file. These statements prevent the st and sg drivers from
# being unloaded from memory.

forceload: drv/st

forceload: drv/sg

[Gale_De_Los_San]

Here's the lowdown on the memory issue / system slowdown we were seeing on our Solaris 10 master server:

"The latest update is that we have hopefully tracked the memory leak happening on our master server to an issue with VXFS. VXFS is the filesystem that the netbackup image catalog resides on. Veritas has confirmed that there is a possible memory leak when a VXFS filesystem on Solaris 10 is put under heavy operation. Veritas has provided a rolling patch for this issue."

This patch was applied last Sunday .. and so far with as many as 56 streams (standard backups via NFS to NetApp filers) .. and a few test NDMP policies running (which shouldn't impact the master server at all) .. Memory has dipped only to 10 GB free (out of 16GB)

[Dennis_Strom]

I suppose the reason I have not seen this is we do not use VXFS. Strickly Disksuite.

[Stumpr2]

Here's the latest info from symantec I got from a call I made complaining about slow GUI

The symptoms on Solaris 10 Platform
-- Intermittent high cpu usage
-- Intermittent performance issues
-- Memory leaks / memory resource outage
-- Socket connect errors, socket resource outage


Recommended Action
1) Disable SUN tcp_fusion code until SUN has a fix:
echo "set ip:do_tcp_fusion = 0x0" >> /etc/system
Then reboot so this can take effect.


2) Update kernel patch to 118833-24 for memory management, socket management fixes
See http://patches.sun.com/clusters/10_Recommended.README

seems like memory issues like Gale De Los Santos was talking about earlier. Any update Gale?BS

[Dennis_Strom]

Bob, is you Sol 10 box a Media Server or Master Server? My Sol 10 box just sits there and spins away. Not a hickup yet. But then I am not stressing it real hard either.

[Stumpr2]

double duty as master/media. I know that is not acceptable solution but the State of Michigan is really having financial pains. I did just get a media server added and I am in the process of offloading clients from the master to it. It'll take a while. Actually there are not that many backups that run during the day when the GUI is most often used.

This memory leakage thing has my interest.

[Dennis_Strom]

I brought it up in our meeting since we are strickly a Sun/Solaris shop, well my group is anyway. My master sever is also a media server (Spectralogic 64k) but it is running Sol9. I have not had any problems with this setup. I am a bit surprised I do not seem to be having problems with my Sol10 media server. But really my media server is way over powered. This was a box that was supposed to be used for some other stuff that I started using to "test things out" :) Possesion is 9/10's of the law, right?