cancel
Showing results for 
Search instead for 
Did you mean: 

VCS for Solaris10 Swap used out issue

zhangzw311
Not applicable

Hi, Experts there

I encounterd a problem confused me onsite. Swap useage on the master server is 100%. Although we already have 128g physical  memory and 80g Swap. Asscociated Info as below.

Any suggestion? Thanks very much!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

VCS version

root@HOSTNAME> pkginfo -l VRTSvcs
   PKGINST:  VRTSvcs
      NAME:  Veritas Cluster Server by Symantec
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  5.0
   BASEDIR:  /
    VENDOR:  Symantec Corporation
      DESC:  Veritas Cluster Server by Symantec
    PSTAMP:  Veritas-5.0MP3RP3-11/27/09-09:17:00
  INSTDATE:  Jan 29 2011 12:09
    STATUS:  completely installed
     FILES:      176 installed pathnames
                  26 shared pathnames
                   4 linked files
                  50 directories
                  90 executables
              149372 blocks used (approx)

OS

root@HOSTNAME> uname -a
SunOS HOSTNAME 5.10 Generic_142900-03 sun4u sparc SUNW,SPARC-Enterprise

SWAP INFO

root@ df -lh | grep swap               //////// I add 30g swap
swap                    30G   2.3M    30G     1%    /etc/svc/volatile
swap                    30G    65M    30G     1%    /tmp
swap                    30G    96K    30G     1%    /var/run
swap                    30G     0K    30G     0%    /dev/vx/dmp
swap                    30G     0K    30G     0%    /dev/vx/rdmp
root@ swap -l
swapfile             dev  swaplo blocks   free
/dev/md/dsk/d20     85,20     16 524976 524976
/dev/vx/dsk/ossdg/swap 337,102024     16 22527984 22527984
/dev/vx/dsk/ossdg/swap-2 337,102000     16 67108848 67108848
/dev/vx/dsk/ossdg/swap-3 337,102002     16 41943024 41943024
/dev/vx/dsk/ossdg/swap-4 337,102003     16 62914544 62914544

Engine Log
 

root@ pwd
/var/VRTSvcs/log
root@ tail -50 engine_A.log
2011/02/02 13:28:25 VCS INFO V-16-6-0 (yginmaster) postonline:Executing /ericsson/hacs/scripts/postonline.sh with arg0=yginmaster, arg1=Oss
2011/02/02 13:28:25 VCS INFO V-16-6-0 (yginmaster) postonline.sh:Oss:Nothing done for group Oss
2011/02/02 13:28:25 VCS INFO V-16-6-0 (yginmaster) postonline:Completed execution of /ericsson/hacs/scripts/postonline.sh for group Oss
2011/02/02 13:28:26 VCS INFO V-16-6-15002 (yginmaster) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postonline yginmaster Oss   successfully
2011/03/06 03:08:32 VCS ERROR V-16-2-13027 (yginmaster) Resource(cluster_maint) - monitor procedure did not complete within the expected time.
2011/03/08 14:31:10 VCS ERROR V-16-2-13027 (yginmaster) Resource(vcsweb) - monitor procedure did not complete within the expected time.
2011/03/13 11:01:10 VCS ERROR V-16-2-13027 (yginmaster) Resource(vcsweb) - monitor procedure did not complete within the expected time.
2011/03/15 09:45:31 VCS ERROR V-16-2-13027 (yginmaster) Resource(cluster_maint) - monitor procedure did not complete within the expected time.
2011/03/21 07:10:13 VCS INFO V-16-1-50086 Swap usage on yginmaster is 63%
2011/03/21 09:10:13 VCS INFO V-16-1-50086 Swap usage on yginmaster is 63%
2011/03/21 11:10:13 VCS INFO V-16-1-50086 Swap usage on yginmaster is 63%
2011/03/21 13:10:13 VCS INFO V-16-1-50086 Swap usage on yginmaster is 68%
2011/03/21 15:10:13 VCS INFO V-16-1-50086 Swap usage on yginmaster is 68%
2011/03/21 17:10:13 VCS INFO V-16-1-50086 Swap usage on yginmaster is 68%
2011/03/21 18:36:13 VCS NOTICE V-16-1-50086 Swap usage on yginmaster is 72%
2011/03/21 20:36:13 VCS NOTICE V-16-1-50086 Swap usage on yginmaster is 72%
2011/03/21 22:36:13 VCS NOTICE V-16-1-50086 Swap usage on yginmaster is 72%
2011/03/22 00:36:13 VCS NOTICE V-16-1-50086 Swap usage on yginmaster is 76%
2011/03/22 02:36:13 VCS NOTICE V-16-1-50086 Swap usage on yginmaster is 76%
2011/03/22 04:36:13 VCS NOTICE V-16-1-50086 Swap usage on yginmaster is 79%
2011/03/22 05:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 80%
2011/03/22 07:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 81%
2011/03/22 09:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 82%
2011/03/22 11:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 84%
2011/03/22 13:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 85%
2011/03/22 15:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 87%
2011/03/22 17:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 88%
2011/03/22 19:19:43 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 89%
2011/03/22 21:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 91%
2011/03/22 23:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 92%
2011/03/23 01:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 93%
2011/03/23 03:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 95%
2011/03/23 05:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 96%
2011/03/23 07:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 97%
2011/03/23 09:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 99%
2011/03/23 11:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 100%
2011/03/23 13:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 100%
2011/03/23 15:10:13 VCS WARNING V-16-1-50086 Swap usage on yginmaster is 89%
2011/03/23 17:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 91%
2011/03/23 19:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 92%
2011/03/23 21:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 93%
2011/03/23 23:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 95%
2011/03/24 01:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 96%
2011/03/24 03:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 97%
2011/03/24 05:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 98%
2011/03/24 07:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 99%
2011/03/24 09:10:13 VCS CRITICAL V-16-1-50086 Swap usage on yginmaster is 100%
2011/03/24 09:26:47 VCS WARNING V-16-1-10023 Agent Monitor not sending alive messages since Thu Mar 24 09:24:35 2011
2011/03/24 09:26:47 VCS WARNING V-16-1-53025 Agent Monitor has faulted; ipm connection was lost; restarting the agent
2011/03/24 09:26:47 VCS NOTICE V-16-1-10016 Agent /opt/VRTSvcs/bin/Monitor/MonitorAgent for resource type Monitor successfully started at Thu Mar 24 09:26:47 2011

1 REPLY 1

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hello,

Understand that you have 128G of physical & 80G of swap memory, however did  you got a chance to look at server by unix tools (like top/prstat) to see how really was server doing during that time ?

Did you got a chance to check sar reports or detailed ps ouputs to see usage in VSZ ? DO you have any historic performance data or graphs to inidicate the utilizatoin ?

 

Gaurav