VCS 6.0.1 on Solaris x64 apache resource not coming up

Question

&nbsp;
2013/11/12 15:31:44 VCS INFO V-16-1-10298 Resource res_mysql (Owner: Unspecified, Group: testgrp) is online on solaris10-1 (VCS initiated)
2013/11/12 15:31:44 VCS NOTICE V-16-1-10301 Initiating Online of Resource res_apache (Owner: Unspecified, Group: testgrp) on System solaris10-1
2013/11/12 15:32:36 VCS NOTICE V-16-10061-20284 (solaris10-1) Apache:res_apache:online:KillPIDandAllChildren:Proc:[SIGTERM] delivered to:
PID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CMD
----------------
[12992] &nbsp; &nbsp; &nbsp;[/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout]
----------------
Total: [1]
2013/11/12 15:32:59 VCS NOTICE V-16-10061-20284 (solaris10-1) Apache:res_apache:online:KillPIDandAllChildren:Proc:[SIGKILL] delivered to:
PID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CMD
----------------
[12992] &nbsp; &nbsp; &nbsp;[/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout]
----------------
Total: [1]
2013/11/12 15:33:20 VCS ERROR V-16-10061-20460 (solaris10-1) Apache:res_apache:online:Sys:RunWithEnvCmdWithOutputWithTimeOut:The command [/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout] did not complete within [10] seconds
2013/11/12 15:33:41 VCS ERROR V-16-10061-20376 (solaris10-1) Apache:res_apache:online:&lt;Apache::GetMonitorTimeout&gt; The command line [/opt/VRTSvcs/bin/hatype-value Apache MonitorTimeout] did not complete within the allotted
&nbsp; &nbsp; &nbsp; &nbsp;amount of time ( [10] seconds )
2013/11/12 15:34:02 VCS ERROR V-16-10061-20312 (solaris10-1) Apache:res_apache:online:&lt;Apache::ArgsValid&gt; SecondLevelMonitorTimeOut must be less than MonitorTimeOut.
2013/11/12 15:34:09 VCS ERROR V-16-2-13027 (solaris10-1) Resource(res_apache) - monitor procedure did not complete within the expected time.
2013/11/12 15:34:35 VCS NOTICE V-16-10061-20284 (solaris10-1) Apache:res_apache:monitor:KillPIDandAllChildren:Proc:[SIGTERM] delivered to:
PID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CMD
----------------
[13080] &nbsp; &nbsp; &nbsp;[/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout]
----------------
Total: [1]
2013/11/12 15:34:58 VCS NOTICE V-16-10061-20284 (solaris10-1) Apache:res_apache:monitor:KillPIDandAllChildren:Proc:[SIGKILL] delivered to:
PID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CMD
----------------
[13080] &nbsp; &nbsp; &nbsp;[/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout]
----------------
Total: [1]
2013/11/12 15:35:29 VCS ERROR V-16-10061-20460 (solaris10-1) Apache:res_apache:monitor:Sys:RunWithEnvCmdWithOutputWithTimeOut:The command [/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout] did not complete within [10] seconds
2013/11/12 15:36:01 VCS NOTICE V-16-10061-20284 (solaris10-1) Apache:res_apache:monitor:KillPIDandAllChildren:Proc:[SIGTERM] delivered to:
PID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CMD
----------------
[13140] &nbsp; &nbsp; &nbsp;[/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout]
----------------
Total: [1]
2013/11/12 15:36:07 VCS ERROR V-16-2-13066 (solaris10-1) Agent is calling clean for resource(res_apache) because the resource is not up even after online completed.
2013/11/12 15:36:11 VCS ERROR V-16-10061-20376 (solaris10-1) Apache:res_apache:monitor:&lt;Apache::GetMonitorTimeout&gt; The command line [/opt/VRTSvcs/bin/hatype-value Apache MonitorTimeout] did not complete within the allotted
&nbsp; &nbsp; &nbsp; &nbsp;amount of time ( [10] seconds )
2013/11/12 15:36:53 VCS NOTICE V-16-10061-20284 (solaris10-1) Apache:res_apache:monitor:KillPIDandAllChildren:Proc:[SIGKILL] delivered to:
PID &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;CMD
----------------
[13140] &nbsp; &nbsp; &nbsp;[/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout]
----------------
Total: [1]
&nbsp;
&nbsp;
&nbsp;
&nbsp;

bash-3.2# cat /etc/VRTSvcs/conf/config/main.cf
	include "OracleASMTypes.cf"
	include "types.cf"
	include "Db2udbTypes.cf"
	include "MySQLTypes51.cf"
	include "OracleTypes.cf"
	include "SybaseTypes.cf"
	&nbsp;
	cluster ourcluster (
	&nbsp; &nbsp; &nbsp; &nbsp; UserNames = { admin = aPQiPKpMQlQQoYQkPN }
	&nbsp; &nbsp; &nbsp; &nbsp; ClusterAddress = "10.0.0.43"
	&nbsp; &nbsp; &nbsp; &nbsp; Administrators = { admin }
	&nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	system solaris10-1 (
	&nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	system solaris10-2 (
	&nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	group ClusterService (
	&nbsp; &nbsp; &nbsp; &nbsp; SystemList = { solaris10-1 = 0, solaris10-2 = 1 }
	&nbsp; &nbsp; &nbsp; &nbsp; AutoStartList = { solaris10-1, solaris10-2 }
	&nbsp; &nbsp; &nbsp; &nbsp; OnlineRetryLimit = 3
	&nbsp; &nbsp; &nbsp; &nbsp; OnlineRetryInterval = 120
	&nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; IP webip (
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Device = e1000g0
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Address = "10.0.0.43"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; NetMask = "255.255.255.0"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; NIC csgnic (
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Device = e1000g0
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; webip requires csgnic
	&nbsp;
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; // resource dependency tree
	&nbsp; &nbsp; &nbsp; &nbsp; //
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;group ClusterService
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;{
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;IP webip
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;{
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;NIC csgnic
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;}
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;}
	&nbsp;
	&nbsp;
	group testgrp (
	&nbsp; &nbsp; &nbsp; &nbsp; SystemList = { solaris10-1 = 0, solaris10-2 = 1 }
	&nbsp; &nbsp; &nbsp; &nbsp; ContainerInfo = { Name = testzone, Type = Zone, Enabled = 1 }
	&nbsp; &nbsp; &nbsp; &nbsp; AutoStartList = { solaris10-1, solaris10-2 }
	&nbsp; &nbsp; &nbsp; &nbsp; Administrators = { admin }
	&nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; Apache res_apache (
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ResLogLevel = TRACE
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; httpdDir = "/apps2/apache2/bin"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; PidFile = "/apps2/apache2/logs/httpd.pid"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; HostName = testzone
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; User = root
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ConfigFile = "/apps2/apache2/conf/httpd.conf"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; MySQL res_mysql (
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Critical = 0
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ResLogLevel = TRACE
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; MySQLAdminPasswd = mypwd
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; BaseDir = "/opt/mysql/mysql"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; DataDir = "/opt/mysql/mysql/data"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; MyCnf = "/opt/mysql/mysql/my.cnf"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; HostName = testzone
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; Zone vcszone (
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; Zpool vcspool (
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; PoolName = spool
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; AltRootPath = "/"
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ZoneResName = vcszone
	&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; res_apache requires res_mysql
	&nbsp; &nbsp; &nbsp; &nbsp; res_mysql requires vcszone
	&nbsp; &nbsp; &nbsp; &nbsp; vcszone requires vcspool
	&nbsp;
	&nbsp;
	&nbsp; &nbsp; &nbsp; &nbsp; // resource dependency tree
	&nbsp; &nbsp; &nbsp; &nbsp; //
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;group testgrp
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;{
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;Apache res_apache
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;{
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;MySQL res_mysql
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;{
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Zone vcszone
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;{
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Zpool vcspool
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;}
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;}
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;}
	&nbsp; &nbsp; &nbsp; &nbsp; // &nbsp; &nbsp; &nbsp;}
	&nbsp;
	&nbsp;
	bash-3.2#
	&nbsp;

&nbsp;

kjbss · Answer

The problem is that the command:
/opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout
is taking more than 10 seconds to execute.&nbsp; That is crazy-long for such a simple command -- on a normal system this command completes in 0.01 seconds:
timex /opt/VRTSvcs/bin/hatype -value Apache MonitorTimeout
60

real           0.01
user           0.00
sys            0.00

So you need to manually run that command and see what you get.
If it is hanging, you need to investigate why.&nbsp; You may need to restart some parts of the VCS service-set; though it might simply be easier to restart VCS on that server.
Is there more than one VCS node in the cluster?&nbsp; If so, try that command on another node; that should indicate whether the problem is node-specific, of "cluster-wide"...
-HTH
&nbsp;

Forum Discussion

VCS 6.0.1 on Solaris x64 apache resource not coming up

Related Content

apache in cluster on solaris 10

SORT Site Information for VCS Apache Agent - Solaris SPARC

Synchronise fails msmq not enough resources

Solaris 11.4 support

httpdDir attribute of Apache resource

Recent Discussions

registration

Configuring SQL Server in a VMware environment?

Configure two Mount type resources of nfs FStype attribute using the same share

order

key registration and reservation