cancel
Showing results for 
Search instead for 
Did you mean: 

Note able to configure the HA Proxy with Application Agent

lazaur
Level 4

Hi Guys,

 

I am not able to configure the HA proxy with application AGENT. its not giving the status after probed completed.

Application HAPROXY (
                Critical = 0
                StartProgram = "/usr/src/haproxy-1.5.12/HAPROXY start"
                StopProgram = " /usr/src/haproxy-1.5.12/HAPROXY stop"
                CleanProgram = "/usr/src/haproxy-1.5.12/HAPROXY stop"
                PidFiles = { "/var/run/haproxy.pid" }
                MonitorProcesses = { haproxy }
                )


------------------------------------------------------------------------------------------------------------------

ApplicationAgent  logs --

2015/07/06 15:57:15 VCS DBG_AGDEBUG Calling monitor for resource HAPROXY
        VCSAgType.C:call_monitor[1492]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:monitor program <> before squeeze()
        Application.C:isMonitorProgramConfigured[1286]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:inside function squeeze(),arguments passed is <>
        Application.C:squeeze[1339]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Exiting from function squeeze, no of tokens are <0> , returning <-1>
        Application.C:squeeze[1379]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:monitor program <> after squeeze()
        Application.C:isMonitorProgramConfigured[1298]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Exiting from isMonitorProgramConfigured, return code <0>
        Application.C:isMonitorProgramConfigured[1299]
2015/07/06 15:57:15 VCS DBG_1 Application:HAPROXY:monitor:UseSUDash:<0>.
        Application.C:application_monitor[167]
2015/07/06 15:57:15 VCS DBG_1 Application:HAPROXY:monitor:State=<0>, IState=<1>
        Application.C:application_monitor[188]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Validating and retrieving the information for user <root>
        Application.C:application_monitor[190]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Retrieving information for User <root>
        Application.C:getuserinfo[1233]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Retrieved information for: User <root> Home Directory </root> Login Name <root> Default Shell </bin/bash>
        Application.C:getuserinfo[1252]
2015/07/06 15:57:15 VCS DBG_AGDEBUG Monitor-Info is 5
        VCSAgPrimitives.C:VCSAgGetMonitorLevel[4588]
2015/07/06 15:57:15 VCS DBG_AGDEBUG LevelTwoMonitorFreq is 1
        VCSAgPrimitives.C:VCSAgIsL2MFSet[4504]
2015/07/06 15:57:15 VCS DBG_3 Application:HAPROXY:monitor:level_one_monitor = [1], level_two_monitor = [2], l2mf set [1]
        Application.C:application_monitor[211]
2015/07/06 15:57:15 VCS DBG_1 Application:HAPROXY:monitor:Total number of Pid Files specified:1.
        Application.C:application_monitor[226]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Checking for PidFile:</var/run/haproxy.pid>
        Application.C:application_monitor[235]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:inside function squeeze(),arguments passed is </var/run/haproxy.pid>
        Application.C:squeeze[1339]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Exiting from function squeeze, no of tokens are <1> , returning <0>
        Application.C:squeeze[1379]
2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Checking for pid in PidFile:</var/run/haproxy.pid>
        Application.C:application_monitor[249]
2015/07/06 15:57:15 VCS DBG_4 Application:HAPROXY:monitor:Got pid-file:/var/run/haproxy.pid to check for the pid.
        Application.C:pidExists[419]
2015/07/06 15:57:15 VCS DBG_3 Application:HAPROXY:monitor:Found pid (17961) from pid-file (/var/run/haproxy.pid).
        Application.C:pidExists[457]
2015/07/06 15:57:15 VCS DBG_2 Application:HAPROXY:monitor:Pid-file:</var/run/haproxy.pid>; checking the existence of pid:<17961>
        Application.C:pidExists[467]
2015/07/06 15:57:15 VCS DBG_3 Application:HAPROXY:monitor:Error occured. Pid-file:</var/run/haproxy.pid>; Pid (17961) does not exists.
        Application.C:pidExists[472]

        Application.C:pidExists[472]
2015/07/06 15:57:15 VCS DBG_4 Application:HAPROXY:monitor:Status got after checking Pid File (/var/run/haproxy.pid):OFFLINE.
        Application.C:application_monitor[255]
2015/07/06 15:57:15 VCS DBG_AGINFO Resource(HAPROXY) - monitor entry point exited with a confidence value 0.
        VCSAgType.C:call_monitor[1584]
2015/07/06 15:57:15 VCS DBG_AGINFO HAPROXY reported state (Offline) & conf_level (0)
        VCSAgRes.C:call_entry_point[1599]
2015/07/06 15:57:15 VCS DBG_AGDEBUG Canceling timer for (HAPROXY) op(1608)
        VCSAgTimer.C:_cancel[947]
==============================================

Engine_A_logs.

VCS_CONF=/etc/VRTSvcs
VCS_DIAG=/var/VRTSvcs
VCS_HOME=/opt/VRTSvcs
VCS_LOG_AGENT_NAME=
VCS_LOG_CATEGORY=6
VCS_LOG_SCRIPT_NAME=hatrigger
VCS_LOG=/var/VRTSvcs
########## Other Environment Variables ##########
CONSOLETYPE=vt
LANG=en_US.UTF-8
LANGSH_SOURCED=1
LD_LIBRARY_PATH=/opt/VRTSvcs/lib:
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/sbin:/usr/sbin:/bin:/usr/bin:/opt/VRTSvcs/bin
previous=N
PREVLEVEL=N
PWD=/var/VRTSvcs/diag/had
runlevel=5
RUNLEVEL=5
SHLVL=5
SYSTEMCTL_SKIP_REDIRECT=true
TERM=linux
UPSTART_EVENTS=runlevel
UPSTART_INSTANCE=
UPSTART_JOB=rc
_=/usr/bin/env

2015/07/06 15:36:40 VCS INFO V-16-6-15002 (DTPJBS01) hatrigger:hatrigger executed /opt/VRTSvcs/bin/internal_triggers/dump_tunables DTPJBS01 1 successfully
2015/07/06 15:36:42 VCS NOTICE V-16-1-10438 Group ClusterService has been probed on system DTPJBS01
2015/07/06 15:36:43 VCS INFO V-16-1-10304 Resource JBOSS (Owner: Unspecified, Group: JBOSS_APP) is offline on DTPJBS01 (First probe)
2015/07/06 15:36:43 VCS NOTICE V-16-1-10438 Group JBOSS_APP has been probed on system DTPJBS01
2015/07/06 15:36:43 VCS INFO V-16-1-50007 Initiating auto-start online of group JBOSS_APP
2015/07/06 15:36:43 VCS INFO V-16-1-10493 Evaluating DTPJBS01 as potential target node for group JBOSS_APP
2015/07/06 15:36:43 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group JBOSS_APP on all nodes
2015/07/06 15:36:43 VCS NOTICE V-16-1-10301 Initiating Online of Resource JBOSS (Owner: Unspecified, Group: JBOSS_APP) on System DTPJBS01
2015/07/06 15:36:44 VCS NOTICE V-16-1-10438 Group VCShmg has been probed on system DTPJBS01
2015/07/06 15:36:44 VCS NOTICE V-16-1-10435 Group VCShmg will not start automatically on System DTPJBS01 as the system is not a part of AutoStartList attribute of the group.
2015/07/06 15:36:56 VCS INFO V-16-1-10298 Resource JBOSS (Owner: Unspecified, Group: JBOSS_APP) is online on DTPJBS01 (VCS initiated)
2015/07/06 15:36:56 VCS NOTICE V-16-1-10447 Group JBOSS_APP is online on system DTPJBS01

2015/07/06 15:52:41 VCS INFO V-16-1-55029 Resource HAPROXY in offline state received recurring offline message on system DTPJBS02
2015/07/06 15:52:55 VCS NOTICE V-16-1-10460 Clearing start attribute for resource HAPROXY of group HAProxy on node DTPJBS01
2015/07/06 15:52:55 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group HAProxy on all nodes
2015/07/06 15:52:55 VCS NOTICE V-16-1-10301 Initiating Online of Resource HAPROXY (Owner: Unspecified, Group: HAProxy) on System DTPJBS02
2015/07/06 15:52:55 VCS INFO V-16-1-50159 User fired command: hares -online HAPROXY  DTPJBS02  from node DTPJBS01
2015/07/06 15:54:56 VCS ERROR V-16-2-13066 (DTPJBS02) Agent is calling clean for resource(HAPROXY) because the resource is not up even after online completed.
2015/07/06 15:54:57 VCS INFO V-16-2-13068 (DTPJBS02) Resource(HAPROXY) - clean completed successfully.
2015/07/06 15:54:57 VCS INFO V-16-2-13071 (DTPJBS02) Resource(HAPROXY): reached OnlineRetryLimit(0).
2015/07/06 15:54:57 VCS ERROR V-16-1-54031 Resource HAPROXY (Owner: Unspecified, Group: HAProxy) is FAULTED on sys DTPJBS02
2015/07/06 15:54:57 VCS NOTICE V-16-1-10446 Group HAProxy is offline on system DTPJBS02
2015/07/06 15:54:57 VCS INFO V-16-6-15015 (DTPJBS02) hatrigger:/opt/VRTSvcs/bin/triggers/resfault is not a trigger scripts directory or can not be executed
2015/07/06 15:55:09 VCS INFO V-16-1-50159 User fired command: hares -clear HAPROXY  DTPJBS02  from node DTPJBS01
2015/07/06 15:55:09 VCS INFO V-16-1-10307 Resource HAPROXY (Owner: Unspecified, Group: HAProxy) is offline on DTPJBS02 (Not initiated by VCS)
2015/07/06 15:55:13 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group HAProxy on all nodes
2015/07/06 15:55:13 VCS NOTICE V-16-1-10301 Initiating Online of Resource HAPROXY (Owner: Unspecified, Group: HAProxy) on System DTPJBS02
2015/07/06 15:55:13 VCS INFO V-16-1-50159 User fired command: hares -online HAPROXY  DTPJBS02  from node DTPJBS01
2015/07/06 15:57:14 VCS ERROR V-16-2-13066 (DTPJBS02) Agent is calling clean for resource(HAPROXY) because the resource is not up even after online completed.
2015/07/06 15:57:15 VCS INFO V-16-2-13068 (DTPJBS02) Resource(HAPROXY) - clean completed successfully.
2015/07/06 15:57:15 VCS INFO V-16-2-13071 (DTPJBS02) Resource(HAPROXY): reached OnlineRetryLimit(0).
2015/07/06 15:57:15 VCS ERROR V-16-1-54031 Resource HAPROXY (Owner: Unspecified, Group: HAProxy) is FAULTED on sys DTPJBS02
2015/07/06 15:57:15 VCS NOTICE V-16-1-10446 Group HAProxy is offline on system DTPJBS02
2015/07/06 15:57:15 VCS INFO V-16-6-15015 (DTPJBS02) hatrigger:/opt/VRTSvcs/bin/triggers/resfault is not a trigger scripts directory or can not be executed
2015/07/06 16:02:38 VCS INFO V-16-1-50159 User fired command: hares -clear HAPROXY  DTPJBS02  from node DTPJBS01
2015/07/06 16:02:38 VCS INFO V-16-1-10307 Resource HAPROXY (Owner: Unspecified, Group: HAProxy) is offline on DTPJBS02 (Not initiated by VCS)
2015/07/06 16:02:43 VCS INFO V-16-1-55029 Resource HAPROXY in offline state received recurring offline message on system DTPJBS02
2015/07/06 16:03:29 VCS INFO V-16-1-50159 User fired command: haconf -dump from node DTPJBS01
2015/07/06 18:48:51 VCS INFO V-16-1-53504 VCS Engine Alive message!!
2015/07/06 22:48:52 VCS INFO V-16-1-53504 VCS Engine Alive message!!
2015/07/07 02:48:53 VCS INFO V-16-1-53504 VCS Engine Alive message!!
2015/07/07 06:48:54 VCS INFO V-16-1-53504 VCS Engine Alive message!!

 

Please hlep.

 

 
Sarfraz Qazi
Certified
8 REPLIES 8

lazaur
Level 4

HI Guys,

 

Any solution.

 
Sarfraz Qazi
Certified

sudhir_h
Level 4
Employee

Hi,

 

As per the logs it seems that the Process id that is set in the PID file is not existing.

 

<snip>

2015/07/06 15:57:15 VCS DBG_5 Application:HAPROXY:monitor:Checking for pid in PidFile:</var/run/haproxy.pid>
        Application.C:application_monitor[249]
2015/07/06 15:57:15 VCS DBG_4 Application:HAPROXY:monitor:Got pid-file:/var/run/haproxy.pid to check for the pid.
        Application.C:pidExists[419]
2015/07/06 15:57:15 VCS DBG_3 Application:HAPROXY:monitor:Found pid (17961) from pid-file (/var/run/haproxy.pid).
        Application.C:pidExists[457]
2015/07/06 15:57:15 VCS DBG_2 Application:HAPROXY:monitor:Pid-file:</var/run/haproxy.pid>; checking the existence of pid:<17961>
        Application.C:pidExists[467]
2015/07/06 15:57:15 VCS DBG_3 Application:HAPROXY:monitor:Error occured. Pid-file:</var/run/haproxy.pid>; Pid (17961) does not exists.
        Application.C:pidExists[472]         Application.C:pidExists[472]
2015/07/06 15:57:15 VCS DBG_4 Application:HAPROXY:monitor:Status got after checking Pid File (/var/run/haproxy.pid):OFFLINE.

</snip>

 

1. Set ManageFaults to None for the Service group before performing the below steps, so that you can capture the output after online is initated from VCS.

2. check manually if the process with the PID mentioned in the file "/var/run/haproxy.pid" exists?

3. Provide the output of the command "ps" with long listing command and "grep" for "haproxy.

4. If you provide only "MonitorProcesses", does it report online?

Regards,

Sudhir

 

 

 

 

 

 

mikebounds
Level 6
Partner Accredited

Although you can use both PidFiles and MonitorProcesses, I would use one of the other as I can't see any benefit using both.

For MonitorProcesses you need to use the full string as see by "ps -ef"

Mike

lazaur
Level 4

Hi Sudhir/Mike,

now the setup is working fine but i m not able to monitor the process by Monitoring script. when giving PID its working fine.

below is monitoring script and this is working fine.

#!/bin/sh
APPLICATION_IS_ONLINE=110
APPLICATION_IS_OFFLINE=100
if [ -f /var/run/haproxy.pid ] ; then # add any steps, if required
exit $APPLICATION_IS_ONLINE
else
exit $APPLICATION_IS_OFFLINE
fi


and when giving script file it stop working i used the -x option also instead -f.

#!/bin/sh
APPLICATION_IS_ONLINE=110
APPLICATION_IS_OFFLINE=100
if [ -f  /opt/hastatus ] ; then # add any steps, if required
exit $APPLICATION_IS_ONLINE
else
exit $APPLICATION_IS_OFFLINE
fi

Script file.

#!/bin/bash
    ps -ef | grep haproxy | grep -v grep | awk ' { print $2 } '

Please help

 
Sarfraz Qazi
Certified

mikebounds
Level 6
Partner Accredited

If you are just monitoring a process then you do not need to use MonitorProgram - just use MonitorProcesses and enter full string in CMD column of "ps -ef" output OR (not and) just specify pid file.

If this does not work - then please post outout of:

hares -display -type Application | awk '$2 ~ /MonitorProgram|MonitorProcesses|PidFiles/'

ps -ef | grep haproxy

cat /var/run/haproxy.pid 

 

Mike

sudhir_h
Level 4
Employee

Hi,

 

Using MonitorProcess would make it easier and simpler rather than using MonitorProgram. You could also take advantage of IMF for this resource to enable faster fault detection.

Run the monitor script manually and provide the exit code.

#echo $?

regards,

Sudhir

 

lazaur
Level 4

Hi Mike,

 

please find the out put as now i am using the MQ agen to monitor it.

 

[root@AOSCEDA01 ActiveMQ]# ps -ef | grep /opt/apache-activemq-5.11.1 | grep -v  grep
activemq  6814     1  8 11:47 ?        00:00:03 /usr/bin/java -Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/apache-activemq-5.11.1/conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/opt/apache-activemq-5.11.1/tmp -Dactivemq.classpath=/opt/apache-activemq-5.11.1/conf: -Dactivemq.home=/opt/apache-activemq-5.11.1 -Dactivemq.base=/opt/apache-activemq-5.11.1 -Dactivemq.conf=/opt/apache-activemq-5.11.1/conf -Dactivemq.data=/opt/apache-activemq-5.11.1/data -jar /opt/apache-activemq-5.11.1/bin/activemq.jar start
[root@AOSCEDA01 ActiveMQ]# ps -ef | grep "java.*activemq.home=/opt/apache-activemq-5.11.1" | grep -v  grep
activemq  6814     1  6 11:47 ?        00:00:03 /usr/bin/java -Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/apache-activemq-5.11.1/conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/opt/apache-activemq-5.11.1/tmp -Dactivemq.classpath=/opt/apache-activemq-5.11.1/conf: -Dactivemq.home=/opt/apache-activemq-5.11.1 -Dactivemq.base=/opt/apache-activemq-5.11.1 -Dactivemq.conf=/opt/apache-activemq-5.11.1/conf -Dactivemq.data=/opt/apache-activemq-5.11.1/data -jar /opt/apache-activemq-5.11.1/bin/activemq.jar start
[root@AOSCEDA01 ActiveMQ]# hares -display -type ActiveMQ | awk '$2 ~ /MonitorProgram|MonitorProcesses|PidFiles|MonitorProcessPatterns/'
ACTIVEMQ     MonitorProcessPatterns  global     java.*activemq.home=/opt/apache-activemq-5.11.1
ACTIVEMQ     MonitorProgram          global
ACTIVEMQ     MonitorProgramFrequency global     0
ACTIVEMQ     PidFilesPatterns        global
[root@AOSCEDA01 ActiveMQ]#

 

 

 
Sarfraz Qazi
Certified

mikebounds
Level 6
Partner Accredited

So what issue do you see now in VCS.  I would set ResLogLevel to TRACE to get more information and post output of logs with TRACE set if you can't see the issue (if using VCS 6.2 then you will need to use LogDbg, however, historically LogDbg is no where near as useful as ResLogLevel, so might not show issue).

You could also try using PidFilesPatterns attribute.

Mike