cancel
Showing results for 
Search instead for 
Did you mean: 

Resource became OFFLINE unexpectedly on its own

chuckchang23
Level 3

Hi Experts,

 

I would like to know if there is any other way to check what is causing some resources to go down unexpectedly?

 

 

May 15 14:54:46 EMMDPD07 AgentFramework[7135]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13067 Thread(9) Agent is calling clean for resource(Server1) because the resource became OFFLINE unexpectedly, on its own.
May 15 14:54:47 EMMDPD07 Had[6891]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13067 (EMMDPD07) Agent is calling clean for resource(Server1) because the resource became OFFLINE unexpectedly, on its own.
May 15 14:54:48 EMMDPD07 AgentFramework[7135]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13068 Thread(9) Resource(Server1) - clean completed successfully.
May 15 14:54:48 EMMDPD07 AgentFramework[7135]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13073 Thread(9) Resource(Server1) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 1) the resource.
May 15 14:54:48 EMMDPD07 Had[6891]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13073 (EMMDPD07) Resource(Server1) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 1) the resource.
 
 
Any comment or suggestion is appreciated :)
 
1 ACCEPTED SOLUTION

Accepted Solutions

mikebounds
Level 6
Partner Accredited

I don't understand how you got resource Server1 to go online in the first place:

As stinsong says, Mediator is not a standard resource type that is found in the types files you have included in your main.cf, but I can only think you added it to the types.cf file as if Mediator type didn't exist, then "hares -add" command would have failed, or if you edited the main.cf, then VCS would not have started - so assuming you did added a Mediator type the other problems are:

  1. You have mulitple Mediator resources (Server1, Server2, Server3, Server4) that all have the same attributes (i.e they all have no attributes), so they cannot be distinuished from each other, so if VCS detects Mediator application is down, how does it know which of the 4 it is - it can't know.
     
  2. You can't just add Mediator to types file - it needs to have corresponding agent files in /opt/VRTSvcs/bin or /opt/VRTSagents/ha/bin

Mike

View solution in original post

11 REPLIES 11

stinsong
Level 5

Hi Chuck,

Pls paste VCS configure file /etc/VRTSvcs/conf/config/main.cf to check what the Resource(Server1) is.

And from the logs above, it's the resource offline out of VCS which is typically caused by the resource itself.

chuckchang23
Level 3

Hi Stinsong

 

 

# cat  /etc/VRTSvcs/conf/config/main.cf
include "types.cf"
include "CFSTypes.cf"
include "CVMTypes.cf"
 
cluster MediatorCluster (
        UserNames = { vcsguest = cD2a90jzh1hgg, vcsop = j5rBONYy1OtL6,
                 vcsadm = "sEFEKH1CaHW5.",
                 root = PHbwhE5x2iM9I,
                 admin = bMMgMKlTMoMU }
        Administrators = { vcsadm, root, admin }
        UseFence = SCSI3
        HacliUserLevel = COMMANDROOT
        )
 
system EMMDPD03 (
        )
 
system EMMDPD04 (
        )
 
system EMMDPD05 (
        )
 
system EMMDPD06 (
        )
 
system EMMDPD07 (
        )
 
group FMMgrp (
        SystemList = { EMMDPD03 = 2, EMMDPD04 = 1 }
        AutoStartList = { EMMDPD03 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC FMMgrp_IP (
                Address = "10.226.138.137"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        ORACLE fmm (
                )
 
        Proxy FMMgrp_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        Tomcat fmmweb (
                )
 
        requires group SENTINELgrp online global soft
        FMMgrp_IP requires FMMgrp_NIC_PROXY
        fmm requires FMMgrp_IP
        fmmweb requires fmm
 
 
        // resource dependency tree
        //
        //      group FMMgrp
        //      {
        //      Tomcat fmmweb
        //          {
        //          ORACLE fmm
        //              {
        //              IPMultiNIC FMMgrp_IP
        //                  {
        //                  Proxy FMMgrp_NIC_PROXY
        //                  }
        //              }
        //          }
        //      }
 
 
group Mediator1 (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        AutoStartList = { EMMDPD03 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC Mediator1_IP (
                Address = "10.226.138.140"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        Mediator Server1 (
                )
 
        Proxy Mediator1_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        requires group ServerGroup1_DG online local firm
        Mediator1_IP requires Mediator1_NIC_PROXY
        Server1 requires Mediator1_IP
 
 
        // resource dependency tree
        //
        //      group Mediator1
        //      {
        //      Mediator Server1
        //          {
        //          IPMultiNIC Mediator1_IP
        //              {
        //              Proxy Mediator1_NIC_PROXY
        //              }
        //          }
        //      }
 
 
group Mediator2 (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        AutoStartList = { EMMDPD04 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC Mediator2_IP (
                Address = "10.226.138.141"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        Mediator Server2 (
                )
 
        Proxy Mediator2_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        requires group ServerGroup1_DG online local firm
        Mediator2_IP requires Mediator2_NIC_PROXY
        Server2 requires Mediator2_IP
 
 
        // resource dependency tree
        //
        //      group Mediator2
        //      {
        //      Mediator Server2
        //          {
        //          IPMultiNIC Mediator2_IP
        //              {
        //              Proxy Mediator2_NIC_PROXY
        //              }
        //          }
        //      }
 
 
group Mediator3 (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        AutoStartList = { EMMDPD05 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC Mediator3_IP (
                Address = "10.226.138.142"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        Mediator Server3 (
                )
 
        Proxy Mediator3_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        requires group ServerGroup1_DG online local firm
        Mediator3_IP requires Mediator3_NIC_PROXY
        Server3 requires Mediator3_IP
 
 
        // resource dependency tree
        //
        //      group Mediator3
        //      {
        //      Mediator Server3
        //          {
        //          IPMultiNIC Mediator3_IP
        //              {
        //              Proxy Mediator3_NIC_PROXY
        //              }
        //          }
        //      }
 
 
group Mediator4 (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        AutoStartList = { EMMDPD06 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC Mediator4_IP (
                Address = "10.226.138.143"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        Mediator Server4 (
                )
 
        Proxy Mediator4_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        requires group ServerGroup1_DG online local firm
        Mediator4_IP requires Mediator4_NIC_PROXY
        Server4 requires Mediator4_IP
 
 
        // resource dependency tree
        //
        //      group Mediator4
        //      {
        //      Mediator Server4
        //          {
        //          IPMultiNIC Mediator4_IP
        //              {
        //              Proxy Mediator4_NIC_PROXY
        //              }
        //          }
        //      }
 
 
group Mediator5 (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        AutoStartList = { EMMDPD06 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC Mediator5_IP (
                Address = "10.226.138.144"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        Mediator Server5 (
                )
 
        Proxy Mediator5_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        requires group ServerGroup1_DG online local firm
        Mediator5_IP requires Mediator5_NIC_PROXY
        Server5 requires Mediator5_IP
 
 
        // resource dependency tree
        //
        //      group Mediator5
        //      {
        //      Mediator Server5
        //          {
        //          IPMultiNIC Mediator5_IP
        //              {
        //              Proxy Mediator5_NIC_PROXY
        //              }
        //          }
        //      }
 
 
group Network (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        Parallel = 1
        AutoStartList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06, EMMDPD07 }
        )
 
        MultiNICA MultiNICA_CH (
                Device @EMMDPD03 = { e1000g0 = "10.226.138.132",
                         igb0 = "10.226.138.132" }
                Device @EMMDPD04 = { e1000g0 = "10.226.138.133",
                         igb0 = "10.226.138.133" }
                Device @EMMDPD05 = { e1000g0 = "10.226.138.134",
                         igb0 = "10.226.138.134" }
                Device @EMMDPD06 = { e1000g0 = "10.226.138.135",
                         igb0 = "10.226.138.135" }
                Device @EMMDPD07 = { e1000g0 = "10.226.138.136",
                         igb0 = "10.226.138.136" }
                NetMask = "255.255.255.224"
                ArpDelay = 5
                RouteOptions = "10.226.138.129"
                IfconfigTwice = 1
                NetworkHosts = { "10.226.138.129" }
                )
 
        MultiNICA MultiNICA_OM (
                Device @EMMDPD03 = { e1000g1 = "10.226.147.144" }
                Device @EMMDPD04 = { e1000g1 = "10.226.147.145" }
                Device @EMMDPD05 = { e1000g1 = "10.226.147.146" }
                Device @EMMDPD06 = { e1000g1 = "10.226.147.147" }
                Device @EMMDPD07 = { e1000g1 = "10.226.147.148" }
                NetMask = "255.255.254.0"
                ArpDelay = 5
                RouteOptions = "10.226.146.1"
                IfconfigTwice = 1
                NetworkHosts = { "10.226.146.1", "10.226.147.148" }
                )
 
        Phantom Phantom (
                )
 
 
 
        // resource dependency tree
        //
        //      group Network
        //      {
        //      MultiNICA MultiNICA_CH
        //      MultiNICA MultiNICA_OM
        //      Phantom Phantom
        //      }
 
 
group Oracle1 (
        SystemList = { EMMDPD03 = 1, EMMDPD04 = 2, EMMDPD05 = 3, EMMDPD06 = 4,
                 EMMDPD07 = 5 }
        AutoStartList = { EMMDPD04 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC Oracle1_IP (
                Address = "10.226.138.138"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        ORACLE bgw (
                )
 
        Proxy Oracle1_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        requires group ora_DG online local firm
        Oracle1_IP requires Oracle1_NIC_PROXY
        bgw requires Oracle1_IP
 
 
        // resource dependency tree
        //
        //      group Oracle1
        //      {
        //      ORACLE bgw
        //          {
        //          IPMultiNIC Oracle1_IP
        //              {
        //              Proxy Oracle1_NIC_PROXY
        //              }
        //          }
        //      }
 
 
group SENTINELgrp (
        SystemList = { EMMDPD03 = 2, EMMDPD04 = 1, EMMDPD05 = 2, EMMDPD06 = 3,
                 EMMDPD07 = 4 }
        AutoStartList = { EMMDPD05 }
        FailOverPolicy = RoundRobin
        )
 
        IPMultiNIC SENTINELgrp_IP (
                Address = "10.226.138.139"
                NetMask = "255.255.255.224"
                MultiNICResName = MultiNICA_CH
                IfconfigTwice = 1
                )
 
        Proxy SENTINELgrp_NIC_PROXY (
                TargetResName = MultiNICA_CH
                )
 
        Sentinel licserv (
                )
 
        requires group lic_DG online local firm
        SENTINELgrp_IP requires SENTINELgrp_NIC_PROXY
        SENTINELgrp_IP requires licserv
 
 
        // resource dependency tree
        //
        //      group SENTINELgrp
        //      {
        //      IPMultiNIC SENTINELgrp_IP
        //          {
        //          Proxy SENTINELgrp_NIC_PROXY
        //          Sentinel licserv
        //          }
        //      }
 
 
group ServerGroup1_DG (
        SystemList = { EMMDPD03 = 0, EMMDPD04 = 1, EMMDPD05 = 2, EMMDPD06 = 3,
                 EMMDPD07 = 4 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06, EMMDPD07 }
        )
 
        CFSMount cfsmount2 (
                Critical = 0
                MountPoint = "/var/opt/BGw/ServerGroup1"
                BlockDevice = "/dev/vx/dsk/bgw1dg/vol01"
                MountOpt @EMMDPD03 = "cluster"
                MountOpt @EMMDPD04 = "cluster"
                MountOpt @EMMDPD05 = "cluster"
                MountOpt @EMMDPD06 = "cluster"
                MountOpt @EMMDPD07 = "cluster"
                NodeList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06,
                         EMMDPD07 }
                )
 
        CVMVolDg cvmvoldg2 (
                Critical = 0
                CVMDiskGroup = bgw1dg
                CVMActivation @EMMDPD03 = sw
                CVMActivation @EMMDPD04 = sw
                CVMActivation @EMMDPD05 = sw
                CVMActivation @EMMDPD06 = sw
                CVMActivation @EMMDPD07 = sw
                )
 
        requires group cvm online local firm
        cfsmount2 requires cvmvoldg2
 
 
        // resource dependency tree
        //
        //      group ServerGroup1_DG
        //      {
        //      CFSMount cfsmount2
        //          {
        //          CVMVolDg cvmvoldg2
        //          }
        //      }
 
 
group cvm (
        SystemList = { EMMDPD03 = 0, EMMDPD04 = 1, EMMDPD05 = 2, EMMDPD06 = 3,
                 EMMDPD07 = 4 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06, EMMDPD07 }
        )
 
        CFSfsckd vxfsckd (
                ActivationMode @EMMDPD03 = { bgw1dg = sw, ora1dg = sw, lic1dg = sw,
                         fmm1dg = sw }
                ActivationMode @EMMDPD04 = { bgw1dg = sw, ora1dg = sw, lic1dg = sw,
                         fmm1dg = sw }
                ActivationMode @EMMDPD05 = { bgw1dg = sw, lic1dg = sw, fmm1dg = sw,
                         ora1dg = sw }
                ActivationMode @EMMDPD06 = { bgw1dg = sw, lic1dg = sw, fmm1dg = sw,
                         ora1dg = sw }
                ActivationMode @EMMDPD07 = { bgw1dg = sw, lic1dg = sw, fmm1dg = sw,
                         ora1dg = sw }
                )
 
        CVMCluster cvm_clus (
                CVMClustName = MediatorCluster
                CVMNodeId = { EMMDPD03 = 0, EMMDPD04 = 1, EMMDPD05 = 2,
                         EMMDPD06 = 3,
                         EMMDPD07 = 4 }
                CVMTransport = gab
                CVMTimeout = 200
                )
 
        CVMVxconfigd cvm_vxconfigd (
                Critical = 0
                CVMVxconfigdArgs = { syslog }
                )
 
        cvm_clus requires cvm_vxconfigd
        vxfsckd requires cvm_clus
 
 
        // resource dependency tree
        //
        //      group cvm
        //      {
        //      CFSfsckd vxfsckd
        //          {
        //          CVMCluster cvm_clus
        //              {
        //              CVMVxconfigd cvm_vxconfigd
        //              }
        //          }
        //      }
 
 
group lic_DG (
        SystemList = { EMMDPD03 = 0, EMMDPD04 = 1, EMMDPD05 = 2, EMMDPD06 = 3,
                 EMMDPD07 = 4 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06, EMMDPD07 }
        )
 
        CFSMount cfsmount5 (
                Critical = 0
                MountPoint = "/var/opt/sentinel"
                BlockDevice = "/dev/vx/dsk/lic1dg/vol01"
                MountOpt @EMMDPD03 = "cluster"
                MountOpt @EMMDPD04 = "cluster"
                MountOpt @EMMDPD05 = "cluster"
                MountOpt @EMMDPD06 = "cluster"
                MountOpt @EMMDPD07 = "cluster"
                NodeList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06,
                         EMMDPD07 }
                )
 
        CFSMount cfsmount6 (
                Critical = 0
                MountPoint = "/var/opt/mediation/fmmdb"
                BlockDevice = "/dev/vx/dsk/fmm1dg/vol01"
                MountOpt @EMMDPD03 = "cluster"
                MountOpt @EMMDPD04 = "cluster"
                MountOpt @EMMDPD05 = "cluster"
                MountOpt @EMMDPD06 = "cluster"
                MountOpt @EMMDPD07 = "cluster"
                NodeList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06,
                         EMMDPD07 }
                )
 
        CVMVolDg cvmvoldg5 (
                Critical = 0
                CVMDiskGroup = lic1dg
                CVMActivation @EMMDPD03 = sw
                CVMActivation @EMMDPD04 = sw
                CVMActivation @EMMDPD05 = sw
                CVMActivation @EMMDPD06 = sw
                CVMActivation @EMMDPD07 = sw
                )
 
        CVMVolDg cvmvoldg6 (
                Critical = 0
                CVMDiskGroup = fmm1dg
                CVMActivation @EMMDPD03 = sw
                CVMActivation @EMMDPD04 = sw
                CVMActivation @EMMDPD05 = sw
                CVMActivation @EMMDPD06 = sw
                CVMActivation @EMMDPD07 = sw
                )
 
        requires group cvm online local firm
        cfsmount5 requires cvmvoldg5
        cfsmount6 requires cvmvoldg6
 
 
        // resource dependency tree
        //
        //      group lic_DG
        //      {
        //      CFSMount cfsmount5
        //          {
        //          CVMVolDg cvmvoldg5
        //          }
        //      CFSMount cfsmount6
        //          {
        //          CVMVolDg cvmvoldg6
        //          }
        //      }
 
 
group ora_DG (
        SystemList = { EMMDPD03 = 0, EMMDPD04 = 1, EMMDPD05 = 2, EMMDPD06 = 3,
                 EMMDPD07 = 4 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06, EMMDPD07 }
        )
 
        CFSMount cfsmount4 (
                Critical = 0
                MountPoint = "/var/opt/mediation/ora"
                BlockDevice = "/dev/vx/dsk/ora1dg/vol01"
                MountOpt @EMMDPD03 = "cluster"
                MountOpt @EMMDPD04 = "cluster"
                MountOpt @EMMDPD05 = "cluster"
                MountOpt @EMMDPD06 = "cluster"
                MountOpt @EMMDPD07 = "cluster"
                NodeList = { EMMDPD03, EMMDPD04, EMMDPD05, EMMDPD06,
                         EMMDPD07 }
                )
 
        CVMVolDg cvmvoldg4 (
                Critical = 0
                CVMDiskGroup = ora1dg
                CVMActivation @EMMDPD03 = sw
                CVMActivation @EMMDPD04 = sw
                CVMActivation @EMMDPD05 = sw
                CVMActivation @EMMDPD06 = sw
                CVMActivation @EMMDPD07 = sw
                )
 
        requires group cvm online local firm
        cfsmount4 requires cvmvoldg4
 
 
        // resource dependency tree
        //
        //      group ora_DG
        //      {
        //      CFSMount cfsmount4
        //          {
        //          CVMVolDg cvmvoldg4
        //          }
        //      }
 
 
 
 

arangari
Level 5

what is the type-definition and the entrypoint implmentations for agent Mediator?

look at the agent's log file

 

stinsong
Level 5

From the main.cf could tell the resource "server1" is a blank defined resource with resource type Mediator. 

But chuck, the resource type Mediator is not exist from VCS. You cannot define a resource that does not exist from any *Types.cf files.

So it's not VCS problem. Either there should no use of such resource Mediator or missing type defination.

mikebounds
Level 6
Partner Accredited

I don't understand how you got resource Server1 to go online in the first place:

As stinsong says, Mediator is not a standard resource type that is found in the types files you have included in your main.cf, but I can only think you added it to the types.cf file as if Mediator type didn't exist, then "hares -add" command would have failed, or if you edited the main.cf, then VCS would not have started - so assuming you did added a Mediator type the other problems are:

  1. You have mulitple Mediator resources (Server1, Server2, Server3, Server4) that all have the same attributes (i.e they all have no attributes), so they cannot be distinuished from each other, so if VCS detects Mediator application is down, how does it know which of the 4 it is - it can't know.
     
  2. You can't just add Mediator to types file - it needs to have corresponding agent files in /opt/VRTSvcs/bin or /opt/VRTSagents/ha/bin

Mike

chuckchang23
Level 3

Hi Mikebounds and Stingson,

Thank you both for the inputs :)

Anyway, I got this from the type.cf

type Mediator (

 

        static str LogLevel
        static int OnlineRetryLimit = 1
        static int OnlineTimeout = 60
        static int RestartLimit = 1
        static str ArgList[] = { Dummy }
        str Dummy
)
 
and it seems to have agents 
 
EMMDPD07:/# cd /opt/VRTSvcs/bin
EMMDPD07:/opt/VRTSvcs/bin# ls Mediator/
MediatorAgent  clean          monitor        offline        online
 
 
but still, is there a way to check what caused the sudden offline?
 

chuckchang23
Level 3

Hi arangari, 

im sorry im not familiar with "entrypoint" how can i get the agent's log file?

 

 

arangari
Level 5

 @Chuckchang23

1. Is the Mediator agent developed as per the instructions from 'Agent Developer Guide' ? Please refer to this guide from SORT.

2. You need to look at the logic in 'monitor' file  under /opt/VRTSvcs/bin/Mediator to determine why the resource could be detected as offline. 

3. See if there have been events related to the 'server5' instance of the resource which lead the 'monitor' determining the resource as offline.

 

I would strongly urge to refer to 'VCS Administrator guide' too. 

Satish_K__Pagar
Level 5
Employee

VCS reports the resource as OFFLINE when the monitor entry point of the agent exits with a value 100. What could help you understand in this case, is go through the monitor program "/opt/VRTSvcs/bin/Mediator/monitor" , see when it is exiting the with the code 100. That would help you determine what could be the situation when VCS reported the resource as OFFLINE.

stinsong
Level 5

Hi chuck,

Seems you have customized types.cf for the resource Mediator.

But like mike said, there are 4 resources of Mediator without any attributes. You should use proxy resource base on same resource for different service group.

Anyway you can check on the agent log /var/VRTSvcs/log/Mediator_A.log if there is.

mikebounds
Level 6
Partner Accredited

Can you explain more about the Mediator application.

As it stands, you have not described any "instance" information in your resource which implies there is only one Mediator application that all 4 service group requires as oppose to each service group requiring its own instance of the Mediator.  You have defined "offline" for Mediator and therefore when you offline one service group it will offline the Mediator application so the other 3 service groups will see the Mediator application go offline and hence report "resource became OFFLINE unexpectedly, on its own".

You only have an IP in the Mediator service groups, so I don't know if this means config is unfinished and there will be other resources or it is only IP - if it is only IP, then this perhaps suggests that you have 4 instances of Mediator, in which case you need an attribute to distinguish the Mediators so VCS knows which Mediator to start, stop and monitor for each group.

So we need to know if you have one Mediator  or 4 and if there is only one, could it be running on both nodes at the same time and have  Mediator1, Mediator2, Mediator3 & Mediator4 split between the nodes or should all 4 Mediators always be on the same server.

Mike