10-09-2019 09:28 AM
hi,
I would like to expose my Cluster configuration problem based on Infoscale 7.4 installed on Linux virtual environments (2 VMs), here to get some advices.
- Our application is splitted into three independant sub-applications : [Billing, Rating, Data ]. These 3 Sub-Applications are distributed on the 2 nodes "Active/Active" cluster as follows : [Data & Billing] on Node 1; Rating on Node2. The 3 sub-Applications are associated to a VIP.
- We will create 3 Service Groups (SG) : Data_SG; Billing_SG; Rating_SG.
- Dependencies between these 3 SG need to be set as follows : "Data_SG" will start first, THEN and ONLY when fully started, "Biling_SG" & "Rating_SG" will be started.
- When we start VCS, the SG are launched in a distributed mode, as mentioned above.
- In a distributed mode, "Billing_SG" leverages it own configuration File to start; "Rating_SG" leverages it own configuration file to start as well.
- In a Standalone Mode ("Billing_SG" and "Rating_SG" running on the same node), these 2 SG leverages a third configuration file which encapsulates the configuration of the 2 SG.
- So, if the "Billing_SG" OR the "Rating_SG" fails, they are supposed to try to restart on their original node to remain in a distributed configuration. But if it is not possible, both SG will be hosted on the same Cluster node, in StandAlone mode. In that case, there is a constraint which needs to be managed: we need to leverage the third configuration file as described just above. So it means that before switching the failed SG on the second node, it implies that we stop the remaining running SG, change the configuration files, and then restart the two SG on the same node.
I tried to illustrated different Used Cases in the Attached tab.
There is an other important constraint which is :
- If the "Data_SG" failed, we need to stop the "Billing_SG" and "Rating_SG" before trying to restart the "Data_SG", and then restart the "Billing_SG" and "Rating_SG" in their original location, where they were running before we stop them.
Based on these constraints, if you have any advice, It would be greetly appreciated
Regards
hubert
10-09-2019 09:41 AM
Within InfoScale (VCS), we have the idea of per/node attributes. By default, when you edit an attribute within a resrouce, it is for all nodes. If needed, one can have different values for each node. In your case, you would have a per/node start script or per/node config file for your application.
Hope that helps.
Cheers
10-15-2019 08:22 AM
Hi Sir ,
Thanks for your reply.
I try to understand how I could implement your remarks in our configuration, i mean the one relating to the start Scripts ("you would have a per/node start script or per/node config file for your application").
Based on my previous explanations and the different Use Cases depicted in the document I attached to the initial discussion, Node1 is able to run one, two, or all Service Groups. the main problem is linked to the way to differenciate what is running on each node, even if the initial one is the one presented as "Use Case1" in my Attached document.
I tried to illustrate more precisely the configuration files used in the different use cases ...
Hope it clarifies a bit .
Thanks for your comments.
Rgds
Hubert
10-15-2019 09:41 AM
Hi ,
@CliffordB : To be more complete, I guess you refer to the following (hereafter an example) regarding attributes :
group billing_sg (
SystemList = { node1 = 0, node2 = 1 }
AutoStart = 0
Parallel = 1
PreOnline = 1
)
Application billing_app (
StartProgram @node1 = "/apps/scripts/cluster/iSer_SRV_Node1_N.sh start"
StartProgram @node2 = "/apps/scripts/cluster/iSer_SRV_Node2_N.sh start"
....
)
is it something like that ?
When this application (Billing) is switching on Node2, the second one (Rating) is stopped first and both are restarted using the same script and same configuration file.
Hope it is more clear.
Rgds
hubert
10-16-2019 01:20 AM
Hi Hubert,
What you need to do is build intelligence into your start up scripts. It should do something like this
Determine the number of active nodes in the cluster (can use output from gabconfig or something to figure this out)
if number of active nodes is less than 2, use combo configuration file, else use regular configuration file
You don't need to use local variable because your use case doesn't really require it. The service groups should also not be parallel (as shown in main.cf snippet)
Cheers
Riaan
10-16-2019 10:23 AM
hi,
I tried to find the appropriate configuration to be compliant with the requirements. I described it in the attached document.
The different Service Groups behaviors are described also. Now I need to elaborate the required intelligence to be compliant with the requirements.
I think I will have to elaborate the intelligence in a "preonline" trigger, set as Global or Local, which will be invoked in case of manual group online, group autostart, or group failover.
If someone has some other good ideas, any good idea, I would appreciate a lot.
Thanks
Rgds
10-17-2019 01:27 AM
Hi Hubert,
Your requirements has 2 aspects that need to be addressed;
I would focus on point 1 first, and then worry about point 2.
10-17-2019 06:23 AM
Hi,
Thank for your contribution.
The solution could be something like that:
Two SGs dedicated to the Billing & Rating VIP, will be created:
- billing_VIP_sg,
- rating_VIP_sg.
These 2 SGs will contain only the VIP resource associated to each application; they will have the “AutoStart” property set to 1. These two SG will contain a “postonline” trigger in charge of starting their respective Application SG listed below, via the "hagrp -online <xxxxx_APP_SG -sys node" command :
- billing_APP_sg,
- rating_APP_sg.
The Application Service Groups will embed a “preonline” trigger containing the intelligence, able to manage the start method, according to the Use Cases defined previously. The Application SGs will have the “AutoStart” property set to 0.
is there some incompatibilities between the triggers "postonline" and "preonline" ?
do they trigger both ?
your remarks are welcome regarding this approach, to be able to be compliant with the initial problem.
Thanks for your cooperation, your advice.
Rgds
hubert
10-17-2019 07:35 AM
Hi Hubert
I've spent quite a while checking the available options but the biggest problem is offlining the parent (upper level) SG when there has been no error in the child SG. Neither VCS nor VIOM can do that so the script has to.
I think this logic is required.
It's a little complicated :)
10-18-2019 10:50 AM
Hi,
Does it exist a way to know that VCS has just been started via "hastart" and the different group in "Autostart" mode just been started, versus a way to know that a SG (or multiple) has just Switch-over or Fail-over ?
thanks for your cooperation
Rgds
H.
10-18-2019 11:42 AM
You could use hagrp -state -sys "system name" to check if the system is faulted or not. It's not so much about hastart or autostart, but rather whether the SG can run where it should, or if there has been a failure, and only one node is available.
11-21-2019 12:21 PM
Hi,
I am still working on this topics , but not everyday...
I am using the triggers available and add some perl code inside.
Is there somewhere some guide which explains how to write the code even if i already wrote some, or some examples, even complex one, which shows some tips and tricks ...
For example within a "resstatechange" trigger, I would like to know if there are some capabilities to enrich variables with the result of a cluster command like the following one or with basic system commands :
my $groupState = `hagrp -state groupname -sys node1`
The previous command doesnt work at all but i need such command .. how can I do that ?
Thanks for your help.
rgds
Hubert
11-22-2019 10:39 AM
hi,
Does anybody knows how to call a script (shell script) from a Veritas Cluster scripts ?
Is it like that :
VCSAG_SYSTEM("/opt/VRTSvcs/bin/triggers/billingAPPsg/ResStateChange.sh $ARGV[0] $ARGV[1] $ARGV[2] $ARGV[3]");
or like that :
SYSTEM("/opt/VRTSvcs/bin/triggers/billingAPPsg/ResStateChange.sh $ARGV[0] $ARGV[1] $ARGV[2] $ARGV[3]");
I read it was as the second method , starting with "system" but no way to trace what is happening inside the shell script.
I am looking for complex examples, to see what is it possible to do inside.
thanks for you cooperation and help.
Rgds
hubert