cancel
Showing results for 
Search instead for 
Did you mean: 

Application restart needs database restart also

sanderfiers
Level 3
Partner Accredited

Hi

Can I create an extra dependency between this two main Resources I have?

I have a Service Group with two main Agent Resources, next to the usual Resources (storage, networking, ...) :

  • an Application Agent Resource Type for my application
  • a Database Agent Resource Type for my database
  1. The first one, is for a custom application. I am using a StartProgram Attribute to start it, a StopProgram Attribute to stop it. For monitoring the application, I am using only the MonitorProgram Attribute (so no PIDFiles Attribute and no MonitorProcess Attribute). Which will check for a process from the systems process list output and returns the correct VCS exit code. The output displayed by the PS command does not match exactly, hence why MonitorProcess is not used but MonitorProgram.
  2. The second one is an official VCS Agent from a database manufacturer.

Now this is all working very fine. However the application does not work functional correclty when only the application restarts. It needs a fresh restart of the database also. How can I tackle this issue incase of a failure of my application Resource (wich has happened before)? Because a restart of the database can only happen when the application really restarts. Not in case of an online or offline of my complete Service Group.

Thanks in avance for you help,

Sander

1 ACCEPTED SOLUTION

Accepted Solutions

sanderfiers
Level 3
Partner Accredited

Hi Gaurav,

Thanks for taking the time to reply.

Yes, my first reaction to my customers request "to always restart the database prior to restarting the application, when the application crashes",
was to implement a CleanProgram for the application Resource, that would next to killing the actual application, would also (first nice,then forced) restart the database. The application Resource didn't have a CleanProgram before, so that was a good idea anyways.
And that was indeed the reason for my other thread regarding the CleanProgram and especially regarding its order.
My only fear was that HAD, via the monitoring of the database Agent, would get notified that the database Resource is offline.
During testing I found out that is was so fast that HAD never notices the database being offline. But maybe, this is because it's part of a CleanProgram?
I don't really know, would be nice to know though.
However, only a database-restart inside this CleanProgram does not always fullfill the requirements... so I was stumbled upon the idea of - like you call it - splitting the action.

I found out via my customer that when the application crashes, most of the time, the process itself is gone (so no PID to kill).
Hence why restarting the database via the CleanProgram of my application Resource is not always a solution.
But, since the application can never functionally work without a fresh database restart, the database-restart can safely stay inside the CleanProgram of the application Resource.

EDIT:
Actually the database-restart is not needed at all in the CleanProgram of the application Resource, right?

Because the StartProgram is always executed after a CleanProgram, the database will always be restarted via the StartProgram.
Even worse, the CleanProgram will do an unncessary database-restart.


Just incase the application is crashed but still has a PID, the CleanProgram will succesfully restart the database and next the application.

To handle the issue when the appliation has crashed and no PID is available, I would modify the StartProgram of the application Resource.
The StartProgram of the application Resource will always restart the database prior to starting the actual application.
The only thing to verify is the duration of this all (especially when keeping the monitoring of the database Resource in mind).
A bad consequence is that the database will be restarted unncessary during a normal online of the complete Service Group when starting or during failover.

 

 

 

View solution in original post

3 REPLIES 3

sanderfiers
Level 3
Partner Accredited

Hi,

I was thinking about a double restart of my database. Technically it should work. It's not really 'done' to restart something twice, but if it works? What do you guys think?

My suggestion:
Modify the StartProgram of my Application Resource to execute first a restart of the database and second a start of the application.

This way it should handle all possible scenario's:

  • When my application crashes, the StartProgram will always first restart my database and second start the application.
  • When failingover, the database will be started by my Database Resource and then the Application Resource will restart the database and start the application.

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hello,

I agree to your approach ..

Either

1. You can make a database restart as part of your Application StartProgram so that DB is restarted before app does

2. As per your other thread, as part of cleanscript for your app resource, if your app consumes restartlimit & clean script is called, it takes action to shutdown the DB & restarts it. OR you can split the action, i.e. shutdown the DB as part of app cleanscript & start the DB again using startprogram of app resource.

 

G

sanderfiers
Level 3
Partner Accredited

Hi Gaurav,

Thanks for taking the time to reply.

Yes, my first reaction to my customers request "to always restart the database prior to restarting the application, when the application crashes",
was to implement a CleanProgram for the application Resource, that would next to killing the actual application, would also (first nice,then forced) restart the database. The application Resource didn't have a CleanProgram before, so that was a good idea anyways.
And that was indeed the reason for my other thread regarding the CleanProgram and especially regarding its order.
My only fear was that HAD, via the monitoring of the database Agent, would get notified that the database Resource is offline.
During testing I found out that is was so fast that HAD never notices the database being offline. But maybe, this is because it's part of a CleanProgram?
I don't really know, would be nice to know though.
However, only a database-restart inside this CleanProgram does not always fullfill the requirements... so I was stumbled upon the idea of - like you call it - splitting the action.

I found out via my customer that when the application crashes, most of the time, the process itself is gone (so no PID to kill).
Hence why restarting the database via the CleanProgram of my application Resource is not always a solution.
But, since the application can never functionally work without a fresh database restart, the database-restart can safely stay inside the CleanProgram of the application Resource.

EDIT:
Actually the database-restart is not needed at all in the CleanProgram of the application Resource, right?

Because the StartProgram is always executed after a CleanProgram, the database will always be restarted via the StartProgram.
Even worse, the CleanProgram will do an unncessary database-restart.


Just incase the application is crashed but still has a PID, the CleanProgram will succesfully restart the database and next the application.

To handle the issue when the appliation has crashed and no PID is available, I would modify the StartProgram of the application Resource.
The StartProgram of the application Resource will always restart the database prior to starting the actual application.
The only thing to verify is the duration of this all (especially when keeping the monitoring of the database Resource in mind).
A bad consequence is that the database will be restarted unncessary during a normal online of the complete Service Group when starting or during failover.