Forum Discussion

Zahid_Haseeb's avatar
Zahid_Haseeb
Moderator
13 years ago

Some questions Regarding Heartbeat

Kindly correct if I am wrong   1.) What happen when both Heartbeat link fails ? Split Brain situation.   2.) What we do to getrid Split Brain situation ? Add a Low Priority Heartbeat but ...
business continuity
Cluster Server
  • mikebounds's avatar
    mikebounds
    13 years ago
    1. If all 2 heartbeats fails at the same time you will get split-brain
       
    2. To help prevent split-brain you can :
      a. Add more than 2 heartbeats (high-pri or low-pri) - these should be completely independent - different NIC cards going to different switches
      b. Use I/O and/or Server (CPS) Fencing
      c. Make diskgroup dependent on Application IP in service group, so IP comes up first.  This means, even if you don't configure virtual App IP in VCS as a low-pri HB, if both heartbeats fails and App network is OK, then when SG tries to online it will fail when it tries to online IP and hence will prevent split-brain (prevents importing diskgroup and corrupting storage)  This is also protects against human errror if you force stop cluster and then start VCS up on inactive node
       
    3. When there is only one heartbeat left, if Node fails, then service groups will not fail over, but if a resource faults and node stays up, then service group can still fail over
       
    4. (and 5) If you have ANY 2 heartbeats remaining (including low-pri), then node failure results in service group failure as usual.  i.e failure of heartbeats when there are still 2 left has no effect on failover of service groups.

    Mike

mikebounds's avatar
mikebounds
Level 6
13 years ago
  1. If all 2 heartbeats fails at the same time you will get split-brain
     
  2. To help prevent split-brain you can :
    a. Add more than 2 heartbeats (high-pri or low-pri) - these should be completely independent - different NIC cards going to different switches
    b. Use I/O and/or Server (CPS) Fencing
    c. Make diskgroup dependent on Application IP in service group, so IP comes up first.  This means, even if you don't configure virtual App IP in VCS as a low-pri HB, if both heartbeats fails and App network is OK, then when SG tries to online it will fail when it tries to online IP and hence will prevent split-brain (prevents importing diskgroup and corrupting storage)  This is also protects against human errror if you force stop cluster and then start VCS up on inactive node
     
  3. When there is only one heartbeat left, if Node fails, then service groups will not fail over, but if a resource faults and node stays up, then service group can still fail over
     
  4. (and 5) If you have ANY 2 heartbeats remaining (including low-pri), then node failure results in service group failure as usual.  i.e failure of heartbeats when there are still 2 left has no effect on failover of service groups.

Mike