Bonded interface or multiple private networks for heartbeat?

[szhargrave]

Hi

We're configuring Cluster Filesystem and I'm trying to understand from a best practice point of view the best way to configure heartbeats.  The servers are all within a HP BladeSystem enclosure with Virtual Connect, so any heartbeat networks are essentially inter-enclosure virtual networks.  Each blade has 4 NICs in total.  The way I see it we have 2 options: -

Option 1) Configure 2 NICs in a bonded interface (bond1) and configure this is the private heartbeat, with a low-priority hearbeat on the public bond (bond).

Option 2) Configure 2 hearbeat nics, not bonded (eth2 and eth3) as 2 separate private heartbeats, with a low-priority hearbeat on the public bond (bond).

There's no physical difference between the 2, they'll still use the Blade enclosure backplane, so just wondered what is perceived to be the best option from best-practice perspective, and why?

Thanks, Simon

[mikebounds · Accepted solution]

Option 2 is the best if you can ensure all 3 heartbeats links are independent (go through separate switches).

This option is better as you have 3 heartbeats rather than 2.  With option 1, if you loose the public n/w, from a VCS perspective you have 1 link remaining (even though this is 2 physical links) and therefore the cluster will go into Jeopardy. 

Jeopardy means if you loose a node, then other node can not distinuish this between lost of the last HB and as VCS does not know if node is down or last hb went down, it will not take any action.  With option 2,  if you loose the public n/w, from a VCS perspective you have 2 links remaining and therefore if you loose a node, then as the other node sees 2 heartbeats disappear at the same time and the HBs are independent it assumes the node went down and takes action.

So basically, by not configuring heartbeats in a bond, you give VCS more information about what physical NICs are working.

Mike

[szhargrave]

Thanks Mike, that makes perfect sense.