Heartbeat via SAN

[bsobek]

Hi,

does anybody has experiences with heartbeats via the SAN? We want to test this configuration. We use solaris with Emulex-HBA's. It should be possible to tunnel LLT over UDP. 

Thanks!

Greets

Bjoern

[Eric_Hennessey1]

I'm not sure why you'd want to do this...just because maybe you can, doesn't mean you SHOULD.  ;)

But if your hardware allows you to enable an IP interface on the HBA, you should be able to enable LLT over UDP on that IP interface.  But I would check with support for a support statement on that configuration before proceeding with it.

[TempoVisitor]

I had this tried under Solaris, with Emulex HBA, on VCS 4.1

Sending GAB/LLT packets over lpfc0 just configured as an ethernet NIC.

I just had the /etc/llttab modified to point on lpfc:0 instead of eri:0

Not even UDP.

The same fibre was used for accessing data disks on the array, but with a brocade switch which all nodes were connected to.

That works .... but i didnt push that on production, nor analysed the traffic between data and VCS packets.

However it seems that these GAB/LLT packets could cause problem with other hardware on the san, like Tape libraries libs. That's waht I had been told.

I did all the test about jeopardy, low priority HB, and I/O fencing. It was fine.

Cheers.

[dguennel]

I'm also in strong interest about hearbeat over SAN, because there is not gabdisk support since version 5.

i like to describe why an alternate Hearbeat way is so importend:
if you think about a backup datacenter ( only as a Datacenter without employes ) and one Clusternode should stand in that datacenter.
You plan your cluster with two heartbeat IF over seperate Lines to the backup datacenter.
So far ... everything perfect.

You decide to use both clusternodes for applications / or Databases, just let both nodes working...
And you use I/O Fencing just to take care against split brain... ( i guess that config looks fair ).

But what happens when a big networkfailure stops the whole transfer to ur backup datacenter ????
One of ur cluster nodes crashes ... maybe that one that is in ur primary Datacenter and all services falls to the backup datacenter.

... ok u run into a big problem .. because u have no network to the backup datacenter,, so how u can transfer the services to ur primary datacenter ??..

Before Version 5 we use the on to the gabdisk to prepare us against networkfailure and that works perfect.

Why !!!!!!  is that feature not more available ???????..

It 's not more possible to build up HA Solutions in that limited way,, so im also searching an alternate way for
Heartbeating between the cluster nodes ,, or i have to search an other cluster SW.
( i dont like... )


Is there really no way to get back the gabdisk feature ??

So an alternate way will be ( maybe ) IP over FC ,, just to use other lines... not perfect but better then nothing..

Is there a roadmap to make this as a supported way ??..

Need ur help ....

Thanks a lot!

Best regards
Dominik

[Roger_Zimmerman]

Hi,

I see all this things very relaxed. Of course you can use FC for LLT, but also with FC as carrier you use DLPI for the LLT packages. And if your DLPI driver is crashing (ok, it is a very old and well testet driver, so it is unlikely that it fails without very good reasons) you also loose the LLT connection via FC.

On the other hand, gabdisk never was a really good solution. It shrinked the cluster size, was unstable in many ways and if you had a major desaster in your communication (as you mentioned in your posting, loosing all connections to your other data center) you could reach a split brain! So, I am happy to blow this thing away. And the substitution is fencing!  Fencing is 100% sure if you have only the sligthest chance for split brain! And here again you have a protocol copletely beside network and DLPI! And in addition you switch on the resevation button in disk groups for data protection ...

When I startet to implement clusters back in 1992 I learned: data consistancy is more important then availability. And I learned one thing what would solve all the problems in loosing all communication links in the same time: redundancy. Discrete physical ways, discrete logical ways. And a network connection to each datacenter on different ways from different providers. And then maybe there is a lopri link (ouch, sorry ... :-))

So, in fact I'm very fine with LLT (on whatever carrier) and Fencing. It's completely enough and it is split brain proof.

Happy clustering ...

@kerkael

You should take care for that your SAN is correctly zoned, then you have no interference between HBAs with IP switched on and your FCP (Fibre Channel Embedded SCSI) devices. You should have separate zones for your SCSI traffic and one dedicated zone which contains only the two HBAs that needs to communicate with each other. One HBA can be member in more than one SAN zone, so this is easy. Especialy you should prevent zones with one tape library and more than one HBA, this cries for trouble!

Happy zoning ...

Message Edited by Roger Zimmermann on 11-20-2007 06:12 AM