Sun v440 CE interface VCS problems

[Vic_Engle]

I'm building a 3 node cluster from 2 Sun 280r's and a Sun V440, each with 2 100Mb interconnects and each node connected via FC to shared storage. When I have the 3 nodes generate IO to the shared storage I end up with one node fenced out or otherwise faulted and I see messages suggesting the problem is interconnect latency during the concurrent IO load.

I've been scanning the docs looking for guidance on interconnect speed and can't seem to find what I'm looking for. Can someone point me to documentation that spells out interconnect requirements for heavy concurrent IO from 3 or more nodes?

Also, it seems to always be the 440 that gets fenced out during heavy concurrent I/O and I am beginning to suspect the "ce" interfaces used in the 440 for the interconnect might be the problem. Has anyone encountered this before.

The OS is Solaris 10 Update 3.

Thanks,
Vic Engle

[Gene_Henriksen]

The interconnect speeds for LLT/GAB are more than adequate at 100Mbs. The question is are you getting any messages about "llt not called for xxx ticks" which would indicate the cpu is too busy to handle LLT communications.

You mentioned "fenced" so I assume you are running IO Fencing, is this true?