cancel
Showing results for 
Search instead for 
Did you mean: 

Configure LLT And GAB In A Veritas Cluster

joaotelles
Level 4

Hi,

Im about to add a new node to a cluster (Solaris OS and version 5.0 for the cluster)  and I have set all the cabling to have the HB network working. The heartbeat links are connected correctly using a switch over the same VLAN.

I was checking if the HB links are up but not plumed, but they are in this state unknown.

root# dladm show-dev
igb0            link: up        speed: 1000  Mbps       duplex: full
igb1            link: unknown   speed: 0     Mbps       duplex: unknown
igb2            link: unknown   speed: 0     Mbps       duplex: unknown
igb3            link: unknown   speed: 0     Mbps       duplex: unknown
usbecm0         link: up        speed: 10    Mbps       duplex: full
 

Its the interfaces igb1 and igb2.

Im new at this and comparing to the other nodes and internet examples, their HB interfaces are up but NOT plumbed...

So, these interfaces - igb1 and igb2.- should be up just by plugging the cable or should I issue a command to do so?

Is there any checnking I can do to see where the problem is, if there is any?

Tks in advance,

Joao

 

 

2 ACCEPTED SOLUTIONS

Accepted Solutions

Gaurav_S
Moderator
Moderator
   VIP    Certified

Sorry, couldn't see previous comments for some reason ... pinging on self machine is not gonna make much sense ...

Because heartbeat is already up & running in the cluster & if cluster is a production cluster, I wouldn't recommend to try plumbing an IP on heartbeat.

Your ultimate task is to add the node to cluster, so anyways you would need to install VCS packages. You can install VCS packages (download from symantec portal) with that, you will get VRTSllt package.

with VRTSllt installed & all the cluster config files (llttab, llthosts,gabtab) populated, you can then run the dlpiping or lltping tests as explained in first step

 

G

View solution in original post

mikebounds
Level 6
Partner Accredited

If you can't assign IPs to other node as it is live, then you cant test using IPs.  There shouldn't be an issue assigning IPs to a live node from a technical point of view, but running any command on a live server can be risky if you make a mistake (hence why most live systems will have change control)

Mike

View solution in original post

20 REPLIES 20

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hi Joao,

LLT status up is good & LLT interfaces will not be plumbed...

you can use dlpiping command or lltping or command to view the llt status ...

see this article for using dlpiping

http://www.symantec.com/docs/TECH19998

see below doc for using lltping

http://sfdoccentral.symantec.com/sf/5.0/solaris/manpages/vcs/lltping_1m.html

 

G

 

 

joaotelles
Level 4

Tks for the fast reply.

I havent installed it yet so I dont have the veritas packages installed yet to run either of the commands you sent...

I think I will have to install it and then test right?

Is there any command like dladm to test it? I mean a unix command...

 

Tks,

Joao

 

 

mikebounds
Level 6
Partner Accredited

To test heartbeats, I ususally just temporarily plumb them in with test addresses and use ping to test  - so something like:

on node1 give IPs

1.1.1.1

1.1.2.1

and on node 2

1.1.1.2

1.1.2.2

all with netmask 255.255.0.0.

And if ping doesn't work then you know it is not cabled correctly or there is an issue with VLAN where as if you use the unfamilar dlpiping tool you won't be sure if you are using tool correctly.

Mike

joaotelles
Level 4

Tks Mike.

Sorry for the newbie question...so I will plumb the interfaces in the new node with dummy IPs and try to ping them from the other node... is this what you suggested?

Btw, the interface be up and not plumbed for the Veritas installation, right? And this up of the link should be automaticallly when you plug the cable or a command is necessary?

 

Tks,

Joao

 

 

mikebounds
Level 6
Partner Accredited

Yes, use dummy IPs to test links, but when you install VCS you don't need IPs - it will still work with IPs, but they should be removed to prevent the network being used by anything else.

The link being UP means something is plugged in the NIC, but it is not necessarily plugged into the right place.

Mike

joaotelles
Level 4

Hi Mike,

Tks for the help so far.

I have done what you said and the ping failed.

root # ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 10.129.68.47 netmask ffffffc0 broadcast 10.129.68.63
        ether 0:10:e0:3e:c3:1c
igb1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
        inet 1.1.1.1 netmask ffff0000 broadcast 1.255.255.255
        ether 0:10:e0:3e:c3:1d
igb2: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
        inet 1.1.1.2 netmask ffff0000 broadcast 1.255.255.255
        ether 0:10:e0:3e:c3:1e

usbecm0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
        inet 169.254.182.77 netmask ffffff00 broadcast 169.254.255.255
        ether 2:21:28:57:47:17

root # dladm show-dev
igb0            link: up        speed: 1000  Mbps       duplex: full
igb1            link: up        speed: 1000  Mbps       duplex: full
igb2            link: up        speed: 1000  Mbps       duplex: full

igb3            link: unknown   speed: 0     Mbps       duplex: unknown
usbecm0         link: up        speed: 10    Mbps       duplex: full

In bold the interfaces that were unknokn staete before the plubing and these are the ones to be used for the HB.

Have I missed anything?

Used these commands:

root@# ifconfig igb2 plumb
root@ # ifconfig igb2 1.1.1.2
root@ # ifconfig igb2 netmask 255.255.0.0
 

The ping from the other node failed to the 1.1.1.1 or 1.1.1.2

 

Tks in advance ,

Joao

 

 

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hi Joao,

you should give 1.1.1.1 on first node on igb1 & 1.1.1.2 on second node on igb1 ..

similarly 1.1.2.1 on igb2 on first node & 1.1.2.2 on second node on igb2

 

at this point 1.1.1.1 & 1.1.1.2 are assigned to same server which should ideally ping (on same server) however try above to confirm if connectivity is correct...

 

G

joaotelles
Level 4

Will this test work if the nodes are connected through using a switch over a VLAN?

joaotelles
Level 4

I cant do this test because the other node the interface has a different name and the cluster is currently running there. So, I cant assign an IP to the interface of the other node and test this...

 

I can ping on the own machine but this doesnt prove much...

joaotelles
Level 4

If I umplumb it it gets to the unkknown state again.

root@dp-node9 # ifconfig igb1 unplumb
root@dp-node9 # netstat -in
Name  Mtu  Net/Dest      Address        Ipkts  Ierrs Opkts  Oerrs Collis Queue
lo0   8232 127.0.0.0     127.0.0.1      388    0     388    0     0      0
igb0  1500 10.129.68.0   10.129.68.47   15569708 0     7256551 0     0      0
igb2  1500 1.1.0.0       1.1.1.2        12943  0     24     0     0      0
usbecm0 1500 169.254.182.0 169.254.182.77 6      0     1126   0     0      0

root@dp-node9 #  dladm show-dev
igb0            link: up        speed: 1000  Mbps       duplex: full
igb1            link: unknown   speed: 1000  Mbps       duplex: full
igb2            link: up        speed: 1000  Mbps       duplex: full
igb3            link: unknown   speed: 0     Mbps       duplex: unknown
usbecm0         link: up        speed: 10    Mbps       duplex: full
 

Is this expected?

 

 

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hi,

yes, ideally it should work .. because you are configuring the IP address of same subnet to communicate over VLAN ... 

if in case above test is not working,

1. recheck connectivity, try connecting via cross cables if possible to eliminate the problem with switch.

2. If cross cable is not available, check the switch logs to confirm if packets are getting dropped.

 

G

Gaurav_S
Moderator
Moderator
   VIP    Certified

Sorry, couldn't see previous comments for some reason ... pinging on self machine is not gonna make much sense ...

Because heartbeat is already up & running in the cluster & if cluster is a production cluster, I wouldn't recommend to try plumbing an IP on heartbeat.

Your ultimate task is to add the node to cluster, so anyways you would need to install VCS packages. You can install VCS packages (download from symantec portal) with that, you will get VRTSllt package.

with VRTSllt installed & all the cluster config files (llttab, llthosts,gabtab) populated, you can then run the dlpiping or lltping tests as explained in first step

 

G

mikebounds
Level 6
Partner Accredited

You had 2 issues:

  1. You assigned 1.1.1.1 and 1.1.1.2 to first server, which are both in the same subnet rather than 1.1.1.1 and 1.1.2.1
  2. You didn't UP the interface so you should add "ifconfig igb2 up" or you can replace all commands by one command - "ifconfig igb2 plumb 1.1.2.1 netmask 255.255.0.0 up"

Mike

mikebounds
Level 6
Partner Accredited

If you can't assign IPs to other node as it is live, then you cant test using IPs.  There shouldn't be an issue assigning IPs to a live node from a technical point of view, but running any command on a live server can be risky if you make a mistake (hence why most live systems will have change control)

Mike

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I agree with Mike - there is no problem adding a IP address to private NICs, even if it is a live cluster.

Plumbing IPs on separate subnets is the quickest and best way to test comms.

joaotelles
Level 4

Tks Marianne. The problem is that I dont have the permission to do it :(

I will install the packages on the new node and use the llt tools and try to figure this out.

Let you know how it goes... Tks for the help so far.

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hi,

Was wondering if you've got a chance to test this ?

 

G

joaotelles
Level 4

Im having problems on getting a license for the new node... As soon as I install the SW in the new node and test I post here.

Tks,

Joao

joaotelles
Level 4

Hi,

 

I have done some testing here and after make all the configurations: new node is dp-e9

root@dp-e9 # cat /etc/llthosts
0 DP-e5
1 DP-e6
2 DP-e4
3 DP-e8
4 dp-e9


root@dp-e9 # cat /etc/gabtab
/sbin/gabconfig -c -n5

root@dp-e9 # cat /etc/llttab
set-node dp-e9
set-cluster 10000
link igb1 /dev/igb1 - ether - -
link igb2 /dev/igb2 - ether - -
 

root@dp-node9 # dladm show-dev
igb0            link: up        speed: 1000  Mbps       duplex: full
igb1            link: up        speed: 1000  Mbps       duplex: full
igb2            link: up        speed: 1000  Mbps       duplex: full
igb3            link: unknown   speed: 0     Mbps       duplex: unknown
usbecm0         link: up        speed: 10    Mbps       duplex: full
 

In the other node I have:

root@DP-e6 # /sbin/gabconfig -a
GAB Port Memberships
================================
Port a gen   406917 membership 01234
Port b gen   406915 membership 0123
Port b gen   406915    visible ;   4
Port h gen   406914 membership 0123
Port h gen   406914    visible ;   4
 

Looks like everyhthing is fine...

 

Tks for the help.