VCS 5.0 NIC replaces

[enzo68]

Hi to all,

we have a two nodes veritas cluster server 5.0 on REDHAT 5.5; below the output of command "hastatus -sum"

-- SYSTEM STATE

-- System               State                Frozen              

A  dbcccrm1             RUNNING              0                    

A  dbcccrm2             RUNNING              0                    

-- GROUP STATE

-- Group           System               Probed     AutoDisabled    State          

B  Oracle_GENCFG   dbcccrm1             Y          N               ONLINE         

B  Oracle_GENCFG   dbcccrm2             Y          N               OFFLINE        

B  Oracle_GENOUT   dbcccrm1             Y          N               OFFLINE        

B  Oracle_GENOUT   dbcccrm2             Y          N               ONLINE         

B  networks        dbcccrm1             Y          N               ONLINE         

B  networks        dbcccrm2             Y          N               ONLINE    

About the cluster network configuration we have configured two heartbeat nic and one bond lan data ( it's include two eth ).

So, we have 4 eth for each server ( two heartbeat and one bond data ).

We must change all NIC on both the servers because they are afflicted by bugs.

We have two possible scenarios ( really we don't know what of them at the moment :(

1) install the four new NIC without take off the current NICs.

2) replace the current NICs with the new ones.

I need ( hope! ) a your suggestions on how to proceed in both the situations.

Best regards,

Vincenzo

[joseph_dangelo]

Vincenzo,

Let me see if I am understanding correctly,  your current NIC configuration is being effected by a bug and you are required to replace the physical devices themselves?

Can you post the output from the following:

/etc/llttab

#>lltstat -nvv

that is -n v v not w :o)

If you must, you can install new nics, add them as LLT links then remove the original LLT links and finally remove the original NICS.

Joe D

[Yasuhisa_Ishika]

I recommend you to replace the current NICs with the new ones because if once you configured newly added NICs while old NICS attached, you have to reconfigure ethX at removal of old NICs later. Instance number of eth will be changed after reboot. Make sure to edit /etc/llttab and change MAC addresses of heartbeat NIC in it.

[joseph_dangelo]

Excellent point.  Best bet is to shutdown the system and replace the NIC's in the same PCI slots. You should then retain the same Interface number.

[enzo68]

Hi Guys,

Thanks for your response.

>Let me see if I am understanding correctly,  
>your current NIC configuration is being effected by a bug and you are required to replace the physical devices themselves?

No exactly. Really the physical Nics are affected by bugs, no their configuration.


I have attached the output of "lltstat -nvv" and the llttab.

Below I try to better define as to proceed:


a) So, as you explain me if the nics are replaced I'll have to only  change the heartbeat's Mac in the llttab on both nodes. Is it enough ?


b) Instead, if the new NICs are added to servers ( it' s a possibility because the olds nics are "Integrated" Quad Port Multifunction Gigabit )

I think to proceed in this way:

Current Nics --> eth2, eth3 as heartbeat nic; eth0 and eth1 as bond0

1) add two new eth ( ethx and ethy for example) as heartbeat nic in the cluster ( so,  I'll have four heartbeat (eth2, eth3, ethx, ethy ) ).

2) add two new eth ( ethz and ethj for example) in the bond0 ( so, the bond0 will involve four eth ( eth0, eth1, ethz, ethj )).


Finally I'll remove the olds eth from heartbeat and from bond0, so, the cluster will be migrated on the new NIC.


What do you think about that ? Is it possible to add ( and hence remove ) in the cluster new heartbeat eth ? What is the vcs command to do that ?



Thanks in advance,

Vincenzo

[mikebounds · Accepted solution]

There is no command to change llt MAC addresses - you need to edit llttab file.  This file is only read when llt is configured (started), normally only at boot time.  To do this without rebooting:

Stop VCS (optioning with "-force" flag to leave applications runnings)

 hastop -all [-force]  (run on one node)

Stop GAB and LLT on BOTH nodes:

 gabconfig -U
lltconfig -U

If you haven't edited llttab file, then do this now, then start LLT and GAB on both nodes

 lltconfig -c
. /etc/gabtab  (this should run gabconfig -c - n 2)

Check output of "lltstat -nvv" to see new MAC addresses and that both links are UP

Start VCS

 hastart (on both nodes)

There should be nothing to change in VCS (main.cf) as its reference "bond0" is not changing.

Mike

[enzo68]

Hi Mike,

the your response resolve in details the procedure I'll have to do about the a) point ( Nics are replaced so the eth ids of news Nic stay the same of old ones ).


Remain for me to understand what I'll have to do when news Nic are added and olds ones ( with bugs ) aren't removed ( point (b of previous my entry describe what I thought to do with blue language ), so I would like to have your help about this facet.



Thanks a lot,
Vincenzo

[mikebounds]

I don't see any text in blue colour, but I think you are referring to can you add 2 NICs to bond when old 2 are still there.  The NIC agent in VCS will probably be not be aware you are adding and removing NICs to the bond.  If you look at Bundled Agents guide then this explains what the NIC agent does:

If the NIC maintains its connection status, the agent uses MII to

determine the status of the resource.

If the NIC does not maintain its connection status, the agent verifies

that the NIC is configured. The agent then sends a ping to all the

hosts that are listed in the NetworkHosts attribute. If the ping test is

successful, it marks the NIC resource ONLINE.

So if the connection state is maintanined when adding and removing NICs then all is ok, but as a precautionly measure you could freeze the all your service groups, or you could change bond while VCS is down, so if you follow llt change procedure in last email, change bond before running "hastart".

For heartbeats there is really no point leaving old NICs when you add new NICs as you will have to restart LLT to see new NICs, but if you did leave them there, then just add new NICs first in the llttab file as if a NIC is not present (as oppose to not working) when llt start then the LLT start procedure bails out and no LLT NICS would be started.  So if you do want all 4 NICs in LLT at some point for some reason then put new NICS above old NICs in llttab file and if then if llt were to start and old NICS had been removed then llt start procedure would bail out after starting new NICs so you would be ok

Mike