MSMQ Outgoing Queues referencing failed EV server - using Building Blocks for High Availability
Hi Everyone We make use of Building Blocks to build High Availability in our EV environment. We have come across a peculiar problem and wanted to know if anyone here has noted a similar issue. Scenario: We are in the archiving schedule and there are (say) 5 EV Mailbox Archiving Servers archiving 5 Exchange Servers. There are cross-references across EV and Exchange Servers i.e. the Storage Service for an archive is on an EV server other than the one which is running the Archival Task. Problem: If an EV Mailbox Archiving Servers fails, we use Building Blocks to quickly bring up services on a standby server by running Update Service Locations. However, we notice that items are still in archive pending state in user mailboxes being archived by other EV Servers. New items are also going into archive pending state across all user mailboxes. Analysis: We have already run a flushdns (or Clear-DNSClientCache in Powershell) across all EV servers. In the MSMQ Outgoing Queues, we noticed that there are queues which still resolve to the old failed servers and are in Failed to Connect or Waiting to Connect state, since MSMQ does not resolve to the EV aliases but the EV server hostnames. Solution: To resolve the issue, we need to add a host file entry resolving the hostname of the failed server to the IP address of the now active server and restart the storage/taskcontroller service. Has anyone of you come across a similar situation? We wish to avoid making host file entries and make the failover process seamless.
I want to switch the application to minos2，but I cannot select the system
I want to switch the application to minos2，but I cannot select the system No system，why？ root@minos1(/)# hastatus -sum -- SYSTEM STATE -- System State Frozen A minos1 RUNNING 0 -- GROUP STATE -- Group System Probed AutoDisabled State B ClusterService minos1 Y N ONLINE B EMS minos1 Y N ONLINE B replication minos1 Y N ONLINE -- REMOTE CLUSTER STATE -- Cluster State N minos2 RUNNING -- REMOTE SYSTEM STATE -- cluster:system State Frozen O minos2:minos2 RUNNING 0
setting llt (low prio net) in a bond configured in a bridge
Hi, I want to set a low prio network over a bond which is part of a bridge. I have a cluster with two nodes. This is the network configuration (it is the same for both nodes): node 2: [root@node2 ]# cat /proc/version Linux version 2.6.32-504.el6.x86_64 (mockbuild@x86-023.build.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Tue Sep 16 01:56:35 EDT 2014 [root@node2 ]# ifconfig | head -n 24 bond0 Link encap:Ethernet HWaddr 52:54:00:14:13:21 inet6 addr: fe80::5054:ff:fe14:1321/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:761626 errors:0 dropped:0 overruns:0 frame:0 TX packets:605968 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1188025449 (1.1 GiB) TX bytes:582093867 (555.1 MiB) br0 Link encap:Ethernet HWaddr 52:54:00:14:13:21 inet addr:10.10.11.102 Bcast:10.10.11.255 Mask:255.255.255.0 inet6 addr: fe80::5054:ff:fe14:1321/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:49678 errors:0 dropped:0 overruns:0 frame:0 TX packets:50264 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:44061727 (42.0 MiB) TX bytes:28800387 (27.4 MiB) eth0 Link encap:Ethernet HWaddr 52:54:00:14:13:21 UP BROADCAST RUNNING PROMISC SLAVE MULTICAST MTU:1500 Metric:1 RX packets:761626 errors:0 dropped:0 overruns:0 frame:0 TX packets:605968 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1188025449 (1.1 GiB) TX bytes:582093867 (555.1 MiB) [root@node2 ]# brctl show bridge name bridge id STP enabled interfaces br0 8000.525400141321 no bond0 [root@node2 ]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: eth0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth0 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: 52:54:00:14:13:21 Slave queue ID: 0 node 1: [root@node1]# ifconfig | head -n 24 bond0 Link encap:Ethernet HWaddr 52:54:00:2E:6D:23 inet6 addr: fe80::5054:ff:fe2e:6d23/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:816219 errors:0 dropped:0 overruns:0 frame:0 TX packets:668207 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1194971130 (1.1 GiB) TX bytes:607831273 (579.6 MiB) br0 Link encap:Ethernet HWaddr 52:54:00:2E:6D:23 inet addr:10.10.11.101 Bcast:10.10.11.255 Mask:255.255.255.0 inet6 addr: fe80::5054:ff:fe2e:6d23/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:813068 errors:0 dropped:0 overruns:0 frame:0 TX packets:640374 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1181039350 (1.0 GiB) TX bytes:604216197 (576.2 MiB) eth0 Link encap:Ethernet HWaddr 52:54:00:2E:6D:23 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:816219 errors:0 dropped:0 overruns:0 frame:0 TX packets:668207 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1194971130 (1.1 GiB) TX bytes:607831273 (579.6 MiB) [root@node1]# brctl show bridge name bridge id STP enabled interfaces br0 8000.5254002e6d23 no bond0 [root@node1 ]# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: eth0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth0 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: 52:54:00:2e:6d:23 Slave queue ID: 0 the llt configuration files are the following: [root@node1 ]# cat /etc/llttab set-node node2 set-cluster 1042 link eth3 eth3-52:54:00:c3:a0:55 - ether - - link eth2 eth2-52:54:00:35:f6:a5 - ether - - link-lowpri bond0 bond0 - ether - - [root@node1 ]# cat /etc/llttab set-node node1 set-cluster 1042 link eth3 eth3-52:54:00:bc:9b:e5 - ether - - link eth2 eth2-52:54:00:31:fb:31 - ether - - link-lowpri bond0 bond0 - ether - - However this seems that is not working. When I check the llt status, the node thinks the interface is down in the other one. [root@node2 ]# lltstat -nvv | head LLT node information: Node State Link Status Address 0 node1 OPEN eth3 UP 52:54:00:BC:9B:E5 eth2 UP 52:54:00:31:FB:31 bond0 DOWN * 1 node2 OPEN eth3 UP 52:54:00:C3:A0:55 eth2 UP 52:54:00:35:F6:A5 bond0 UP 52:54:00:14:13:21 [root@node2 ]# lltstat -nvv | head LLT node information: Node State Link Status Address * 0 node1 OPEN eth3 UP 52:54:00:BC:9B:E5 eth2 UP 52:54:00:31:FB:31 bond0 UP 52:54:00:2E:6D:23 1 node2 OPEN eth3 UP 52:54:00:C3:A0:55 eth2 UP 52:54:00:35:F6:A5 bond0 DOWN Do you know if I have something worng? Is this a valid configuration? Thanks, Javier
deleting rlink that's having "secondary_config_err" flag
hello, in my VCS global cluster my ORAGrp resource group is partially online since my rvgres is offline, i am suspecting the issue in the below rlink. i am trying to dissociate thi rlink(below:rlk_sys1-DB-rep_DB_r) and dettach it in order to delete it but i am not able to succeed. below are some output from the system. root@sys2# vxprint -P Disk group: DBhrDG TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 rl rlk_sys1-DB-rep_DB_r DB_rvg CONNECT - - ACTIVE - - rl rlk_sys1-rep_DB-rvg DB-rvg ENABLED - - PAUSE - - root@sys2# vxrlink -g DBhrDG dis rlk_sys1-rep_DB-rvg VxVM VVR vxrlink ERROR V-5-1-3520 Rlink rlk_sys1-rep_DB-rvg can not be dissociated if it is attached root@sys2# vxrlink -g DBhrDG det rlk_sys1-rep_DB-rvg VxVM VVR vxrlink ERROR V-5-1-10128 Operation not allowed with attached rlinks root@sys2# vxedit -g DBhrDG rm rlk_sys1-rep_DB-rvg VxVM vxedit ERROR V-5-1-3540 Rlink rlk_sys1-rep_DB-rvg is not disabled, use -f flag root@sys2# vxedit -g DBhrDG -f rm rlk_sys1-rep_DB-rvg VxVM vxedit ERROR V-5-1-3541 Rlink rlk_sys1-rep_DB-rvg is not dissociated root@sys2# vxprint -Vl Disk group: DBhrDG Rvg: DB-rvg info: rid=0.1317 version=0 rvg_version=41 last_tag=11 state: state=CLEAN kernel=DISABLED assoc: datavols=(none) srl=(none) rlinks=rlk_sys1-rep_DB-rvg exports=(none) vsets=(none) att: rlinks=rlk_sys1-rep_DB-rvg flags: closed secondary disabled detached passthru logging device: minor=26012 bdev=343/26012 cdev=343/26012 path=/dev/vx/dsk/DBhrDG/DB-rvg perms: user=root group=root mode=0600 Rvg: DB_rvg info: rid=0.1386 version=13 rvg_version=41 last_tag=12 state: state=ACTIVE kernel=ENABLED assoc: datavols=sys1_DB_Process,sys1_DB_Script,... srl=sys1_DB_SRL rlinks=rlk_sys1-DB-rep_DB_r exports=(none) vsets=(none) att: rlinks=rlk_sys1-DB-rep_DB_r flags: closed secondary enabled attached logging device: minor=26014 bdev=343/26014 cdev=343/26014 path=/dev/vx/dsk/DBhrDG/DB_rvg perms: user=root group=root mode=0600 please advise regards