Failaover time more than 1 minute

it is take 12 second

Hi Solom,

It seems because of there are many disks and volumes in the second DG you added into VCS.

The time costs on importing DG depends on how many disks and volumes in the DG. Because with importing DG, all disks will be read and all volumes will be online(start) at the same time. So typically, when there are many disks and volumes, the import or online DG/DG resource will take long time to complete.

Another thing is that if you configured all volumes as volume resource in the "Volume" resource. If so, VCS volume agent will wait until all volume to start/stop when the service group online/offline to complate.

I hava one  disk  at each DG and 5 volumes mounts on the disk i think not many .

And when i force the node1 ore node2 the failover taking 10 second .

Regards

can you attach engine_A.log & main.cf for us ?

G

I attached .

Thanks

Hi, solom  

  You mean this log :

======

2014/04/22 16:10:32 VCS INFO V-16-1-50135 User admin fired command: hagrp -switch INT-PRI  TCPRI-CLU2  localclus  from ::ffff:10.100.208.76
2014/04/22 16:10:32 VCS NOTICE V-16-1-10208 Initiating switch of group INT-PRI from system TCPRI-CLU1 to system TCPRI-CLU2
2014/04/22 16:10:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:10:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:10:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:10:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:10:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:10:33 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:10:34 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:10:35 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:10:35 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:10:41 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:10:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:10:48 VCS INFO V-16-1-10305 Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:10:48 VCS NOTICE V-16-1-10446 Group INT-PRI is offline on system TCPRI-CLU1
2014/04/22 16:10:48 VCS NOTICE V-16-1-10301 Initiating Online of Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:10:48 VCS NOTICE V-16-10031-1513 (TCPRI-CLU2) DiskGroup:PRI-INT:online:Diskgroups will be imported with reservations.
2014/04/22 16:10:54 VCS WARNING V-16-10031-1509 (TCPRI-CLU2) DiskGroup:PRI-INT:online:vxdg import succeeded on Disk Group PRI-INT.
2014/04/22 16:10:54 VCS NOTICE V-16-10031-1559 (TCPRI-CLU2) DiskGroup:PRI-INT:online:Volumes in DiskGroup PRI-INT will be started automatically as part of import command,the system level autostartvolume is set On 
2014/04/22 16:10:55 VCS INFO V-16-1-10298 Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:10:55 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:10:55 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:10:55 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:10:55 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:10:55 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:10:58 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:10:59 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:11:00 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:11:01 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:11:02 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:11:02 VCS NOTICE V-16-1-10447 Group INT-PRI is online on system TCPRI-CLU2

==============

or  this time:

===========

2014/04/22 16:20:16 VCS INFO V-16-1-50135 User admin fired command: hagrp -switch INT-PRI  TCPRI-CLU2  localclus  from ::ffff:10.100.208.76
2014/04/22 16:20:16 VCS NOTICE V-16-1-10208 Initiating switch of group INT-PRI from system TCPRI-CLU1 to system TCPRI-CLU2
2014/04/22 16:20:16 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:20:16 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:20:16 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:20:16 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:20:16 VCS NOTICE V-16-1-10300 Initiating Offline of Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:20:17 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:20:18 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:20:19 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:20:19 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:20:20 VCS INFO V-16-1-10305 Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:20:20 VCS NOTICE V-16-1-10300 Initiating Offline of Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU1
2014/04/22 16:23:21 VCS WARNING V-16-6-16100 (TCPRI-CLU1) chkvxconfigd:The VxVM process vxconfigd seems to be un-responsive. Stopping vxnotify process, so that resources get unregistered from AMF monitoring
2014/04/22 16:23:21 VCS INFO V-16-2-13717 (TCPRI-CLU1) Output of the completed operation (imf_getnotification) 
==============================================
Cannot continue monitoring event
Got notification for group: PRI-INT

==============================================

2014/04/22 16:25:22 VCS WARNING V-16-2-13011 (TCPRI-CLU1) Resource(PRI-INT): offline procedure did not complete within the expected time.
2014/04/22 16:25:22 VCS ERROR V-16-2-13063 (TCPRI-CLU1) Agent is calling clean for resource(PRI-INT) because offline did not complete within the expected time.
2014/04/22 16:26:23 VCS ERROR V-16-2-13006 (TCPRI-CLU1) Resource(PRI-INT): clean procedure did not complete within the expected time.
2014/04/22 16:27:24 VCS INFO V-16-1-10305 Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) is offline on TCPRI-CLU1 (VCS initiated)
2014/04/22 16:27:24 VCS NOTICE V-16-1-10446 Group INT-PRI is offline on system TCPRI-CLU1
2014/04/22 16:27:24 VCS NOTICE V-16-1-10301 Initiating Online of Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:27:24 VCS NOTICE V-16-10031-1513 (TCPRI-CLU2) DiskGroup:PRI-INT:online:Diskgroups will be imported with reservations.
2014/04/22 16:27:45 VCS WARNING V-16-10031-1509 (TCPRI-CLU2) DiskGroup:PRI-INT:online:vxdg import succeeded on Disk Group PRI-INT.
2014/04/22 16:27:45 VCS NOTICE V-16-10031-1559 (TCPRI-CLU2) DiskGroup:PRI-INT:online:Volumes in DiskGroup PRI-INT will be started automatically as part of import command,the system level autostartvolume is set On 
2014/04/22 16:27:45 VCS INFO V-16-2-13717 (TCPRI-CLU2) Output of the completed operation (imf_getnotification) 
==============================================
Got notification for group: PRI-INT

==============================================

2014/04/22 16:27:46 VCS INFO V-16-1-10298 Resource PRI-INT (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:27:46 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:27:46 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:27:46 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:27:46 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:27:46 VCS NOTICE V-16-1-10301 Initiating Online of Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) on System TCPRI-CLU2
2014/04/22 16:27:49 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INT-HS (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:28:05 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INTJRNALT (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:28:06 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INTBACKUP (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:28:07 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INTJRNPRI (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:28:08 VCS INFO V-16-1-10298 Resource TRAKPRIVOL-INT-DB (Owner: Unspecified, Group: INT-PRI) is online on TCPRI-CLU2 (VCS initiated)
2014/04/22 16:28:08 VCS NOTICE V-16-1-10447 Group INT-PRI is online on system TCPRI-CLU2

===============

the last one

Hi Solom,

even in the last one the online procedure took only a few seconds.

What took long was the offline procedure because according to the logs the vxconfigd is either not running or not responding:

2014/04/22 16:23:21 VCS WARNING V-16-6-16100 (TCPRI-CLU1) chkvxconfigd:The VxVM process vxconfigd seems to be un-responsive. Stopping vxnotify process, so that resources get unregistered from AMF monitoring

vxconfigd is the main VxVM daemon, who manages the import/export of diskgroups and start/stop of volumes.

Can you check if vxconfigd is running?

#ps -ef | grep vxconfigd

if running check if it is enabled:

#vxdctl mode

if disabled try to enable:

#vxdctl enable

or

vxconfigd -m enable

if vxconfigd is not running or can't be enabled try to restart:

#vxconfigd -k -x syslog

regards,
Dan

[root@TCPRI-CLU1 ~]# vxdctl mode
mode: enabled
[root@TCPRI-CLU1 ~]#