VCS with VxVM - newgroup is auto-disabled in cluster
Hi Guys, new to Veritas World. trying to create and Active/passive on two node VCS cluster with VxVM. Installed SFCFS filesets on two AIX nodes. I have a shared disk of 5G on both nodes. I'm able to deport Diskgroup and import on second node and mount filesystem. The error which I'm getting is - # hagrp -online newgroup -sys tiefaphap603 VCS WARNING V-16-1-10159 Group newgroup is auto-disabled in cluster. This can happen if group is not probed on all alive nodes in group's SystemList or VCS engine is not running on all alive nodes in group's SystemList. #vi main.cf include "OracleASMTypes.cf" include "types.cf" include "CFSTypes.cf" include "Db2udbTypes.cf" include "OracleTypes.cf" include "SybaseTypes.cf" cluster tstcls ( UserNames = { admin = bklJknKikEkqGfhFhh } ClusterAddress = "10.68.73.180" Administrators = { admin } ) system tiefaphap603 ( ) system tiefaphap604 ( ) group ClusterService ( SystemList = { tiefaphap603 = 0, tiefaphap604 = 1 } AutoStartList = { tiefaphap603, tiefaphap604 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = en0 Address = "10.68.73.180" NetMask = "255.255.255.0" ) NIC csgnic ( Device = en0 NetworkHosts @tiefaphap603 = { "10.68.73.205" } NetworkHosts @tiefaphap604 = { "10.68.73.184" } ) webip requires csgnic // resource dependency tree // // group ClusterService // { // IP webip // { // NIC csgnic // } // } group newgroup ( SystemList = { tiefaphap603 = 0, tiefaphap604 = 1 } AutoStartList = { tiefaphap603 } ) DiskGroup data_dg ( DiskGroup = clshareddg ) Mount mnt ( MountPoint = "/clsfs02" BlockDevice = "/dev/vx/dsk/clshareddg/clslv02" FSType = vxfs ) mnt requires data_dg // resource dependency tree // // group newgroup // { // Mount mnt // { // DiskGroup data_dg // } // }Solved6.1KViews0likes4Comments
Resource faults issue
Hi Team, Many times we face resource faults alerts on our AIX server,but when we check it shows all are running fine. Also, when checked logs it shows not (initiated by VCS) alerts. Our main concern is to troubleshoot, why we get these alerts and if we get these alerts then why cluster resource is not showing faulty. Below are the logs, -- SYSTEM STATE -- System State Frozen A xxxibm012 RUNNING 0 A xxxibm014 RUNNING 0 -- GROUP STATE -- Group System Probed AutoDisabled State B ClusterService xxxibm012 Y N ONLINE B ClusterService xxxibm014 Y N OFFLINE B DB_INSIGHT_STAGE xxxibm012 Y N ONLINE B DB_INSIGHT_STAGE xxxibm014 Y N OFFLINE ============================================================= 2015/04/21 10:14:53 VCS INFO V-16-1-53504 VCS Engine Alive message!! 2015/04/21 12:57:32 VCS WARNING V-16-10011-5611 (clnibm014) NIC:csgnic:monitor:Second PingTest failed for Virtual Interface en4. Resource is OFFLINE 2015/04/21 12:57:32 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg from localhost 2015/04/21 12:57:33 VCS ERROR V-16-1-54031 Resource csgnic (Owner: Unspecified, Group: ClusterService) is FAULTED on sys clnibm014 2015/04/21 12:57:33 VCS INFO V-16-6-0 (clnibm014) resfault:(resfault) Invoked with arg0=clnibm014, arg1=csgnic, arg2=ONLINE 2015/04/21 12:57:49 VCS INFO V-16-6-15002 (clnibm014) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/resfault clnibm014 csgnic ONLINE successfully 2015/04/21 12:58:18 VCS ERROR V-16-1-54031 Resource proxy_DB_INSPRD (Owner: Unspecified, Group: DB_INSIGHT_STAGE) is FAULTED on sys clnibm014 2015/04/21 12:58:18 VCS INFO V-16-6-0 (clnibm014) resfault:(resfault) Invoked with arg0=clnibm014, arg1=proxy_DB_INSPRD, arg2=ONLINE 2015/04/21 12:58:29 VCS INFO V-16-6-15002 (clnibm014) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/resfault clnibm014 proxy_DB_INSPRD ONLINE successfully 2015/04/21 12:58:33 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Primary test to confirm Online status succeeded. from localhost 2015/04/21 12:58:34 VCS INFO V-16-1-10299 Resource csgnic (Owner: Unspecified, Group: ClusterService) is online on clnibm014 (Not initiated by VCS) 2015/04/21 12:58:34 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group ClusterService on all nodes 2015/04/21 12:58:34 VCS NOTICE V-16-1-51034 Failover group ClusterService is already active. Ignoring Restart 2015/04/21 12:59:18 VCS INFO V-16-1-10299 Resource proxy_DB_INSPRD (Owner: Unspecified, Group: DB_INSIGHT_STAGE) is online on clnibm014 (Not initiated by VCS) 2015/04/21 12:59:18 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group DB_INSIGHT_STAGE on all nodes 2015/04/21 12:59:18 VCS NOTICE V-16-1-51034 Failover group DB_INSIGHT_STAGE is already active. Ignoring Restart 2015/04/21 12:59:34 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Primary test to confirm Online status succeeded. from localhost 2015/04/21 13:18:53 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Relying on secondary test to confirm Online status. from localhost 2015/04/21 13:19:34 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Primary test to confirm Online status succeeded. from localhost 2015/04/21 13:44:49 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Relying on secondary test to confirm Online status. from localhost 2015/04/21 13:45:34 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Primary test to confirm Online status succeeded. from localhost 2015/04/21 14:14:54 VCS INFO V-16-1-53504 VCS Engine Alive message!! 2015/04/21 16:48:59 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Relying on secondary test to confirm Online status. from localhost 2015/04/21 16:49:34 VCS INFO V-16-1-50135 User root fired command: hares -modify csgnic ConfidenceMsg Primary test to confirm Online status succeeded.Solved4.7KViews1like9Comments
VCS Database Service Group Service Group Fails After Initiating Offline
Hi, We have 2 Storage Foundation for Oracle RAC 5.1SP1 serversrunning on AIX 5.3 Servers. We have Application Service Groupand DatabaseService Groupwhich is both active active. I encountered a problem when i initiated manual offline of the Service group. Application Service Group is ok. But in the Database Service Group it failedand the 2 servers unpectedly restarted and i encountered split-brain condition. Here's the engineA.log: ServiceGroup_PRD Application successfull offiline 2015/06/24 01:17:14 VCS INFO V-16-1-50135 User root fired command: hagrp -offline ServiceGroup_PRD Server1 from localhost 2015/06/24 01:17:14 VCS NOTICE V-16-1-10167 Initiating manual offline of group SERVICEGROUP_PRD on system SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource archive2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource backup2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource redolog5_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource redolog6_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u12_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u13_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u14_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:14 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u15_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:17 VCS INFO V-16-1-10305 Resource redolog5_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:17 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgredolog5 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:19 VCS INFO V-16-1-10305 Resource u12_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:19 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu12 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:19 VCS INFO V-16-1-10305 Resource redolog6_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:19 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgredolog6 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:20 VCS INFO V-16-1-10305 Resource oradgredolog5 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:20 VCS INFO V-16-1-10305 Resource u14_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:20 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu14 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:21 VCS INFO V-16-1-10305 Resource oradgu12 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:21 VCS INFO V-16-1-10305 Resource u15_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:21 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu15 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:21 VCS INFO V-16-1-10305 Resource oradgredolog6 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:21 VCS INFO V-16-1-10305 Resource oradgu14 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:21 VCS INFO V-16-1-10305 Resource u13_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:21 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu13 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:24 VCS INFO V-16-1-10305 Resource oradgu15 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:24 VCS INFO V-16-1-10305 Resource oradgu13 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:32 VCS INFO V-16-1-10305 Resource archive2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgarchive2 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:32 VCS INFO V-16-1-10305 Resource backup2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:32 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgbackup2 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER1 2015/06/24 01:17:33 VCS INFO V-16-1-10305 Resource oradgarchive2 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:34 VCS INFO V-16-1-10305 Resource oradgbackup2 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:17:35 VCS NOTICE V-16-1-10446 Group SERVICEGROUP_PRD is offline on system SERVER1 2015/06/24 01:17:35 VCS INFO V-16-6-15002 (SERVER1) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/lvmvg_postoffline SERVER1 SERVICEGROUP_PRD successfully 2015/06/24 01:17:35 VCS INFO V-16-6-0 (SERVER1) postoffline:Invoked with arg0=SERVER1, arg1=SERVICEGROUP_PRD 2015/06/24 01:17:36 VCS INFO V-16-6-15002 (SERVER1) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline SERVER1 SERVICEGROUP_PRD successfully 2015/06/24 01:17:41 VCS NOTICE V-16-1-10167 Initiating manual offline of group SERVICEGROUP_PRD on system SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource archive2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource backup2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource redolog5_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource redolog6_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u12_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u13_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u14_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:41 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u15_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:43 VCS INFO V-16-1-10305 Resource redolog5_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:43 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgredolog5 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:43 VCS INFO V-16-1-10305 Resource archive2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:43 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgarchive2 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:43 VCS INFO V-16-1-10305 Resource backup2_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:43 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgbackup2 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:43 VCS INFO V-16-1-10305 Resource u12_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:43 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu12 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:44 VCS INFO V-16-1-10305 Resource u13_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:44 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu13 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:44 VCS INFO V-16-1-10305 Resource oradgredolog5 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:44 VCS INFO V-16-1-10305 Resource u14_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:44 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu14 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:44 VCS INFO V-16-1-10305 Resource u15_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:44 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgu15 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:44 VCS INFO V-16-1-10305 Resource redolog6_mnt (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:44 VCS NOTICE V-16-1-10300 Initiating Offline of Resource oradgredolog6 (Owner: unknown, Group: SERVICEGROUP_PRD) on System SERVER2 2015/06/24 01:17:45 VCS INFO V-16-1-10305 Resource oradgbackup2 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:45 VCS INFO V-16-1-10305 Resource oradgarchive2 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:46 VCS INFO V-16-1-10305 Resource oradgu12 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:46 VCS INFO V-16-1-10305 Resource oradgu14 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:46 VCS INFO V-16-1-10305 Resource oradgu13 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:47 VCS INFO V-16-1-10305 Resource oradgu15 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:47 VCS INFO V-16-1-10305 Resource oradgredolog6 (Owner: unknown, Group: SERVICEGROUP_PRD) is offline on SERVER2 (VCS initiated) 2015/06/24 01:17:47 VCS NOTICE V-16-1-10446 Group SERVICEGROUP_PRD is offline on system SERVER2 2015/06/24 01:17:47 VCS INFO V-16-6-15002 (SERVER2) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/lvmvg_postoffline SERVER2 SERVICEGROUP_PRD successfully 2015/06/24 01:17:47 VCS INFO V-16-6-0 (SERVER2) postoffline:Invoked with arg0=SERVER2, arg1=SERVICEGROUP_PRD2015/06/24 01:17:48 VCS INFO V-16-6-15002 (SERVER2) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline SERVER2 SERVICEGROUP_PRDsuccessfully ServiceGroup_DBPRD Database fails tooffline 2015/06/24 01:18:51 VCS INFO V-16-1-50135 User root fired command: hagrp -offline SERVICEGROUP_DBPRD SERVER1 from localhost 2015/06/24 01:18:51 VCS NOTICE V-16-1-10167 Initiating manual offline of group SERVICEGROUP_DBPRD on system SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource archive_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource backup_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource ocr_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource quorum_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource redolog1_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource redolog2_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u02_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u03_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u04_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u05_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u06_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource u07_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:51 VCS NOTICE V-16-1-10300 Initiating Offline of Resource voting_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:52 VCS ERROR V-16-20011-5503 (SERVER1) CFSMount:redolog1_mnt:offline:Umount Failed : Mount Point : /redolog1 2015/06/24 01:18:52 VCS ERROR V-16-20011-5503 (SERVER1) CFSMount:quorum_mnt:offline:Umount Failed : Mount Point : /quorum 2015/06/24 01:18:52 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:redolog1_mnt:offline:Attempting fuser TERM : Mount Point : /redolog1 2015/06/24 01:18:52 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:quorum_mnt:offline:Attempting fuser TERM : Mount Point : /quorum 2015/06/24 01:18:53 VCS INFO V-16-1-10305 Resource ocr_mnt (Owner: unknown, Group: SERVICEGROUP_DBPRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:18:53 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cvmocr (Owner: unknown, Group: SERVICEGROUP_DBPRD) on System SERVER1 2015/06/24 01:18:53 VCS ERROR V-16-20011-5503 (SERVER1) CFSMount:redolog2_mnt:offline:Umount Failed : Mount Point : /redolog2 2015/06/24 01:18:53 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:redolog2_mnt:offline:Attempting fuser TERM : Mount Point : /redolog2 2015/06/24 01:18:55 VCS INFO V-16-1-10305 Resource cvmocr (Owner: unknown, Group: SERVICEGROUP_DBPRD) is offline on SERVER1 (VCS initiated) 2015/06/24 01:18:55 VCS ERROR V-16-20011-5503 (SERVER1) CFSMount:u03_mnt:offline:Umount Failed : Mount Point : /u03 2015/06/24 01:18:55 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:u03_mnt:offline:Attempting fuser TERM : Mount Point : /u03 2015/06/24 01:18:57 VCS ERROR V-16-20011-5503 (SERVER1) CFSMount:u02_mnt:offline:Umount Failed : Mount Point : /u02 2015/06/24 01:18:57 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:u02_mnt:offline:Attempting fuser TERM : Mount Point : /u02 2015/06/24 01:18:58 VCS ERROR V-16-20011-5503 (SERVER1) CFSMount:u04_mnt:offline:Umount Failed : Mount Point : /u04 2015/06/24 01:18:58 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:u04_mnt:offline:Attempting fuser TERM : Mount Point : /u04 2015/06/24 01:19:03 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:redolog1_mnt:offline:Attempting fuser TERM : Mount Point : /redolog1 2015/06/24 01:19:03 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:redolog2_mnt:offline:Attempting fuser TERM : Mount Point : /redolog2 2015/06/24 01:19:03 VCS NOTICE V-16-20011-5510 (SERVER1) CFSMount:quorum_mnt:offline:Attempting fuser TERM : Mount Point : /quorum 2015/06/24 01:40:15 VCS NOTICE V-16-1-11022 VCS engine (had) started 2015/06/24 01:40:15 VCS INFO V-16-1-10196 Cluster logger started 2015/06/24 01:40:15 VCS NOTICE V-16-1-11050 VCS engine version=5.1 2015/06/24 01:40:15 VCS NOTICE V-16-1-11051 VCS engine join version=5.1.00.0 2015/06/24 01:40:15 VCS NOTICE V-16-1-11052 VCS engine pstamp=Veritas-5.1-10/06/09-14:37:00 2015/06/24 01:40:15 VCS NOTICE V-16-1-10114 Opening GAB library 2015/06/24 01:40:16 VCS NOTICE V-16-1-10619 'HAD' starting on: SERVER1 2015/06/24 01:40:16 VCS INFO V-16-1-10125 GAB timeout set to 30000 ms 2015/06/24 01:40:16 VCS NOTICE V-16-1-11057 GAB registration monitoring timeout set to 200000 ms 2015/06/24 01:40:16 VCS NOTICE V-16-1-11059 GAB registration monitoring action set to log system message 2015/06/24 01:40:26 VCS INFO V-16-1-10077 Received new cluster membership 2015/06/24 01:40:26 VCS NOTICE V-16-1-10112 System (SERVER1) - Membership: 0x3, DDNA: 0x0 2015/06/24 01:40:26 VCS NOTICE V-16-1-10322 System (Node '1') changed state from UNKNOWN to INITING 2015/06/24 01:40:26 VCS NOTICE V-16-1-10086 System SERVER1 (Node '0') is in Regular Membership - Membership: 0x3 2015/06/24 01:40:26 VCS NOTICE V-16-1-10086 System (Node '1') is in Regular Membership - Membership: 0x3 2015/06/24 01:40:26 VCS NOTICE V-16-1-10453 Node: 1 changed name from: '' to: 'SERVER2' 2015/06/24 01:40:26 VCS NOTICE V-16-1-10322 System SERVER2 (Node '1') changed state from INITING to CURRENT_DISCOVER_WAIT 2015/06/24 01:40:26 VCS NOTICE V-16-1-10322 System SERVER1 (Node '0') changed state from CURRENT_DISCOVER_WAIT to LOCAL_BUILD 2015/06/24 01:40:26 VCS NOTICE V-16-1-10322 System SERVER2 (Node '1') changed state from CURRENT_DISCOVER_WAIT to CURRENT_PEER_WAIT 2015/06/24 01:40:28 VCS NOTICE V-16-1-52006 UseFence=SCSI3. Fencing is enabled 2015/06/24 01:40:28 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying... 2015/06/24 01:40:43 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying... 2015/06/24 01:40:58 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying... 2015/06/24 01:41:13 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying... 2015/06/24 01:41:28 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying... 2015/06/24 01:41:43 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying... 2015/06/24 01:41:58 VCS CRITICAL V-16-1-10031 VxFEN driver not configured. VCS Stopping. Manually restart VCS after configuring fencing The solution i did here was to run vxfenclearpre. Is there any logs i need to check why the servers faults and restarted unexpectedly after service group offline? Any help will appreciate. Thanks.Solved3.7KViews0likes9Comments
VCS stop and service group offline quesiton
I have a quick question regarding service group offline. We only want to bring up/down services when we execute hagrp -online/-offline manually. When VCS shutdown, we don't want to bring up the service offline. I know if we freeze it won't go to offline entry. Do we have any other option? For an exmaple, for only we can disable the autostart. Do we have any thing for offline?Solved3.3KViews0likes2CommentsNeed Active/Active solution
Dear Gurus, My client planning to move some HA products. But they still not sure which one to choose. The requirement is HA should be Active- Active setup in duel DC. They ready to buy some products, I recommend SF cluster but not sure which one is best. Kindly suggest me which combination suits with below products. Available DC Setup : 1 Active DC ,1 Active near DC and 1 DR location Products in hand: 1) Oracle & IBM hardware 2) storage (3 * 20K vmak) 3) EMC vplex 4) DB - Oracle RAC (11gR2) Now which symantec HA clustering topology & method can help to achieve 99.99%. To make active-active in HA any minimum perquisites need to maintain from Network, SAN & storage. How about IBM P780 , EMC vplex with SFRAC for Active - Active- Standby ? Thanks, Jay2.3KViews1like10CommentsUnable to bring the Service Group online.
Hi All, I tried to bring a SG online in a node but it's not comming online. Let me explaing the issue. We did reboot of a node aixprd001 and we found that /etc/filesystem is corrupted so the SG bosinit_SG is in partial state since lot of cluster FS in not mounted. Then we corrected the entry and done the manual mout of all the FS but the SG still show the status partial so we did the bellow command. hagrp -clear bosinit_SG -all Once done the SG is in online state. For safer side we tried to offline the SG and brought it up online again but the SG failed to come online, Bellow is the only error we able find the engine_A.log file. 2014/12/17 06:49:04 VCS NOTICE V-16-1-10166 Initiating manual online of group bosinit_SG on system aixprd001 2014/12/17 06:49:04 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group bosinit_SG on all nodes Please help me by providing suggestion, I will provide the output of logs if needed. Thanks, RufusSolved2KViews0likes4CommentsCan I/O fencing disk be cloned?
hi we are trying to do a storage migration which is runnig SFRAC. so our disks are including oracle data disks and 3 I/O fencing disks. my question is can we use storage cloned(hardware replication) to do the storage migration? I know data disk can be cloned to new storage without problem, but fencing disk I am not sure. Does anyone know I/O fencing disk also can be cloned ? Thanks AllenSolved1.9KViews1like5Commentsunexpected reboot
Hi all, I am runing a SFRAC environnement and one of my 2 nodes cluster frequently reboots unexpectedly. I went through the OS logs and the VCS engine_A but didnt find any clue. Is that an eviction ? when i run lltstat, i can see some errors counters - what does it exactly mean ? LLT errors: 0 Rcv not connected 0 Rcv unconfigured 0 Rcv bad dest address 0 Rcv bad source address 0 Rcv bad generation 0 Rcv no buffer 0 Rcv malformed packet 0 Rcv wrong length packet 0 Rcv bad SAP 0 Rcv bad STREAM primitive 0 Rcv bad DLPI primitive 0 Rcv DLPI error 120 Snd not connected 0 Snd no buffer 0 Snd stream flow drops 42867 Snd no links up 0 Rcv bad checksum 0 Rcv bad udp/ether source address 0 Rcv DLPI link-down error how to efficiently trace these reboots ??Solved1.9KViews0likes5CommentsApplication Agent PID file quesiton
I have created a unix scirpt to do some task and enf of the task it will create a local file in /var/lock/abcd.lock file. I created a application resource and put /var/lock/abcd.lock for PidFiles attribute. The lock file is creating, but not resouce is not making as fault. I am not sure why it is making as fault. If the lock file is there, applicaiton agent shouln't bring the resouce up?Solved1.9KViews0likes8Comments
Need Active/Active solution
Dear Gurus, My client planning to move some HA products. But they still not sure which one to choose. The requirement is HA should be Active- Active setup in duel DC. They ready to buy some products, I recommend SF cluster but not sure which one is best. Kindly suggest me which combination suits with below products. Available DC Setup : 1 Active DC ,1 Active near DC and 1 DR location Products in hand: 1) Oracle & IBM hardware 2) storage (3 * 20K vmak) 3) EMC vplex 4) DB - Oracle RAC (11gR2) Now which symantec HA clustering topology & method can help to achieve 99.99%. To make active-active in HA any minimum perquisites need to maintain from Network, SAN & storage. How about IBM P780 , EMC vplex with SFRAC for Active - Active- Standby ? Thanks, Jay