08-09-2018 07:22 PM
2018/08/09 15:48:46 VCS INFO V-16-1-10305 Resource JavaMethodServer (Owner: unknown, Group: Documentum) is offline on devuaedms12 (VCS initiated)
2018/08/09 15:48:46 VCS NOTICE V-16-1-10446 Group Documentum is offline on system devuaedms12
2018/08/09 15:48:46 VCS INFO V-16-6-15004 (devuaedms12) hatrigger:Failed to send trigger for postoffline; script doesn't exist
2018/08/09 15:48:46 VCS INFO V-16-1-10305 Resource diskgroup_cfs4 (Owner: unknown, Group: Documentum_ENV_B) is offline on devuaedms12 (VCS initiated)
2018/08/09 15:48:46 VCS INFO V-16-1-10305 Resource diskgroup_cfs3 (Owner: unknown, Group: Documentum_ENV_B) is offline on devuaedms12 (VCS initiated)
2018/08/09 15:48:46 VCS NOTICE V-16-1-10446 Group Documentum_ENV_B is offline on system devuaedms12
2018/08/09 15:48:46 VCS NOTICE V-16-1-10300 Initiating Offline of Resource vxfsckd (Owner: unknown, Group: cvm) on System devuaedms12
2018/08/09 15:48:46 VCS INFO V-16-6-15004 (devuaedms12) hatrigger:Failed to send trigger for postoffline; script doesn't exist
2018/08/09 15:48:48 VCS INFO V-16-1-10305 Resource vxfsckd (Owner: unknown, Group: cvm) is offline on devuaedms12 (VCS initiated)
2018/08/09 15:48:48 VCS NOTICE V-16-1-10300 Initiating Offline of Resource qlogckd (Owner: unknown, Group: cvm) on System devuaedms12
2018/08/09 15:48:49 VCS INFO V-16-2-13001 (devuaedms12) Resource(qlogckd): Output of the completed operation (offline)
UX:vxfs qlogprint: INFO: V-3-22897: There are no QuickLog devices active
2018/08/09 15:48:50 VCS INFO V-16-1-10305 Resource qlogckd (Owner: unknown, Group: cvm) is offline on devuaedms12 (VCS initiated)
2018/08/09 15:48:50 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cvm_clus (Owner: unknown, Group: cvm) on System devuaedms12
2018/08/09 15:48:52 VCS ERROR V-16-10001-1005 (devuaedms12) CVMCluster:???:monitor:node - state: out of cluster
reason: user initiated stop
2018/08/09 15:48:53 VCS INFO V-16-1-10305 Resource cvm_clus (Owner: unknown, Group: cvm) is offline on devuaedms12 (VCS initiated)
2018/08/09 15:48:53 VCS NOTICE V-16-1-10446 Group cvm is offline on system devuaedms12
2018/08/09 15:48:53 VCS NOTICE V-16-1-10010 Stopping all agents
2018/08/09 15:48:53 VCS NOTICE V-16-1-10322 System devuaedms12 (Node '1') changed state from LEAVING to EXITING
2018/08/09 15:48:53 VCS NOTICE V-16-1-10322 System devuaedms12 (Node '1') changed state from EXITING to EXITED
08-09-2018 11:44 PM
The server run on Solaris 9 and VCS 4.1 version, it is unable to start up the service group online
root@devuaedms12 # hastatus -sum
-- SYSTEM STATE
-- System State Frozen
A devuaedms11 RUNNING 0
A devuaedms12 RUNNING 0
-- GROUP STATE
-- Group System Probed AutoDisabled State
B Documentum devuaedms11 Y N PARTIAL|FAULTED
B Documentum devuaedms12 Y N PARTIAL
B Documentum_ENV_B devuaedms11 Y N PARTIAL
B Documentum_ENV_B devuaedms12 Y N STARTING|PARTIAL
B MNICB_SITgroup devuaedms11 Y N ONLINE
B MNICB_SITgroup devuaedms12 Y N ONLINE
B cvm devuaedms11 Y N ONLINE
B cvm devuaedms12 Y N ONLINE
B documentum_bkup devuaedms11 Y N ONLINE
B documentum_bkup devuaedms12 Y N OFFLINE
-- RESOURCES FAILED
-- Group Type Resource System
C Documentum ContentServer ContentServer devuaedms11
C Documentum_ENV_B ContentServer ContentServer_B devuaedms11
C Documentum_ENV_B Proxy DocBroker_B devuaedms11
C Documentum_ENV_B Proxy DocBroker_B devuaedms12
Please advice if there is any advice so I can fix it
08-22-2018 03:14 AM
You are only showing us a piece of the log for "user initiated stop".
If something else happened prior to this, you need to check higher up in engine_A log.
I see various errors in engine_A since 13 July, e.g. :
2018/07/13 21:08:27 VCS ERROR V-16-2-13027 (devuaedms11) Resource(ContentServer_B) - monitor procedure did not complete within the expected time.
2018/07/17 07:52:08 VCS INFO V-16-1-10307 Resource SITgroup (Owner: unknown, Group: MNICB_SITgroup) is offline on devuaedms12 (Not initiated by VCS)
2018/07/17 07:52:09 VCS ERROR V-16-10001-5005 (devuaedms11) IPMultiNICB:mIP_bkup:online:Error in configuring IP address
2018/07/17 07:52:11 VCS ERROR V-16-1-10205 Group MNICB_SITgroup is faulted on system devuaedms11
2018/07/17 07:54:57 VCS ERROR V-16-2-13027 (devuaedms11) Resource(ContentServer) - monitor procedure did not complete within the expected time.
2018/07/17 08:00:31 VCS ERROR V-16-2-13210 (devuaedms12) Agent is calling clean for resource(ContentServer) because 4 successive invocations of the monitor procedure did not complete within the expected time.
2018/07/29 01:30:30 VCS ERROR V-16-2-13066 (devuaedms12) Agent is calling clean for resource(ContentServer) because the resource is not up even after online completed.
Many more errors related to network and resources such as ContentServer.
I feel that you need to perform a complete health check of the entire environment as EVERYTHING seems to be very old and out of support.
08-24-2018 06:10 AM
Fully agree with Marianne, get us full logs, also, share the main.cf, what are the dependencies between the groups.