Highlighted

Veritas 7.4 cluster resources goes offline automatically

2019/01/29 22:14:17 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(cfsmount5) because the resource became OFFLINE unexpectedly, on its own.

2019/01/29 22:14:18 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(cfsmount3) because the resource became OFFLINE unexpectedly, on its own.

2019/01/29 22:14:20 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(cfsmount4) because the resource became OFFLINE unexpectedly, on its own.

2019/01/29 22:14:22 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(fm_db_Server5) because the resource became OFFLINE unexpectedly, on its own.

2019/01/29 22:14:22 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(fm_db_ServerERBS) because the resource became OFFLINE unexpectedly, on its own.

2019/01/29 22:14:25 VCS INFO V-16-2-13068 (LSVPRDMMFE4A) Resource(cfsmount4) - clean completed successfully.

2019/01/29 22:14:25 VCS INFO V-16-1-10307 Resource cfsmount4 (Owner: Unspecified, Group: Mediation3_DG) is offline on LSVPRDMMFE4A (Not initiated by VCS)

2019/01/29 22:14:25 VCS INFO V-16-1-50149 VCS shall not initiate failover for service group Mediation3_DG because the faulted resource is not critical and no critical parent resource is affected by the fault

2019/01/29 22:14:25 VCS INFO V-16-6-15015 (LSVPRDMMFE4A) hatrigger:/opt/VRTSvcs/bin/triggers/resfault is not a trigger scripts directory or can not be executed

2019/01/29 22:14:25 VCS INFO V-16-2-13068 (LSVPRDMMFE4A) Resource(cfsmount3) - clean completed successfully.

2019/01/29 22:14:25 VCS INFO V-16-1-10307 Resource cfsmount3 (Owner: Unspecified, Group: Mediation2_DG) is offline on LSVPRDMMFE4A (Not initiated by VCS)

2019/01/29 22:14:25 VCS INFO V-16-1-50149 VCS shall not initiate failover for service group Mediation2_DG because the faulted resource is not critical and no critical parent resource is affected by the fault

2019/01/29 22:14:25 VCS INFO V-16-6-15015 (LSVPRDMMFE4A) hatrigger:/opt/VRTSvcs/bin/triggers/resfault is not a trigger scripts directory or can not be executed

2019/01/29 22:14:25 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(fm_db_ServerTest1) because the resource became OFFLINE unexpectedly, on its own.

2019/01/29 22:14:25 VCS INFO V-16-2-13068 (LSVPRDMMFE4A) Resource(cfsmount5) - clean completed successfully.

2019/01/29 22:14:25 VCS INFO V-16-1-10307 Resource cfsmount5 (Owner: Unspecified, Group: Mediation4_DG) is offline on LSVPRDMMFE4A (Not initiated by VCS)

2019/01/29 22:14:25 VCS INFO V-16-1-50149 VCS shall not initiate failover for service group Mediation4_DG because the faulted resource is not critical and no critical parent resource is affected by the fault

2019/01/29 22:14:26 VCS INFO V-16-6-15015 (LSVPRDMMFE4A) hatrigger:/opt/VRTSvcs/bin/triggers/resfault is not a trigger scripts directory or can not be executed

2019/01/29 22:14:26 VCS INFO V-16-2-13068 (LSVPRDMMFE4A) Resource(fm_db_Server5) - clean completed successfully.

2019/01/29 22:14:26 VCS INFO V-16-2-13068 (LSVPRDMMFE4A) Resource(fm_db_ServerERBS) - clean completed successfully.

2019/01/29 22:14:26 VCS ERROR V-16-2-13073 (LSVPRDMMFE4A) Resource(fm_db_Server5) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 1) the resource.

2019/01/29 22:14:26 VCS ERROR V-16-2-13073 (LSVPRDMMFE4A) Resource(fm_db_ServerERBS) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 1) the resource.

1 Solution

Accepted Solutions
Highlighted
Accepted Solution!

Re: Veritas 7.4 cluster resources goes offline automatically

most common cause of seeing the errors below

 

2019/01/29 22:14:17 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(cfsmount5) because the resource became OFFLINE unexpectedly, on its own.

 

are:

1. there was a storage related issue resulting in cfsmount issue (offline)

2. admin mistake (accidentally umount cfs mounts)

3. server load was too high

since the issue seems occured on one node only, you can check to see if the same cfsmounts are OK on the othwer node(s) and if the OS on the problematic node sees the storage used by the cfsmounts.  if the OS is not able to access the needed storage, check SAN and stoage to make sure the node can access the luns for cfsmounts.  if everything seems OK but cfsmounts are down. run vxdg list to see if the dgs are still o,[prted.  if the dgs are still im,ported, just run hares -online to online the cfsmounts.

you can allso run the command below and post the outout

 

hastatus -sum

 

 

 

View solution in original post

1 Reply
Highlighted
Accepted Solution!

Re: Veritas 7.4 cluster resources goes offline automatically

most common cause of seeing the errors below

 

2019/01/29 22:14:17 VCS ERROR V-16-2-13067 (LSVPRDMMFE4A) Agent is calling clean for resource(cfsmount5) because the resource became OFFLINE unexpectedly, on its own.

 

are:

1. there was a storage related issue resulting in cfsmount issue (offline)

2. admin mistake (accidentally umount cfs mounts)

3. server load was too high

since the issue seems occured on one node only, you can check to see if the same cfsmounts are OK on the othwer node(s) and if the OS on the problematic node sees the storage used by the cfsmounts.  if the OS is not able to access the needed storage, check SAN and stoage to make sure the node can access the luns for cfsmounts.  if everything seems OK but cfsmounts are down. run vxdg list to see if the dgs are still o,[prted.  if the dgs are still im,ported, just run hares -online to online the cfsmounts.

you can allso run the command below and post the outout

 

hastatus -sum

 

 

 

View solution in original post