Forum Discussion

SABB's avatar
14 years ago

VCS volume group issue

I am facing an issue in VCS which is installed in Oracle Solaris 10 9/10 s10x_u9wos_14a X86 servers. The shared volume group suddenly disappeared from both servers and since then I am unable to use VCS. Could you please advise in how to solve this issue

engine_A.log:

2011/12/13 09:32:20 VCS WARNING V-16-10001-1060 (sahomxapp01u) DiskGroup:App_dg:clean:The command 'vxvol -g appdg stopall' failed.  Doing a forced stop
2011/12/13 09:32:21 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (clean)
==============================================
VxVM vxdg ERROR V-5-1-580 Disk group appdg: Flush failed: Disk group not found
VxVM vxdg ERROR V-5-1-581 Disk group appdg: No such disk group is imported
==============================================

2011/12/13 09:32:21 VCS INFO V-16-2-13068 (sahomxapp01u) Resource(App_dg) - clean completed successfully.
2011/12/13 09:32:21 VCS INFO V-16-2-13072 (sahomxapp01u) Resource(App_dg): Agent is retrying online (attempt number 1 of 1).
2011/12/13 09:32:21 VCS WARNING V-16-10001-1014 (sahomxapp01u) DiskGroup:App_dg:online:Diskgroups will be imported without reservations
2011/12/13 09:32:24 VCS WARNING V-16-10001-1016 (sahomxapp01u) DiskGroup:App_dg:online:vxdg import (clear flag) failed.
2011/12/13 09:32:24 VCS WARNING V-16-10001-1017 (sahomxapp01u) DiskGroup:App_dg:online:Trying force import for the diskgroup.
2011/12/13 09:32:24 VCS ERROR V-16-10001-1003 (sahomxapp01u) DiskGroup:App_dg:online:** ERROR: vxdg import (force) failed on Disk Group appdg
2011/12/13 09:32:29 VCS ERROR V-16-10001-1004 (sahomxapp01u) DiskGroup:App_dg:online:** ERROR: vxdg import failed on Disk Group appdg after vxdctl enable
2011/12/13 09:32:29 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (online)
==============================================
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
2011/12/13 09:28:00 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (clean)
==============================================
VxVM vxdg ERROR V-5-1-580 Disk group appdg: Flush failed: Disk group not found
VxVM vxdg ERROR V-5-1-581 Disk group appdg: No such disk group is imported
==============================================

2011/12/13 09:28:00 VCS INFO V-16-2-13068 (sahomxapp01u) Resource(App_dg) - clean completed successfully.
2011/12/13 09:28:00 VCS INFO V-16-2-13072 (sahomxapp01u) Resource(App_dg): Agent is retrying online (attempt number 1 of 1).
2011/12/13 09:28:00 VCS WARNING V-16-10001-1014 (sahomxapp01u) DiskGroup:App_dg:online:Diskgroups will be imported without reservations
2011/12/13 09:28:03 VCS WARNING V-16-10001-1016 (sahomxapp01u) DiskGroup:App_dg:online:vxdg import (clear flag) failed.
2011/12/13 09:28:03 VCS WARNING V-16-10001-1017 (sahomxapp01u) DiskGroup:App_dg:online:Trying force import for the diskgroup.
2011/12/13 09:28:03 VCS ERROR V-16-10001-1003 (sahomxapp01u) DiskGroup:App_dg:online:** ERROR: vxdg import (force) failed on Disk Group appdg
2011/12/13 09:28:07 VCS ERROR V-16-10001-1004 (sahomxapp01u) DiskGroup:App_dg:online:** ERROR: vxdg import failed on Disk Group appdg after vxdctl enable
2011/12/13 09:28:08 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (online)
==============================================
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
==============================================

2011/12/13 09:30:09 VCS ERROR V-16-2-13066 (sahomxapp01u) Agent is calling clean for resource(App_dg) because the resource is not up even after online completed.
2011/12/13 09:30:09 VCS WARNING V-16-10001-1060 (sahomxapp01u) DiskGroup:App_dg:clean:The command 'vxvol -g appdg stopall' failed.  Doing a forced stop
2011/12/13 09:30:10 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (clean)
==============================================
VxVM vxdg ERROR V-5-1-580 Disk group appdg: Flush failed: Disk group not found
VxVM vxdg ERROR V-5-1-581 Disk group appdg: No such disk group is imported
==============================================

2011/12/13 09:30:10 VCS INFO V-16-2-13068 (sahomxapp01u) Resource(App_dg) - clean completed successfully.
2011/12/13 09:30:10 VCS INFO V-16-2-13071 (sahomxapp01u) Resource(App_dg): reached OnlineRetryLimit(1).
2011/12/13 09:30:11 VCS ERROR V-16-1-10303 Resource App_dg (Owner: Unspecified, Group: App_Group) is FAULTED (timed out) on sys sahomxapp01u
2011/12/13 09:30:11 VCS NOTICE V-16-1-10184 Group Retry: retrying group App_Group on System sahomxapp01u (try #2)
2011/12/13 09:30:11 VCS ERROR V-16-1-10205 Group App_Group is faulted on system sahomxapp01u
2011/12/13 09:30:11 VCS NOTICE V-16-1-10446 Group App_Group is offline on system sahomxapp01u
2011/12/13 09:30:11 VCS NOTICE V-16-1-10301 Initiating Online of Resource App_dg (Owner: Unspecified, Group: App_Group) on System sahomxapp01u
2011/12/13 09:30:11 VCS WARNING V-16-10001-1014 (sahomxapp01u) DiskGroup:App_dg:online:Diskgroups will be imported without reservations
2011/12/13 09:30:14 VCS WARNING V-16-10001-1016 (sahomxapp01u) DiskGroup:App_dg:online:vxdg import (clear flag) failed.
2011/12/13 09:30:14 VCS WARNING V-16-10001-1017 (sahomxapp01u) DiskGroup:App_dg:online:Trying force import for the diskgroup.
2011/12/13 09:30:14 VCS ERROR V-16-10001-1003 (sahomxapp01u) DiskGroup:App_dg:online:** ERROR: vxdg import (force) failed on Disk Group appdg
2011/12/13 09:30:19 VCS ERROR V-16-10001-1004 (sahomxapp01u) DiskGroup:App_dg:online:** ERROR: vxdg import failed on Disk Group appdg after vxdctl enable
2011/12/13 09:30:19 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (online)
==============================================
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
VxVM vxdg ERROR V-5-1-10978 Disk group appdg: import failed:
No valid disk found containing disk group
==============================================

2011/12/13 09:32:20 VCS ERROR V-16-2-13066 (sahomxapp01u) Agent is calling clean for resource(App_dg) because the resource is not up even after online completed.
2011/12/13 09:32:20 VCS WARNING V-16-10001-1060 (sahomxapp01u) DiskGroup:App_dg:clean:The command 'vxvol -g appdg stopall' failed.  Doing a forced stop
2011/12/13 09:32:21 VCS INFO V-16-2-13716 (sahomxapp01u) Resource(App_dg): Output of the completed operation (clean)
==============================================
VxVM vxdg ERROR V-5-1-580 Disk group appdg: Flush failed: Disk group not found
VxVM vxdg ERROR V-5-1-581 Disk group appdg: No such disk group is imported
==============================================

(root_sahomxapp01u)@/var/VRTSvcs/log # vxdisk -o alldgs list
DEVICE       TYPE            DISK         GROUP        STATUS
c1t0d0s2     auto:none       -            -            online invalid
emc_clariion0_0 auto:cdsdisk    -            -            error
emc_clariion0_1 auto:cdsdisk    -            -            error
(root_sahomxapp01u)@/var/VRTSvcs/log #
 

(root_sahomxapp01u)@/var/VRTSvcs/log # vxprint -htr

NOTHING
 

  • Did u mean that storage disappeared ? From the logs it appears that some thing happened to storage and it was not accessible.... Could be possible that veritas had issues during that time.... Is your storage back online now ? G
  • You need to troubleshoot between OS and device.

    You should be able to see error messages in /var/adm/messages that should give an indication of what caused the problem.

    I had a similar issue in my previous company where the switch in our lab just 'died' every now and then... The result was the same as in your case.
    In our case, we could just pull out and reinsert the power supplies on the switch and everything would come up again.

    You may have to inspect physical connections and/or switch logs.

  • Did u mean that storage disappeared ? From the logs it appears that some thing happened to storage and it was not accessible.... Could be possible that veritas had issues during that time.... Is your storage back online now ? G