2 nic resources in a global cluster
hi all. I'm using an application in a global cluster environment, configured by a supplier. this application is using 2 NICs: one for the LAN of the management of the servers and one for the LAN of the management of the network and all the network elements. The supplier only configured a resource associated to the NIC of the network and the network elements so, in case of a fault of this NIC the application can switch, while in case of a fault of the NIC of the management the application freezes. This behaviour is obviously unacceptable, but the answer of the supplier was: the application doesn't support this feature (2 NIC resources together). Is this possible ? I know that you cannot know how the application works, but I think that the configuration of the NIC resources is totally indipendent of the application itself, in whatever way it works. Is this correct ? thanks in advance and BR Tiziano2.6KViews0likes1Comment
failed to bring up HeartBeat interface with GAB config
Dear Experts, I'm facing an issue here in cluster setup 8+2 I have managed to create 1+1 cluster and when trying to add the 3rd node I got stuck in gabconfig step below is the /var/adm/messages Jun 21 22:49:50 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartCore: Starting hardware (clients None) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: Phy capabilities (1) (supported 0x00000f50 0x00000000) (link_config 0x01000000 0x00000000) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.warning] WARNING: bnxe3: Phy 10000fdx requested but not supported Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: Phy requesting link config of 0x00000006 Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: 10Gb Full Duplex Rx Flow ON Tx Flow ON Link Up Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartCore: Hardware started (clients None) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartL2: L2 started (clients L2) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopL2: Stopping L2 (clients L2) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopCore: Stopping hardware (clients None) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: Link Down Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopCore: Hardware stopped (clients None) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopL2: L2 stopped (clients None) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartL2: Starting L2 (clients None) Jun 21 22:49:51 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartCore: Starting hardware (clients None) Jun 21 22:49:52 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: Phy capabilities (1) (supported 0x00000f50 0x00000000) (link_config 0x01000000 0x00000000) Jun 21 22:49:52 MM-BL2 bnxe: [ID 801725 kern.warning] WARNING: bnxe3: Phy 10000fdx requested but not supported Jun 21 22:49:52 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: Phy requesting link config of 0x00000006 Jun 21 22:49:52 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: 10Gb Full Duplex Rx Flow ON Tx Flow ON Link Up Jun 21 22:49:52 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartCore: Hardware started (clients None) Jun 21 22:49:52 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStartL2: L2 started (clients L2) Jun 21 22:49:53 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopL2: Stopping L2 (clients L2) Jun 21 22:49:53 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopCore: Stopping hardware (clients None) Jun 21 22:49:53 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: Link Down Jun 21 22:49:53 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopCore: Hardware stopped (clients None) Jun 21 22:49:53 MM-BL2 bnxe: [ID 801725 kern.info] NOTICE: bnxe3: BnxeHwStopL2: L2 stopped (clients None) knowing that I have the same network profiles interfaces for all blades ( HP Proliant blades ) any ideas how this can be fixed ? BR Mahmoud2KViews0likes1Comment
vxconfigd: fatal: relocation error
Hi In my solaris box where I have VXVM,getting the below error-- root@abc # vxconfigd ld.so.1: vxconfigd: fatal: relocation error: file /etc/vx/lib/discovery.d/libvxemc.so: symbol ddl_asl_trace: referenced symbol not found VxVM vxconfigd ERROR V-5-1-0 Killed. root@abc# vxdg list VxVM vxdg ERROR V-5-1-684 IPC failure: Configuration daemon is not accessible root@abc # vxdisk list VxVM vxdisk ERROR V-5-1-684 IPC failure: Configuration daemon is not accessible root@abc # Daemon is not running.Help me to resolve this issue. regards ArupSolved3.2KViews1like7CommentsVVR paused due to network disconnection
Hi all. I have a global cluster with 2 minicluster systems with solaris 10 installed (SPARC) primary: MIVDB01S - 172.22.8.132 secondary: MILDB08S - 10.66.11.148 I stopped the secondary (init 0) for 3 days, and after that I startup the secondary (boot from ok-prompt) and after 1 day I checked for the situation of the replication but: MIVDB01S root vradmin -g datadg printrvg datarvg Replicated Data Set: datarvg Primary: HostName: 172.22.8.132 <localhost> RvgName: datarvg DgName: datadg Secondary: HostName: 10.66.11.148 RvgName: datarvg DgName: datadg vxrlink -g datadg status datarlk Wed Jan 21 09:51:08 2015 VxVM VVR vxrlink INFO V-5-1-12887 DCM is in use on rlink datarlk. DCM contains 874432 Kbytes (1%) of the Data Volume(s). vradmin -g datadg repstatus datarvg Replicated Data Set: datarvg Primary: Host name: 172.22.8.132 RVG name: datarvg DG name: datadg RVG state: enabled for I/O Data volumes: 1 VSets: 0 SRL name: srl_vol SRL size: 1.00 G Total secondaries: 1 Secondary: Host name: 10.66.11.148 RVG name: datarvg DG name: datadg Data status: consistent, behind Replication status: paused due to network disconnection (dcm resynchronization) Current mode: asynchronous Logging to: DCM (contains 874432 Kbytes) (SRL protection logging) Timestamp Information: N/A vxprint -Pl | grep flags flags: write enabled attached consistent disconnected asynchronous dcm_logging resync_paused ----- 1st solution MIVDB01S root vradmin -g datadg resync datarvg ----- 2nd solution Stop vradmin on secondary then on primary # /usr/sbin/vxstart_vvr stop Start vradmin on secondary then on primary # /usr/sbin/vxstart_vvr start Can you help me ?Solved6.9KViews0likes8Comments
Solaris 10 + Zones + ZFS + VCS
Hi all, I don't have a lot of experiences on VCS I have 2 servers on Solaris 10 update 11, installed on ZFS (rpool), and I want to do a Cluster for an application APP Node A = zonea (zonepath for zonea is ZFS / local disk ZFS ) Node B = zoneb (zonepath for zoneb is ZFS / loacl disk ZFS ) The application APP ( FailOver) have to move between zonea and zoneb With that configuration it is possible to do this ? If not what is the best practices : salaris 10 + zfs local zones + VCS 6.0.1 ? Best RegardsSolved1.7KViews0likes1Commentafter mirroring plex, ssh service are not running
hi all, i have some productions server that are using SFHA Cluster. i need to mirroring volume to new storage using. (#vxassist -g DGname mirror VOLname alloc="vxdisk1 vxdisk2 . . . . vxdisk13") after 190 minutes ssh service cannot be access. i aborted the process, because im afraid it will affected another service group and make server get panic and then reboot. if there anyone have some resolve plan like we mirroring in subdisk level?1.1KViews0likes1Commentresource dependency issue
Hi, I need to clarify things. In a 2 nodes vcs cluster we have an group with application resource, mount point resource and disk group resource. Application resource is the parent for mount point resource, and this one is parent for disk group resource. Normally when there is an problem with disk group resource and this goes offline, then all the rest of resources are taken offline by vcs. In the scenario with disk group resource goes offline ,and vcs is not able to take offline the application resource and mount point resource, what I should to perform? In this case the application hang? - I should use the application stop script and cleanup script or kill -9 in order to kill the application processes? - umount with force the mount point? - I should also stop the agents of those resources manually or they are stopped automatically when the resources are offline? If the above are successfully(application resource and mount point resource are offline), the group will start automatically on other node or I should to start it manually? tnx a lot, marius818Views1like1CommentReccuring unmount of Diskgroups
Hello Experts, After finding that some of our DGs were unmounted, i have executed VCS clear command on resources of DG which were faulted and then made them online after it. Eventually, DGs were mounted and mountpoints were present to OS. After 2-3 days, we have noticed that these DGs were unmounted due to failing resources again. 2013/05/03 05:25:52 VCS ERROR V-16-10001-1004 (EMMDPD03) CVMVxconfigd:???:monitor:Core file saved once already 2013/05/03 05:25:53 VCS INFO V-16-2-13001 (EMMDPD03) Resource(cvm_vxconfigd): Output of the completed operation (monitor) 2013/05/03 05:25:53 VCS ERROR V-16-2-13067 (EMMDPD03) Agent is calling clean for resource(cvm_vxconfigd) because the resource became OFFLINE unexpectedly, on its own. 2013/05/03 05:25:54 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cvm_vxconfigd) - clean completed successfully. 2013/05/03 05:25:54 VCS ERROR V-16-2-13073 (EMMDPD03) Resource(cvm_vxconfigd) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 5) the r esource. 2013/05/03 05:25:54 VCS WARNING V-16-10001-1006 (EMMDPD03) CVMVxconfigd:???:online:attempting to start vxconfigd 2013/05/03 05:26:13 VCS NOTICE V-16-2-13076 (EMMDPD03) Agent has successfully restarted resource(cvm_vxconfigd). 2013/05/03 11:59:12 VCS ERROR V-16-10001-1004 (EMMDPD03) CVMVxconfigd:???:monitor:Core file saved once already 2013/05/03 11:59:13 VCS INFO V-16-2-13001 (EMMDPD03) Resource(cvm_vxconfigd): Output of the completed operation (monitor) 2013/05/03 11:59:13 VCS ERROR V-16-2-13067 (EMMDPD03) Agent is calling clean for resource(cvm_vxconfigd) because the resource became OFFLINE unexpectedly, on its own. 2013/05/03 11:59:14 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cvm_vxconfigd) - clean completed successfully. 2013/05/03 11:59:14 VCS ERROR V-16-2-13073 (EMMDPD03) Resource(cvm_vxconfigd) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 5) the r esource. 2013/05/03 11:59:14 VCS WARNING V-16-10001-1006 (EMMDPD03) CVMVxconfigd:???:online:attempting to start vxconfigd 2013/05/03 11:59:32 VCS NOTICE V-16-2-13076 (EMMDPD03) Agent has successfully restarted resource(cvm_vxconfigd). 2013/05/03 14:05:48 VCS INFO V-16-1-50135 User root fired command: hagrp -flush ServerGroup1_DG EMMDPD03 from localhost 2013/05/03 14:06:27 VCS INFO V-16-1-50135 User root fired command: MSG_RES_PROBE cfsmount2 EMMDPD03 from localhost 2013/05/03 14:06:41 VCS INFO V-16-1-50135 User root fired command: hagrp -offline ServerGroup1_DG EMMDPD03 from localhost 2013/05/03 14:06:41 VCS NOTICE V-16-1-10167 Initiating manual offline of group ServerGroup1_DG on system EMMDPD03 2013/05/03 14:07:03 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2 EMMDPD03 from localhost 2013/05/03 14:07:03 VCS INFO V-16-1-10307 Resource cvmvoldg2 (Owner: unknown, Group: ServerGroup1_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:07:09 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg4 EMMDPD03 from localhost 2013/05/03 14:07:09 VCS INFO V-16-1-10307 Resource cvmvoldg4 (Owner: unknown, Group: ora_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:07:12 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg5 EMMDPD03 from localhost 2013/05/03 14:07:12 VCS INFO V-16-1-10307 Resource cvmvoldg5 (Owner: unknown, Group: lic_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:07:15 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg6 EMMDPD03 from localhost 2013/05/03 14:07:15 VCS INFO V-16-1-10307 Resource cvmvoldg6 (Owner: unknown, Group: lic_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:20:11 VCS INFO V-16-1-50135 User root fired command: hagrp -flush ServerGroup1_DG EMMDPD03 from localhost 2013/05/03 14:20:12 VCS INFO V-16-1-10306 Resource cvmvoldg2 (Owner: unknown, Group: ServerGroup1_DG) is offline on EMMDPD03 (Previous State = OFFLINE) 2013/05/03 14:28:05 VCS ERROR V-16-2-13027 (EMMDPD05) Resource(Server3) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:09 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(Server5) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:09 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(Server4) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:11 VCS ERROR V-16-2-13027 (EMMDPD03) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:11 VCS ERROR V-16-2-13210 (EMMDPD03) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not bash-3.00$ cat /var/VRTSvcs/log/engine_A.log | grep 2013/05/ | more 2013/05/02 06:47:26 VCS ERROR V-16-10001-1004 (EMMDPD03) CVMVxconfigd:???:monitor:Core file saved once already 2013/05/02 06:47:27 VCS INFO V-16-2-13001 (EMMDPD03) Resource(cvm_vxconfigd): Output of the completed operation (monitor) 2013/05/02 06:47:27 VCS ERROR V-16-2-13067 (EMMDPD03) Agent is calling clean for resource(cvm_vxconfigd) because the resource became OFFLINE unexpectedly, on its own. 2013/05/02 06:47:28 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cvm_vxconfigd) - clean completed successfully. 2013/05/02 06:47:28 VCS ERROR V-16-2-13073 (EMMDPD03) Resource(cvm_vxconfigd) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 5) the r esource. 2013/05/02 06:47:28 VCS WARNING V-16-10001-1006 (EMMDPD03) CVMVxconfigd:???:online:attempting to start vxconfigd 2013/05/02 06:47:51 VCS NOTICE V-16-2-13076 (EMMDPD03) Agent has successfully restarted resource(cvm_vxconfigd). 2013/05/03 05:25:52 VCS ERROR V-16-10001-1004 (EMMDPD03) CVMVxconfigd:???:monitor:Core file saved once already 2013/05/03 05:25:53 VCS INFO V-16-2-13001 (EMMDPD03) Resource(cvm_vxconfigd): Output of the completed operation (monitor) 2013/05/03 05:25:53 VCS ERROR V-16-2-13067 (EMMDPD03) Agent is calling clean for resource(cvm_vxconfigd) because the resource became OFFLINE unexpectedly, on its own. 2013/05/03 05:25:54 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cvm_vxconfigd) - clean completed successfully. 2013/05/03 05:25:54 VCS ERROR V-16-2-13073 (EMMDPD03) Resource(cvm_vxconfigd) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 5) the r esource. 2013/05/03 05:25:54 VCS WARNING V-16-10001-1006 (EMMDPD03) CVMVxconfigd:???:online:attempting to start vxconfigd 2013/05/03 05:26:13 VCS NOTICE V-16-2-13076 (EMMDPD03) Agent has successfully restarted resource(cvm_vxconfigd). 2013/05/03 11:59:12 VCS ERROR V-16-10001-1004 (EMMDPD03) CVMVxconfigd:???:monitor:Core file saved once already 2013/05/03 11:59:13 VCS INFO V-16-2-13001 (EMMDPD03) Resource(cvm_vxconfigd): Output of the completed operation (monitor) 2013/05/03 11:59:13 VCS ERROR V-16-2-13067 (EMMDPD03) Agent is calling clean for resource(cvm_vxconfigd) because the resource became OFFLINE unexpectedly, on its own. 2013/05/03 11:59:14 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cvm_vxconfigd) - clean completed successfully. 2013/05/03 11:59:14 VCS ERROR V-16-2-13073 (EMMDPD03) Resource(cvm_vxconfigd) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 5) the r esource. 2013/05/03 11:59:14 VCS WARNING V-16-10001-1006 (EMMDPD03) CVMVxconfigd:???:online:attempting to start vxconfigd 2013/05/03 11:59:32 VCS NOTICE V-16-2-13076 (EMMDPD03) Agent has successfully restarted resource(cvm_vxconfigd). 2013/05/03 14:05:48 VCS INFO V-16-1-50135 User root fired command: hagrp -flush ServerGroup1_DG EMMDPD03 from localhost 2013/05/03 14:06:27 VCS INFO V-16-1-50135 User root fired command: MSG_RES_PROBE cfsmount2 EMMDPD03 from localhost 2013/05/03 14:06:41 VCS INFO V-16-1-50135 User root fired command: hagrp -offline ServerGroup1_DG EMMDPD03 from localhost 2013/05/03 14:06:41 VCS NOTICE V-16-1-10167 Initiating manual offline of group ServerGroup1_DG on system EMMDPD03 2013/05/03 14:07:03 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2 EMMDPD03 from localhost 2013/05/03 14:07:03 VCS INFO V-16-1-10307 Resource cvmvoldg2 (Owner: unknown, Group: ServerGroup1_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:07:09 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg4 EMMDPD03 from localhost 2013/05/03 14:07:09 VCS INFO V-16-1-10307 Resource cvmvoldg4 (Owner: unknown, Group: ora_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:07:12 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg5 EMMDPD03 from localhost 2013/05/03 14:07:12 VCS INFO V-16-1-10307 Resource cvmvoldg5 (Owner: unknown, Group: lic_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:07:15 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg6 EMMDPD03 from localhost 2013/05/03 14:07:15 VCS INFO V-16-1-10307 Resource cvmvoldg6 (Owner: unknown, Group: lic_DG) is offline on EMMDPD03 (Not initiated by VCS) 2013/05/03 14:20:11 VCS INFO V-16-1-50135 User root fired command: hagrp -flush ServerGroup1_DG EMMDPD03 from localhost 2013/05/03 14:20:12 VCS INFO V-16-1-10306 Resource cvmvoldg2 (Owner: unknown, Group: ServerGroup1_DG) is offline on EMMDPD03 (Previous State = OFFLINE) 2013/05/03 14:28:05 VCS ERROR V-16-2-13027 (EMMDPD05) Resource(Server3) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:09 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(Server5) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:09 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(Server4) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:11 VCS ERROR V-16-2-13027 (EMMDPD03) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:11 VCS ERROR V-16-2-13210 (EMMDPD03) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not complete within the expected time. 2013/05/03 14:28:12 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cfsmount2) - clean completed successfully. 2013/05/03 14:28:17 VCS ERROR V-16-2-13027 (EMMDPD07) Resource(Server1) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:26 VCS ERROR V-16-2-13027 (EMMDPD04) Resource(Server2) - monitor procedure did not complete within the expected time. 2013/05/03 14:28:31 VCS ERROR V-16-2-13027 (EMMDPD04) Resource(fmmweb) - monitor procedure did not complete within the expected time. 2013/05/03 14:29:06 VCS ERROR V-16-2-13027 (EMMDPD07) Resource(licserv) - monitor procedure did not complete within the expected time. 2013/05/03 14:31:51 VCS ERROR V-16-2-13027 (EMMDPD05) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:31:51 VCS ERROR V-16-2-13210 (EMMDPD05) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not complete within the expected time. 2013/05/03 14:31:51 VCS INFO V-16-2-13026 (EMMDPD07) Resource(Server1) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:31:52 VCS ERROR V-16-2-13027 (EMMDPD04) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:31:52 VCS ERROR V-16-2-13210 (EMMDPD04) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not complete within the expected time. 2013/05/03 14:31:52 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:31:52 VCS ERROR V-16-2-13210 (EMMDPD06) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not complete within the expected time. 2013/05/03 14:31:52 VCS ERROR V-16-2-13027 (EMMDPD07) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:31:52 VCS ERROR V-16-2-13210 (EMMDPD07) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not complete within the expected time. 2013/05/03 14:31:52 VCS INFO V-16-2-13026 (EMMDPD04) Resource(fmmweb) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:31:52 VCS INFO V-16-2-13068 (EMMDPD05) Resource(cfsmount2) - clean completed successfully. 2013/05/03 14:31:52 VCS INFO V-16-2-13082 (EMMDPD05) Resource(cfsmount2) recovered from fault, on its own. 2013/05/03 14:31:53 VCS INFO V-16-2-13026 (EMMDPD04) Resource(Server2) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:31:53 VCS INFO V-16-2-13068 (EMMDPD04) Resource(cfsmount2) - clean completed successfully. 2013/05/03 14:31:53 VCS INFO V-16-2-13026 (EMMDPD06) Resource(Server5) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:31:53 VCS INFO V-16-2-13026 (EMMDPD06) Resource(Server4) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:31:53 VCS INFO V-16-2-13082 (EMMDPD04) Resource(cfsmount2) recovered from fault, on its own. 2013/05/03 14:31:53 VCS INFO V-16-2-13068 (EMMDPD06) Resource(cfsmount2) - clean completed successfully. 2013/05/03 14:31:53 VCS INFO V-16-2-13082 (EMMDPD06) Resource(cfsmount2) recovered from fault, on its own. 2013/05/03 14:31:53 VCS INFO V-16-2-13068 (EMMDPD07) Resource(cfsmount2) - clean completed successfully. 2013/05/03 14:31:53 VCS INFO V-16-2-13082 (EMMDPD07) Resource(cfsmount2) recovered from fault, on its own. 2013/05/03 14:31:53 VCS INFO V-16-2-13026 (EMMDPD05) Resource(Server3) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:32:13 VCS INFO V-16-2-13026 (EMMDPD03) Resource(cfsmount2) - monitor procedure finished successfully after failing to complete within the expected time f or (3) consecutive times. 2013/05/03 14:32:13 VCS INFO V-16-2-13082 (EMMDPD03) Resource(cfsmount2) recovered from fault, on its own. 2013/05/03 14:34:05 VCS ERROR V-16-2-13027 (EMMDPD05) Resource(Server3) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:10 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(Server5) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:10 VCS ERROR V-16-2-13027 (EMMDPD06) Resource(Server4) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:13 VCS ERROR V-16-2-13027 (EMMDPD03) Resource(cfsmount2) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:13 VCS ERROR V-16-2-13210 (EMMDPD03) Agent is calling clean for resource(cfsmount2) because 1 successive invocations of the monitor procedure did not complete within the expected time. 2013/05/03 14:34:14 VCS INFO V-16-2-13068 (EMMDPD03) Resource(cfsmount2) - clean completed successfully. 2013/05/03 14:34:16 VCS ERROR V-16-2-13027 (EMMDPD07) Resource(Server1) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:26 VCS ERROR V-16-2-13027 (EMMDPD04) Resource(Server2) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:30 VCS ERROR V-16-2-13027 (EMMDPD04) Resource(fmmweb) - monitor procedure did not complete within the expected time. 2013/05/03 14:34:40 VCS INFO V-16-1-10305 Resource cfsmount2 (Owner: unknown, Group: ServerGroup1_DG) is offline on EMMDPD03 (VCS initiated) 2013/05/03 14:34:40 VCS NOTICE V-16-1-10446 Group ServerGroup1_DG is offline on system EMMDPD03 2013/05/03 14:34:40 VCS INFO V-16-6-15002 (EMMDPD03) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/nfs_postoffline EMMDPD03 ServerGroup1_DG successfully 2013/05/03 14:34:40 VCS WARNING V-16-0 (EMMDPD03) hatrigger:POSTOFFLINE: EMMDPD03 ServerGroup1_DG 2013/05/03 14:34:41 VCS INFO V-16-6-15002 (EMMDPD03) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline EMMDPD03 ServerGroup1_DG successfully 2013/05/03 14:38:06 VCS INFO V-16-2-13026 (EMMDPD07) Resource(licserv) - monitor procedure finished successfully after failing to complete within the expected time for (2) consecutive times. 2013/05/03 14:42:43 VCS NOTICE V-16-1-10166 Initiating manual online of group ServerGroup1_DG on system EMMDPD03 2013/05/03 14:42:43 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ServerGroup1_DG on node EMMDPD03 2013/05/03 14:42:43 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ServerGroup1_DG on node EMMDPD04 2013/05/03 14:42:43 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ServerGroup1_DG on node EMMDPD05 2013/05/03 14:42:43 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ServerGroup1_DG on node EMMDPD06 2013/05/03 14:42:43 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ServerGroup1_DG on node EMMDPD07 2013/05/03 14:42:43 VCS NOTICE V-16-1-10301 Initiating Online of Resource cvmvoldg2 (Owner: unknown, Group: ServerGroup1_DG) on System EMMDPD03 2013/05/03 14:42:43 VCS WARNING V-16-20007-1074 (EMMDPD03) CVMVolDg:cvmvoldg2:online:setup_vxnotify: old vxnotify of pid 11084 will be killed. my pid is 21744 2013/05/03 14:42:45 VCS INFO V-16-20007-1046 (EMMDPD03) CVMVolDg:cvmvoldg2:online:resource cvmvoldg2 is online 2013/05/03 14:43:17 VCS INFO V-16-1-10298 Resource cvmvoldg2 (Owner: unknown, Group: ServerGroup1_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:43:17 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount2 (Owner: unknown, Group: ServerGroup1_DG) on System EMMDPD03 2013/05/03 14:43:40 VCS INFO V-16-20011-5506 (EMMDPD03) CFSMount:cfsmount2:online:CFSMOUNT Onlined :MountOptions : -F vxfs -o cluster,cluster Block Device /d ev/vx/dsk/bgw1dg/vol01 : MountPoint /var/opt/BGw/ServerGroup1 2013/05/03 14:43:41 VCS INFO V-16-1-10298 Resource cfsmount2 (Owner: unknown, Group: ServerGroup1_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:43:41 VCS NOTICE V-16-1-10447 Group ServerGroup1_DG is online on system EMMDPD03 2013/05/03 14:44:44 VCS NOTICE V-16-1-10166 Initiating manual online of group lic_DG on system EMMDPD03 2013/05/03 14:44:44 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group lic_DG on node EMMDPD03 2013/05/03 14:44:44 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group lic_DG on node EMMDPD04 2013/05/03 14:44:44 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group lic_DG on node EMMDPD05 2013/05/03 14:44:44 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group lic_DG on node EMMDPD06 2013/05/03 14:44:44 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group lic_DG on node EMMDPD07 2013/05/03 14:44:44 VCS NOTICE V-16-1-10301 Initiating Online of Resource cvmvoldg5 (Owner: unknown, Group: lic_DG) on System EMMDPD03 2013/05/03 14:44:44 VCS NOTICE V-16-1-10301 Initiating Online of Resource cvmvoldg6 (Owner: unknown, Group: lic_DG) on System EMMDPD03 2013/05/03 14:44:44 VCS WARNING V-16-20007-1074 (EMMDPD03) CVMVolDg:cvmvoldg5:online:setup_vxnotify: old vxnotify of pid 28916 will be killed. my pid is 13614 2013/05/03 14:44:44 VCS WARNING V-16-20007-1074 (EMMDPD03) CVMVolDg:cvmvoldg6:online:setup_vxnotify: old vxnotify of pid 1860 will be killed. my pid is 13654 2013/05/03 14:44:47 VCS INFO V-16-20007-1046 (EMMDPD03) CVMVolDg:cvmvoldg5:online:resource cvmvoldg5 is online 2013/05/03 14:44:47 VCS INFO V-16-20007-1046 (EMMDPD03) CVMVolDg:cvmvoldg6:online:resource cvmvoldg6 is online 2013/05/03 14:45:19 VCS INFO V-16-1-10298 Resource cvmvoldg6 (Owner: unknown, Group: lic_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:45:19 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount6 (Owner: unknown, Group: lic_DG) on System EMMDPD03 2013/05/03 14:45:19 VCS INFO V-16-1-10298 Resource cvmvoldg5 (Owner: unknown, Group: lic_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:45:19 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount5 (Owner: unknown, Group: lic_DG) on System EMMDPD03 2013/05/03 14:45:19 VCS INFO V-16-20011-5506 (EMMDPD03) CFSMount:cfsmount5:online:CFSMOUNT Onlined :MountOptions : -F vxfs -o cluster,cluster Block Device /d ev/vx/dsk/lic1dg/vol01 : MountPoint /var/opt/sentinel 2013/05/03 14:45:19 VCS INFO V-16-20011-5506 (EMMDPD03) CFSMount:cfsmount6:online:CFSMOUNT Onlined :MountOptions : -F vxfs -o cluster,cluster Block Device /d ev/vx/dsk/fmm1dg/vol01 : MountPoint /var/opt/mediation/fmmdb 2013/05/03 14:45:20 VCS INFO V-16-1-10298 Resource cfsmount6 (Owner: unknown, Group: lic_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:45:20 VCS INFO V-16-1-10298 Resource cfsmount5 (Owner: unknown, Group: lic_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:45:20 VCS NOTICE V-16-1-10447 Group lic_DG is online on system EMMDPD03 2013/05/03 14:45:48 VCS NOTICE V-16-1-10166 Initiating manual online of group ora_DG on system EMMDPD03 2013/05/03 14:45:48 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ora_DG on node EMMDPD03 2013/05/03 14:45:48 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ora_DG on node EMMDPD04 2013/05/03 14:45:48 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ora_DG on node EMMDPD05 2013/05/03 14:45:48 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ora_DG on node EMMDPD06 2013/05/03 14:45:48 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group ora_DG on node EMMDPD07 2013/05/03 14:45:48 VCS NOTICE V-16-1-10301 Initiating Online of Resource cvmvoldg4 (Owner: unknown, Group: ora_DG) on System EMMDPD03 2013/05/03 14:45:48 VCS WARNING V-16-20007-1074 (EMMDPD03) CVMVolDg:cvmvoldg4:online:setup_vxnotify: old vxnotify of pid 16612 will be killed. my pid is 26115 2013/05/03 14:45:50 VCS INFO V-16-20007-1046 (EMMDPD03) CVMVolDg:cvmvoldg4:online:resource cvmvoldg4 is online 2013/05/03 14:46:22 VCS INFO V-16-1-10298 Resource cvmvoldg4 (Owner: unknown, Group: ora_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:46:22 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount4 (Owner: unknown, Group: ora_DG) on System EMMDPD03 2013/05/03 14:46:22 VCS INFO V-16-20011-5506 (EMMDPD03) CFSMount:cfsmount4:online:CFSMOUNT Onlined :MountOptions : -F vxfs -o cluster,cluster Block Device /d ev/vx/dsk/ora1dg/vol01 : MountPoint /var/opt/mediation/ora 2013/05/03 14:46:23 VCS INFO V-16-1-10298 Resource cfsmount4 (Owner: unknown, Group: ora_DG) is online on EMMDPD03 (VCS initiated) 2013/05/03 14:46:23 VCS NOTICE V-16-1-10447 Group ora_DG is online on system EMMDPD03 2013/05/03 15:03:18 VCS ERROR V-16-20007-1017 (EMMDPD03) CVMVolDg:cvmvoldg2:monitor:check_notify_status: can't stabilise vxstat 2013/05/03 15:25:22 VCS ERROR V-16-20007-1017 (EMMDPD03) CVMVolDg:cvmvoldg5:monitor:check_notify_status: can't stabilise vxstat 2013/05/03 15:25:22 VCS ERROR V-16-20007-1017 (EMMDPD03) CVMVolDg:cvmvoldg6:monitor:check_notify_status: can't stabilise vxstat 2013/05/03 15:26:21 VCS ERROR V-16-20007-1017 (EMMDPD03) CVMVolDg:cvmvoldg2:monitor:check_notify_status: can't stabilise vxstat 2013/05/03 15:26:22 VCS ERROR V-16-20007-1017 (EMMDPD03) CVMVolDg:cvmvoldg5:monitor:check_notify_status: can't stabilise vxstat 2013/05/03 15:27:23 VCS ERROR V-16-20007-1017 (EMMDPD03) CVMVolDg:cvmvoldg6:monitor:check_notify_status: can't stabilise vxstat May i ask what is causing it? When checking on the engine logs, resources are failing with out giving a root causeSolved2KViews0likes3Comments
R12: XML Reports end with WARNING Status, Using Virtual Hostname by Veritas HA
Dear Experts, We are using the Veritas HA cluster to configure our ERP Application on Virtual Hostname. With Release 11i every thing was working fine, We have upgraded our Application from 11.5.10.2 to R12, 12.1.3. Now in the Upgraded Application we are unable to open/view any of the XML based Concurrent Reports. All the XML Concurrent Reports are ending in Status WARNING. When we click on the View Out put, we are just getting the XML code and not the XML based reports. The log file of Out Put Postprocessor shows the following errors: ================================================ 5/28/13 12:40:20 PM] [STATEMENT] [960882:RT4152834] Get Output Type [5/28/13 12:40:20 PM] [STATEMENT] [960882:RT4152834] XML file name: /mnt/oracle/ERPROD/inst/apps/ERPROD_erp-lh-dr/logs/appl/conc/out/o4152834.out [5/28/13 12:40:20 PM] [STATEMENT] [960882:RT4152834] XML file is on node: ERP-LH-DR [5/28/13 12:40:20 PM] [UNEXPECTED] [960882:RT4152834] java.sql.SQLException: Exhausted Resultset at oracle.jdbc.driver.OracleResultSetImpl.getString(OracleResultSetImpl.java:1224) at oracle.apps.fnd.cp.util.CpUtil.getCanonicalLocalNode(CpUtil.java:127) at oracle.apps.fnd.cp.opp.XMLPublisherProcessor.process(XMLPublisherProcessor.java:244) at oracle.apps.fnd.cp.opp.OPPRequestThread.run(OPPRequestThread.java:176) [5/28/13 12:40:20 PM] [960882:RT4152834] Completed post-processing actions for request 4152834. And error like below: The error we are getting from the Application log files are related to Hostname. Following is the abstract from the Output Post Processor log. [5/29/13 2:56:02 PM] [UNEXPECTED] [960900:RT4155505] java.sql.SQLException: Exhausted Resultset at oracle.jdbc.driver.OracleResultSetImpl.getString(OracleResultSetImpl.java:1224) at oracle.apps.fnd.cp.util.CpUtil.getCanonicalLocalNode(CpUtil.java:127) at oracle.apps.fnd.cp.opp.XMLPublisherProcessor.process(XMLPublisherProcessor.java:244) at oracle.apps.fnd.cp.opp.OPPRequestThread.run(OPPRequestThread.java:176) =============================================================== Concurrent Manger Log files shows the following error: =========================================================== --------------------------------------------------------------------------- Executing request completion options... Output file size: 430455 ------------- 1) PUBLISH ------------- Beginning post-processing of request 4158856 on node ERP-LH-DR at 30-MAY-2013 18:53:38. Post-processing of request 4158856 failed at 30-MAY-2013 18:53:38 with the error message: One or more post-processing actions failed. Consult the OPP service log for details. -------------------------------------- ------------- 2) PRINT ------------- Not printing the output of this request because post-processing failed. -------------------------------------- Finished executing request completion options. --------------------------------------------------------------------------- Concurrent request completed ========================================================== Please suggests: 1- Is there any way to make these XML reports work with Veritas HA, using Virtual hostname 2- Can we identify or pin point from where the Out Put Postprocessor is setting the Physical Hostname. Regards Ali Ammar6.6KViews1like6Comments