ORA-600 [ksfd_odmcrt8], [ODM ERROR V-41-4-1-105-22 Invalid argument]
hi.. when we open db (Oracle 11.2.0.4 2node RAC in HP), we got the error... Sun Jun 01 23:19:24 2014 ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=xxx.xxx.xxx.xxx)(PORT=1521))' SCOPE=MEMORY SID='MNP001'; ALTER DATABASE MOUNT /* db agent *//* {1:15272:181} */ This instance was first to mount Errors in file /oracle/MNP/saptrace/diag/rdbms/mnp/MNP001/trace/MNP001_ckpt_230.trc (incident=432209): ORA-00600: internal error code, arguments: [ksfd_odmopn2], [ODM ERROR V-41-4-2-313-22 Invalid argument], [], [], [], [], [], [], [], [], [], [] Incident details in: /oracle/MNP/saptrace/diag/rdbms/mnp/MNP001/incident/incdir_432209/MNP001_ckpt_230_i432209.trc [PROD spncpd01:/#] mount -p ....................... /dev/vg00/lvol10 /maxigent50 vxfs ioerror=mwdisable,largefiles,qio,delaylog,dev=4000000a 0 0 /dev/vg00/lvol11 /SIMPANA vxfs ioerror=mwdisable,largefiles,qio,delaylog,dev=4000000b 0 0 xx.xx.89.167:/snctrans /usr/sap/trans nfs rsize=32768,wsize=32768,NFSv3,dev=4000002 0 0 xx.xx.89.167:/sncmnp /sapmnt/MNP nfs rsize=32768,wsize=32768,NFSv3,dev=4000003 0 0 xx.xx.89.167:/sapimage /sapcd nfs rsize=32768,wsize=32768,NFSv3,dev=4000004 0 0 -hosts /net autofs ignore,indirect,nosuid,soft,nobrowse,dev=4000005 0 0 /dev/odm /dev/odm odm smartsync,dev=4000006 0 0 /dev/vx/dsk/ncpdg1/mnpomnp /oracle/MNP vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=20059d8 0 0 /dev/vx/dsk/ncpdg1/mnpstage /oracle/stage/112_64 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=20059d9 0 0 /dev/vx/dsk/ncpdg1/mnpcli /oracle/client vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=20059da 0 0 /dev/vx/dsk/ncpdg3/mnpocr1 /oracle/OCR1 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=2009858 0 0 /dev/vx/dsk/ncpdg3/mnpvote1 /oracle/VOTE1 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=2009859 0 0 /dev/vx/dsk/ncpdg4/mnpocr2 /oracle/OCR2 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=200c350 0 0 /dev/vx/dsk/ncpdg4/mnpvote2 /oracle/VOTE2 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=200c351 0 0 /dev/vx/dsk/ncpdg5/mnpocr3 /oracle/OCR3 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=200e678 0 0 /dev/vx/dsk/ncpdg5/mnpvote3 /oracle/VOTE3 vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=200e679 0 0 /dev/vx/dsk/ncpdg1/mnploga /oracle/MNP/origlogA vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=20059db 0 0 /dev/vx/dsk/ncpdg1/mnplogb /oracle/MNP/origlogB vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=20059dc 0 0 /dev/vx/dsk/ncpdg1/mnpmloga /oracle/MNP/mirrlogA vxfs ioerror=mdisable,largefiles,qio,cluster,crw,delaylog,dev=20059dd 0 0 <--- no using CIO so, we can not use veritas odm library... could you teach me, how we can use veritas odm library ??? ( Is there any issue in veritas / Oracle ?? ) Thanks~~~~1.4KViews0likes5CommentsWhat's the messages of cvm agent.
Hello everyone. My customer have CFS 5.0.1 on hp-ux 11.31. I saw the VCS engine log. Here's the messages : ======================================================================= $ tail -f /var/VRTSvcs/log/en*A* 2010/08/25 09:59:04 VCS INFO V-16-2-13001 (oskrap1p) Resource(cvmvoldg1): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CVMVolDg/ monitor[31]: -: not found. 2010/08/25 09:59:22 VCS INFO V-16-2-13001 (oskrap2p) Resource(cvmvoldg1): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CVMVolDg/ monitor[31]: -: not found. 2010/08/25 10:00:05 VCS INFO V-16-2-13001 (oskrap1p) Resource(cvmvoldg1): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CVMVolDg/ monitor[31]: -: not found. 2010/08/25 10:00:22 VCS INFO V-16-2-13001 (oskrap2p) Resource(cvmvoldg1): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CVMVolDg/ monitor[31]: -: not found. 2010/08/25 10:01:05 VCS INFO V-16-2-13001 (oskrap1p) Resource(cvmvoldg1): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CVMVolDg/ monitor[31]: -: not found. ======================================================================= What is mean the "/opt/VRTSvcs/bin/CVMVolDg/ monitor[31]: -: not found." ? I appreciate if any of you can offer me some advice.Solved1.2KViews0likes4CommentsCan members of a bonded-NIC be configured as VCS LLT links?
Scenario-1: SFRAC 5.0.1and aboveļ¼such as 5.1, 5.1SP1, 6.0, 6.1 ...... HPUX 11.31-0909 Oracle 10.2.0.4 RAC lan3 and lan4 are bonded to be lan901, which is configured as Oracle RAC heartbeat. lan3 and lan4 have different MAC address. I know if the bonded NIC lan901 is configured as a VCS LLT, then the members of lan901, lan3 and lan4, cannot be configured as VCS LLT Links. But, Ifthe bonded NIC lan901 is NOT configured as a VCS LLT,Can lan3 and lan4 be configured as VCS primary and secondary LLT links? ==================================================== Scenario-2: SFRAC Redhat Linux Oracle RAC eth3 and eth4 are bonded to be bond1,which is configured as Oracle RAC heartbeat. eth3 and eth4 and bond1 have the same MAC address. Can eth3 and eth4 be configured as VCS primary and secondary LLT links?Solved1KViews1like2CommentsWhat's the messages ?
hello. I have SFCFS5.0 on hpux 11.31 I saw messages : /opt/VRTSvcs/bin/CFSMount/monitor[47]: 4657 Killed vx_ha_read[37]: 4669 Killed What's mean the "4669 Killed" ? VCS engine_A.log =================================================================================== skip ... 2010/11/18 17:33:01 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount3): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CFSMount/monitor[47]: 4657 Killed vx_ha_read[37]: 4669 Killed /opt/VRTSvcs/bin/CFSMount/monitor[55]: 4676 Killed 2010/11/18 17:33:01 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount4): Output of the completed operation (monitor) /opt/VRTSvcs/bin/CFSMount/monitor[47]: 4675 Killed 2010/11/18 17:33:01 VCS ERROR V-16-2-13067 (eachap2t) Agent is calling clean for resource(cfsmount4) because the resource became OFF LINE unexpectedly, on its own. 2010/11/18 17:33:01 VCS ERROR V-16-2-13067 (eachap2t) Agent is calling clean for resource(cfsmount3) because the resource became OFF LINE unexpectedly, on its own. 2010/11/18 17:33:02 VCS NOTICE V-16-10021-5511 (eachap2t) CFSMount:cfsmount4:clean:clean unexpected offline : Mount Point : /nbsftp 2010/11/18 17:33:02 VCS NOTICE V-16-10021-5511 (eachap2t) CFSMount:cfsmount3:clean:clean unexpected offline : Mount Point : /nbsrun 2010/11/18 17:33:05 VCS NOTICE V-16-10021-5504 (eachap2t) CFSMount:cfsmount4:clean:Force Umount : Mount Point : /nbsftp 2010/11/18 17:33:06 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount4): Output of the completed operation (clean) /opt/VRTSvcs/bin/CFSMount/clean[106]: test: argument expected 2010/11/18 17:33:06 VCS INFO V-16-2-13068 (eachap2t) Resource(cfsmount4) - clean completed successfully. 2010/11/18 17:33:07 VCS NOTICE V-16-10021-5504 (eachap2t) CFSMount:cfsmount3:clean:Force Umount : Mount Point : /nbsrun 2010/11/18 17:33:07 VCS INFO V-16-2-13001 (eachap2t) Resource(cfsmount3): Output of the completed operation (clean) /opt/VRTSvcs/bin/CFSMount/clean[106]: test: argument expected 2010/11/18 17:33:07 VCS INFO V-16-2-13068 (eachap2t) Resource(cfsmount3) - clean completed successfully. 2010/11/18 17:33:07 VCS INFO V-16-1-10307 Resource cfsmount4 (Owner: unknown, Group: cfs) is offline on eachap2t (Not initiated by V CS) 2010/11/18 17:33:08 VCS INFO V-16-6-15004 (eachap2t) hatrigger:Failed to send trigger for resfault; script doesn't exist 2010/11/18 17:33:08 VCS INFO V-16-1-10307 Resource cfsmount3 (Owner: unknown, Group: cfs) is offline on eachap2t (Not initiated by V CS) 2010/11/18 17:33:09 VCS INFO V-16-6-15004 (eachap2t) hatrigger:Failed to send trigger for resfault; script doesn't exist skip ... SYSTEM LOG ================================================================================== skip ... Nov 18 14:40:52 eachap2t su: + 19 root-jeuss Nov 18 16:30:54 eachap2t mountd[4846]: Cannot stat /etc/dfs/sharetab: No such file or directory Nov 18 17:33:01 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13067 Thread(5) Agent is calling clean f or resource(cfsmount4) because the resource became OFFLINE unexpectedly, on its own. Nov 18 17:33:01 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13067 Thread(3) Agent is calling clean f or resource(cfsmount3) because the resource became OFFLINE unexpectedly, on its own. Nov 18 17:33:01 eachap2t Had[13145]: VCS ERROR V-16-1-13067 (eachap2t) Agent is calling clean for resourc e(cfsmount4) because the resource became OFFLINE unexpectedly, on its own. Nov 18 17:33:01 eachap2t Had[13145]: VCS ERROR V-16-1-13067 (eachap2t) Agent is calling clean for resourc e(cfsmount3) because the resource became OFFLINE unexpectedly, on its own. Nov 18 17:33:05 eachap2t vmunix: Phase 9 - /dev/vx/dsk/eachapvg11/lvol4 - Set Primary nodeid to 0 Nov 18 17:33:06 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13068 Thread(5) Resource(cfsmount4) - cl ean completed successfully. Nov 18 17:33:06 eachap2t vmunix: Phase 9 - /dev/vx/dsk/eachapvg11/lvol3 - Set Primary nodeid to 0 Nov 18 17:33:07 eachap2t AgentFramework[13154]: VCS ERROR V-16-1-13068 Thread(3) Resource(cfsmount3) - cl ean completed successfully. Nov 18 22:30:54 eachap2t mountd[4846]: Cannot stat /etc/dfs/sharetab: No such file or directorySolved978Views0likes3CommentsGetting Application Online with correct environment variables
Hello, I'm running Veritas Storage Foundation Cluster 5.0 on HP-UX 11.23. I'm using veritas to start application using Application agent. My script is executed as normal user (not root) and the script starts another java application. Problem is that when I start it from veritas it does not get all the correct environment variables. I can see that the command in online script is su - <user> -c <script>. When I login using the user script works just fine. Also if I execute it as root using su <user> -c script works. My question is that what is the user when veritas start to execute online script? Also I'd like to know is it possible to execute that script without su command because that seems to cause problems in environment variables. br, JP ****EDIT****** Post moved to Cluster Server Forum by Marianne van den BergSolved906Views0likes3CommentsHow does VCS check IF different heartbeat NICs are in same network?
Cluster-A syslog shows: Jun 23 07:22:11 IMDB1 vmunix: LLT WARNING V-14-1-10498 recvarpreq cross links? links 1 and 0 of node 1 connected to same network? Jun 23 07:27:11 IMDB1 vmunix: LLT WARNING V-14-1-10498 recvarpreq cross links? links 1 and 0 of node 1 connected to same network? Jun 23 07:32:11 IMDB1 vmunix: LLT WARNING V-14-1-10498 recvarpreq cross links? links 1 and 0 of node 1 connected to same network? Jun 23 07:57:11 IMDB1 vmunix: LLT WARNING V-14-1-10498 recvarpreq cross links? links 1 and 0 of node 1 connected to same network? Jun 23 08:12:11 IMDB1 vmunix: LLT WARNING V-14-1-10498 recvarpreq cross links? links 1 and 0 of node 1 connected to same network? Jun 23 08:17:11 IMDB1 vmunix: LLT WARNING V-14-1-10498 recvarpreq cross links? links 0 and 1 of node 1 connected to same network? ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The heartbeats NICs are lan901 and lan902. I know VCS heatbeats are broadcast packets.and LLT on each system in the cluster sends heartbeat packets out on all configured LLT interfaces every half second. My questions are: 1) How, by what mechanism, does VCS check IF different heartbeat NICs are in same network (VLAN)? To which kind of address (MAC or OS device names) does the check signals send? 2) How often does it check if NICs are in same VLAN? 3) We have other systems Cluster-B and Cluster-C, they have almost the same network configurations (To be verified), but their syslogs do not have these kind of warnings.Why? Thanks in advance!Solved882Views0likes2Commentsvmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit
Hi, Everyday for the same time, we getting on the syslog below error, Kindly please anyone suggets how to resolve ? Aug 21 21:50:09 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 21 21:50:09 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 22 02:10:05 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 22 02:10:05 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 22 02:35:09 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 22 02:35:09 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 23 14:14:11 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 23 14:14:11 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 23 16:28:11 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 23 16:28:11 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 23 17:16:11 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 23 17:16:12 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 24 09:03:12 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 24 09:03:12 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 24 19:21:13 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 24 20:28:13 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 24 22:31:22 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 25 00:25:13 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 25 00:25:13 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 25 00:58:13 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 25 00:58:13 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 25 17:21:14 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 25 17:21:15 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 26 03:06:15 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 26 03:06:15 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 26 22:31:16 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 01:40:18 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 01:40:18 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 07:30:15 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 07:30:15 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 08:35:19 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 08:35:19 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 15:08:19 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 15:08:19 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 22:40:19 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 28 22:40:19 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 29 15:52:20 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 29 15:52:20 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 29 17:45:20 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 30 01:30:21 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit Aug 30 01:30:21 l165ux12 vmunix: LLT INFO V-14-1-10063 llt_send_port: no memory to xmit780Views0likes2CommentsLHC : IpmHandle::recv _read_errno is 5
Strange Warning in engine_A log at system startup... Background: HP-UX 11i with VCS 5.0.10.2, 3-node cluster. Node 0 in the cluster was rebooted this morning. All processes started normally, node 0 joined the cluster, went from CURRENT_DISCOVER_WAIT to REMOTE_BUILD; System: <node-0> is building configuration from system: <node-1> 2010/06/30 09:14:21 VCS NOTICE V-16-1-10465 Getting snapshot. AutoRestart for all service groups checked: 2010/06/30 09:14:29 VCS NOTICE V-16-1-10181 Group <group1> AutoRestart set to 1 2010/06/30 09:14:30 VCS NOTICE V-16-1-10181 Group <group2> AutoRestart set to 1 (lots of service groups...) Then this warning appears (literally HUNDREDS of lines, all the same; even timestamp): 2010/06/30 09:15:48 VCS WARNING V-16-1-10638 IpmHandle::recv _read_errno is 5. Client (hastatus) Pid (2654) 2010/06/30 09:15:48 VCS WARNING V-16-1-10638 IpmHandle::recv _read_errno is 5. Client (hastatus) Pid (2654) .... 2010/06/30 09:15:58 VCS INFO V-16-1-10466 End of snapshot received from node: 1. snapped_membership: 0x7 current_membership: 0x7 current_jeopardy_membership: 0x0 2010/06/30 09:15:58 VCS WARNING V-16-1-10030 UseFence=NONE. Hence do not need fencing 2010/06/30 09:15:59 VCS NOTICE V-16-1-10467 Replaying broadcast queue. snapped_membership: 0x7 current_membership: 0x7 current_jeopardy_membership: 0x0 2010/06/30 09:15:59 VCS NOTICE V-16-1-10322 System .... (Node '0') changed state from REMOTE_BUILD to RUNNING Service Groups start going online normally. I found a TechNote with similar warining, but the 'Client' in the TN is IPMultiNICB where in this instance the 'Client' is hastatus. No other resemblance to the TN. Has anyone seen this? Any idea what's causing it?745Views0likes4Comments