cancel
Showing results for 
Search instead for 
Did you mean: 

Oracle RAC vote disk cannot be discover on 11.2.0.3 with Solaris 10u10 and SFRAC 6.0.1

stinsong
Level 5

Hi community,

My client met a problem that Oracle RAC vote disk cannot be discover which caused OS keeps rebooting frequently. Oracle has already reply there is a known issue on SFRAC 5.1. But my client is using Oracle RAC 11.2.0.3 with Solaris 10u10 and SFRAC 6.0.1.

Anyone has any clue about this ?

The Grid Alert log shows:

Oracle Grid Alert Log

2013-07-24 13:01:24.186
[ohasd(4929)]CRS-8011:reboot advisory message from host: xhdb-server3, component: cssmonit, with time stamp: L-2013-07-24-11:19:48.012
[ohasd(4929)]CRS-8013:reboot advisory message text: clsnvmon_main: error registering in skgxn rc 3
2013-07-24 13:01:24.210
[ohasd(4929)]CRS-8011:reboot advisory message from host: xhdb-server3, component: cssagent, with time stamp: L-2013-07-24-12:21:07.693
[ohasd(4929)]CRS-8013:reboot advisory message text: clsnvmon_main: error registering in skgxn rc 3
2013-07-24 13:01:24.211
[ohasd(4929)]CRS-8017:location: /var/opt/oracle/lastgasp has 2 reboot advisory log files, 2 were announced and 0 errors occurred
2013-07-24 13:01:43.850
[/u01/app/11.2.0/grid/bin/orarootagent.bin(5084)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/acfsload" spawned by agent "/u01/app/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/xhdb-server3/agent/ohasd/orarootagent_root/orarootagent_root.log"
2013-07-24 13:01:48.976
[ohasd(4929)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running). 
2013-07-24 13:01:49.139
[gpnpd(6273)]CRS-2328:GPNPD started on node xhdb-server3. 
2013-07-24 13:01:52.838
[cssd(6299)]CRS-1713:CSSD daemon is started in clustered mode
2013-07-24 13:01:57.792
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:03.559
[ohasd(4929)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2013-07-24 13:02:12.944
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:28.099
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:43.248
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:58.396
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:03:13.544
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:03:35.359
[cssd(6299)]CRS-1707:Lease acquisition for node xhdb-server3 number 1 completed
2013-07-24 13:03:47.951
[cssd(6299)]CRS-1605:CSSD voting file is online: /ocr2/vdsk; details in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log.
ocssd.log
2013-07-24 13:02:43.247: [ GPNP][6]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2234] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD_xhdb-server3" disco ""
2013-07-24 13:02:43.248: [ CSSD][6]clssnmReadDiscoveryProfile: voting file discovery string(/ocr2/vdsk)
2013-07-24 13:02:43.248: [ CSSD][6]clssnmvDDiscThread: using discovery string /ocr2/vdsk for initial discovery 
2013-07-24 13:02:43.248: [ SKGFD][6]Discovery with str:/ocr2/vdsk:
2013-07-24 13:02:43.248: [ SKGFD][6]UFS discovery with :/ocr2/vdsk:
2013-07-24 13:02:43.248: [ SKGFD][6]OSS discovery with :/ocr2/vdsk:
2013-07-24 13:02:43.248: [ CSSD][6]clssnmvDiskVerify: Successful discovery of 0 disks
2013-07-24 13:02:43.248: [ CSSD][6]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2013-07-24 13:02:43.248: [ CSSD][6]clssnmvFindInitialConfigs: No voting files found
2013-07-24 13:02:43.249: [ CSSD][6](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds
2013-07-24 13:02:43.635: [ CSSD][5]clssscSelect: cookie accept request 100881af0
2013-07-24 13:02:43.636: [ CSSD][5]clssgmAllocProc: (10115de50) allocated
2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: properties of cmProc 10115de50 - 1,2,3,4,5
2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: Connect from con(61d) proc(10115de50) pid(8485) version 11:2:1:4, properties: 1,2,3,4,5
2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: msg flags 0x0000
2013-07-24 13:02:45.166: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:45.167: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:45.167: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(1/100a3b2d0)
2013-07-24 13:02:45.168: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:45.169: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 643
2013-07-24 13:02:46.175: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:46.175: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:46.175: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(2/100a3b2d0)
2013-07-24 13:02:46.176: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:46.176: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 659
2013-07-24 13:02:47.183: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:47.183: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:47.183: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(3/100a3b2d0)
2013-07-24 13:02:47.184: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:47.184: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 66f
2013-07-24 13:02:47.551: [ CSSD][5]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(100d26790) client(0)
2013-07-24 13:02:47.555: [ CSSD][5]clssgmDeadProc: proc 100d26790
2013-07-24 13:02:47.555: [ CSSD][5]clssgmDestroyProc: cleaning up proc(100d26790) con(59f) skgpid ospid 6284 with 0 clients, refcount 0
2013-07-24 13:02:47.555: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 59f
2013-07-24 13:02:47.576: [ CSSD][5]clssscSelect: cookie accept request 100881af0
2013-07-24 13:02:47.576: [ CSSD][5]clssgmAllocProc: (100d26610) allocated
2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: properties of cmProc 100d26610 - 1,2,3,4,5
2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: Connect from con(6c0) proc(100d26610) pid(6284) version 11:2:1:4, properties: 1,2,3,4,5
2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: msg flags 0x0000
2013-07-24 13:02:48.190: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:48.191: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:48.191: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(4/100a3b2d0)
2013-07-24 13:02:48.192: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:48.192: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 6e6
2013-07-24 13:02:48.627: [ CSSD][5]clssscSelect: cookie accept request 10115de50
2013-07-24 13:02:48.627: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 10115de50
2013-07-24 13:02:48.627: [ CSSD][5]clssgmRegisterClient: proc(5/10115de50), client(1/100a3b2d0)
2013-07-24 13:02:48.628: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(10115de50) client(100a3b2d0)
2013-07-24 13:02:48.629: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 6fc

 

Any thoughs could be helpful is appreciate !

3 REPLIES 3

stinsong
Level 5

Ooh, foget to tell that my client has already confirmed about the permission of vote volume and disks is right.

rsharma1
Level 5
Employee Accredited Certified

Hi Stinsong,

                                 from the msg "error registering in skgxn rc 3 2013-07-24 13:01:24.211"

seems like some issue with node membership communication between VCS and Oracle grid. Could you check if the vcsmm port - port "o" is showing up as joined in gabconfig -a o/p for all nodes? If not, probably starting vcsmm manully might help..

 

                      

gaurav_dong
Level 3
Employee Certified

"gabconfig -a" from all the nodes will be helpful.

is the vote disk a veritas volume manager volume ?

Also the main.cf file if you can.

 

Gaurav D