07-25-2017 01:57 AM - edited 07-25-2017 04:00 AM
Hello friends,
I noticed that disks are showing online but both pasths are disabled.
due to which
017/07/24 17:00:08 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount9 (Owner: Unspecified, Group: vrts_vea_cfs_int_cfsmount9) on System oracleprod_02
2017/07/24 17:00:08 VCS WARNING V-16-20011-5502 (oracleprod_02) CFSMount:cfsmount9:online:Mount Failed on this Node MountOptions -t vxfs -o cluster,cluster,mntlock=VCS Block Device /dev/vx/dsk/sfsdg/oss1a-pm_storage : MountPoint /vx/oss1a-pm_storage
2017/07/24 17:00:08 VCS WARNING V-16-20011-5508 (oracleprod_02) CFSMount:cfsmount9:online:Mount Error : UX:vxfs mount.vxfs: ERROR: V-3-20005: read of super-block on /dev/vx/dsk/sfsdg/oss1a-pm_storage failed: Input/output error
UX:vxfs mount.vxfs: ERROR: V-3-24996: Unable to get disk layout version
oracleprod2:~ # vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
emc_clariion0_6 auto:cdsdisk - - online sdf lun RAID_5
emc_clariion0_7 auto:cdsdisk - - online sdg lun RAID_5
emc_clariion0_8 auto:cdsdisk - - online sdh lun RAID_5
emc_clariion0_10 auto:cdsdisk emc_clariion0_10 sfsdg online shared sdi lun RAID_5
emc_clariion0_11 auto:cdsdisk emc_clariion0_11 sfsdg online shared sdj lun RAID_5
emc_clariion0_12 auto:cdsdisk emc_clariion0_12 sfsdg online shared sdk lun RAID_5
emc_clariion0_13 auto:cdsdisk emc_clariion0_13 sfsdg online shared sdl lun RAID_5
emc_clariion0_14 auto:cdsdisk emc_clariion0_14 sfsdg online shared sdm lun RAID_5
emc_clariion0_15 auto:cdsdisk emc_clariion0_15 sfsdg online shared sdn lun RAID_5
emc_clariion0_20 auto:cdsdisk emc_clariion0_20 sfsdg online shared sde lun RAID_10
emc_clariion0_21 auto:cdsdisk emc_clariion0_21 sfsdg online shared sdd lun DISK
sda auto:none - - online invalid sda -
oracleprod2:~ # vxdisk list emc_clariion0_21
Device: emc_clariion0_21
devicetag: emc_clariion0_21
type: auto
clusterid: erharsfs
disk: name=emc_clariion0_21 id=1386161433.22.oracleprod2
group: name=sfsdg id=1382527534.52.oracleprod1
info: format=cdsdisk,privoffset=208,pubslice=3,privslice=3
flags: online ready private autoconfig shared autoimport imported
pubpaths: block=/dev/vx/dmp/emc_clariion0_21s3 char=/dev/vx/rdmp/emc_clariion0_21s3
guid: -
udid: DGC%5FDISK%5FCKM00131301652%5F6006016011B034003C5AC8232730E311
site: optimi
version: 4.1
iosize: min=512 (bytes) max=2048 (blocks)
public: slice=3 offset=65744 len=209649360 disk_offset=48
private: slice=3 offset=208 len=65536 disk_offset=48
update: time=1499988055 seqno=0.132
ssb: actual_seqno=0.0
headers: 0 240
configs: count=1 len=51360
logs: count=1 len=4096
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 enabled
config priv 000256-051423[051168]: copy=01 offset=000192 enabled
log priv 051424-055519[004096]: copy=01 offset=000000 enabled
lockrgn priv 055520-055663[000144]: part=00 offset=000000
Multipathing information:
numpaths: 2
sdd state=enabled type=secondary
sdr state=enabled type=secondary
oracleprod2:~ #
dmpeventlogs showing constant messages like below :-->
Tue Jul 25 02:01:04.000: Reconfiguration is in progress
Tue Jul 25 02:01:04.000: Reconfiguration has finished
Tue Jul 25 02:01:06.000: Reconfiguration is in progress
Tue Jul 25 02:01:06.000: Reconfiguration has finished
Tue Jul 25 06:01:19.000: Reconfiguration is in progress
Tue Jul 25 06:01:19.000: Reconfiguration has finished
Tue Jul 25 09:23:49.000: Reconfiguration is in progress
Tue Jul 25 09:23:49.000: Reconfiguration has finished
Tue Jul 25 09:43:35.000: Reconfiguration is in progress
Tue Jul 25 09:43:35.000: Reconfiguration has finished
Any clues to move forward?
connections to EMC looks okay and all initiator are logged in , registered and connected to storagegroup
One lUN showing as config diabled
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 disabled
config priv 000256-051423[051168]: copy=01 offset=000192 disabled
log priv 051424-055519[004096]: copy=01 offset=000000 disabled
Regards
S.
07-25-2017 03:04 AM
Maybe the connections were coming and going. I'd open a support case to verify.
07-25-2017 04:03 AM
Its a two node cluster and happened with only one node
07-25-2017 08:40 AM
What does syslog show during the issue timeframe ? What is Veritas Version and Linux version here ?
Not sure where did you see both the paths are disabled. The vxdisk list output shows that paths are
enabled. Config copy disabled is not an issue since there are other disks in same diskgroup which will have Enabled config copy.
Please open support case.
07-25-2017 08:59 AM - edited 07-25-2017 09:00 AM
Yes, both the paths are showing as enabled but both are showing as secondary.
Ideally one path shows as primary and other secondary as ita an A/P array.
how can I enable the config in one disk ?
07-25-2017 09:25 AM
Enabling config copy is not going to change anything. We need to find out why it is showing as secondary for both the paths. Please open support case.
07-25-2017 10:33 AM
Hi,
I know about support case .I have asked for help if someone can suggest me.
Please donot reply with same line . It will discourage other helpers too.
Regards
S
07-25-2017 10:36 AM
Unfortunately, The information you have posted is not useful to unerstand the issue and suggest any concrete step to resolve the issue.
07-26-2017 01:50 AM
I am sorry if my information is not correct.
I want to know why dmp is showing both the paths as secondary when both the paths are enabled
vxdmpadm getsubpaths
NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-NAME CTLR ATTRS
================================================================================
sda ENABLED(A) - sda disk c6 -
sdi ENABLED(A) SECONDARY emc_clariion0_10 emc_clariion0 c7 -
sdw ENABLED(A) SECONDARY emc_clariion0_10 emc_clariion0 c8 -
sdj ENABLED(A) SECONDARY emc_clariion0_11 emc_clariion0 c7 -
sdx ENABLED(A) SECONDARY emc_clariion0_11 emc_clariion0 c8 -
sdk ENABLED(A) SECONDARY emc_clariion0_12 emc_clariion0 c7 -
sdy ENABLED(A) SECONDARY emc_clariion0_12 emc_clariion0 c8 -
sdl ENABLED(A) SECONDARY emc_clariion0_13 emc_clariion0 c7 -
sdz ENABLED(A) SECONDARY emc_clariion0_13 emc_clariion0 c8 -
sdaa ENABLED(A) SECONDARY emc_clariion0_14 emc_clariion0 c8 -
sdm ENABLED(A) SECONDARY emc_clariion0_14 emc_clariion0 c7 -
sdab ENABLED(A) SECONDARY emc_clariion0_15 emc_clariion0 c8 -
sdn ENABLED(A) SECONDARY emc_clariion0_15 emc_clariion0 c7 -
sde ENABLED(A) SECONDARY emc_clariion0_20 emc_clariion0 c7 -
sds ENABLED(A) SECONDARY emc_clariion0_20 emc_clariion0 c8 -
sdd ENABLED(A) SECONDARY emc_clariion0_21 emc_clariion0 c7 -
sdr ENABLED(A) SECONDARY emc_clariion0_21 emc_clariion0 c8 -
What could we check further. ?
In what circumstances , it behaves as such.
There are no messages in messages file related to scsi/disks
Regards
S.
10-27-2017 05:04 AM
There are a couple of thing you can do.
they are listed below
1. Run rpm -aq | grep -i vxvm
2. vxdmpadm listctlr all
3. grep -i error /etc/vx/dmpevent.log | grep 2017
4. rm /dev/vx/(r)dmp/*
5. vxconfigd -k
6. vxdmpadm getsubpaths
if you still do not see any primary paths, check the enc clariion settings
11-09-2017 04:05 PM
There are a couple of thing you can do.
they are listed below
1. Run rpm -aq | grep -i vxvm
2. vxdmpadm listctlr all
3. grep -i error /etc/vx/dmpevent.log | grep 2017
4. rm /dev/vx/(r)dmp/*
5. vxconfigd -k
6. vxdmpadm getsubpaths
if you still do not see any primary paths, check the enc clariion settings