cancel
Showing results for 
Search instead for 
Did you mean: 

DMP - vxdmpadm reporting extra paths

Phil_Coleman
Level 2

Hi, Got a bit of a funny on a number of servers after a misconfiguration as part of a SAN migration where a cable was incorrectly plugged, removed, and then plugged in where it was supposed to be. All servers are running Solaris 10 with Storage Foundation 4.1 MP2. Below are details of what's been seen and actions taken to try and tidy up. In format there are a number of devices that don't exist. These were created when the cable was incorrectly plugged. c4t500507630513453Ad8 There's also this unwanted ap_id for that connection which currently shows as being unusable # cfgadm -al Ap_Id Type Receptacle Occupant Condition c0 scsi-bus connected configured unknown c0::dsk/c0t0d0 disk connected configured unknown c0::dsk/c0t1d0 disk connected configured unknown c1 scsi-bus connected configured unknown c1::dsk/c1t4d0 CD-ROM connected configured unknown c4 fc-fabric connected configured unknown c4::500507630513453a disk connected configured unusable <--- This one c4::50050763051882c4 disk connected configured unknown c4::500507630518853a disk connected configured unknown c5 fc connected unconfigured unknown c6 fc-fabric connected configured unknown c6::50050763050882c4 disk connected configured unknown c6::500507630508853a disk connected configured unknown c7 fc connected unconfigured unknown # Querying the enclosure it shows that there are 3 paths to the devices with two active and one disabled # vxdmpadm getdmpnode enclosure=IBM_DS8x000 NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME ========================================================================= c6t500507630508853Ad5s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad2s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad16s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad18s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad23s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad0s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad7s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad8s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad15s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c4t500507630518853Ad21s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad13s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad11s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad9s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad1s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad17s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad19s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad14s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad22s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad4s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c4t500507630518853Ad20s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad3s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad24s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad12s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad10s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad6s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 c6t500507630508853Ad25s2 ENABLED IBM_DS8x00 3 2 1 IBM_DS8x000 # Querying one of the devices shows that it thinks it has three paths, with the incorrect one being the one that shows as being disabled # vxdisk list c6t500507630508853Ad8s2 Device: c6t500507630508853Ad8s2 devicetag: c6t500507630508853Ad8 type: auto hostid: fuj409 disk: name=b05_1423 id=1183556373.64.fuj409 group: name=qvdg00 id=1183557746.95.fuj409 info: format=sliced,privoffset=1,pubslice=4,privslice=3 flags: online ready private autoconfig noautoimport imported pubpaths: block=/dev/vx/dmp/c6t500507630508853Ad8s4 char=/dev/vx/rdmp/c6t500507630508853Ad8s4 privpaths: block=/dev/vx/dmp/c6t500507630508853Ad8s3 char=/dev/vx/rdmp/c6t500507630508853Ad8s3 version: 2.1 iosize: min=512 (bytes) max=2048 (blocks) public: slice=4 offset=0 len=31447680 disk_offset=5760 private: slice=3 offset=1 len=3583 disk_offset=1920 update: time=1260157705 seqno=0.222 ssb: actual_seqno=0.0 headers: 0 248 configs: count=1 len=2615 logs: count=1 len=396 Defined regions: config priv 000017-000247[000231]: copy=01 offset=000000 enabled config priv 000249-002632[002384]: copy=01 offset=000231 enabled log priv 002633-003028[000396]: copy=01 offset=000000 enabled Multipathing information: numpaths: 3 c6t500507630508853Ad8s2 state=enabled c4t500507630518853Ad8s2 state=enabled c4t500507630513453Ad8s2 state=disabled <--- This is the unavailable drive from the output of the format command above # Removed the unwanted ap_id by unconfiguring it, which cleaned up all the unavailable devices in format # cfgadm -c unconfigure c4::500507630513453a # cfgadm -al Ap_Id Type Receptacle Occupant Condition c0 scsi-bus connected configured unknown c0::dsk/c0t0d0 disk connected configured unknown c0::dsk/c0t1d0 disk connected configured unknown c1 scsi-bus connected configured unknown c1::dsk/c1t4d0 CD-ROM connected configured unknown c4 fc-fabric connected configured unknown c4::50050763051882c4 disk connected configured unknown c4::500507630518853a disk connected configured unknown c5 fc connected unconfigured unknown c6 fc-fabric connected configured unknown c6::50050763050882c4 disk connected configured unknown c6::500507630508853a disk connected configured unknown c7 fc connected unconfigured unknown # Device still thinks it has three paths # vxdisk list c6t500507630508853Ad8s2 Device: c6t500507630508853Ad8s2 devicetag: c6t500507630508853Ad8 type: auto hostid: fuj409 disk: name=b05_1423 id=1183556373.64.fuj409 group: name=qvdg00 id=1183557746.95.fuj409 info: format=sliced,privoffset=1,pubslice=4,privslice=3 flags: online ready private autoconfig noautoimport imported pubpaths: block=/dev/vx/dmp/c6t500507630508853Ad8s4 char=/dev/vx/rdmp/c6t500507630508853Ad8s4 privpaths: block=/dev/vx/dmp/c6t500507630508853Ad8s3 char=/dev/vx/rdmp/c6t500507630508853Ad8s3 version: 2.1 iosize: min=512 (bytes) max=2048 (blocks) public: slice=4 offset=0 len=31447680 disk_offset=5760 private: slice=3 offset=1 len=3583 disk_offset=1920 update: time=1260157705 seqno=0.222 ssb: actual_seqno=0.0 headers: 0 248 configs: count=1 len=2615 logs: count=1 len=396 Defined regions: config priv 000017-000247[000231]: copy=01 offset=000000 enabled config priv 000249-002632[002384]: copy=01 offset=000231 enabled log priv 002633-003028[000396]: copy=01 offset=000000 enabled Multipathing information: numpaths: 3 c6t500507630508853Ad8s2 state=enabled c4t500507630518853Ad8s2 state=enabled c4t500507630513453Ad8s2 state=disabled # # vxdmpadm getdmpnode nodename=c6t500507630508853Ad8s2 NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME ========================================================================= c6t500507630508853Ad8s2 ENABLED IBM_DS8x00 3 1 2 IBM_DS8x000 # Ran 'vxdctl enable' and now only see the two paths # vxdisk list c6t500507630508853Ad8s2 Device: c6t500507630508853Ad8s2 devicetag: c6t500507630508853Ad8 type: auto hostid: fuj409 disk: name=b05_1423 id=1183556373.64.fuj409 group: name=qvdg00 id=1183557746.95.fuj409 info: format=sliced,privoffset=1,pubslice=4,privslice=3 flags: online ready private autoconfig noautoimport imported pubpaths: block=/dev/vx/dmp/c6t500507630508853Ad8s4 char=/dev/vx/rdmp/c6t500507630508853Ad8s4 privpaths: block=/dev/vx/dmp/c6t500507630508853Ad8s3 char=/dev/vx/rdmp/c6t500507630508853Ad8s3 version: 2.1 iosize: min=512 (bytes) max=2048 (blocks) public: slice=4 offset=0 len=31447680 disk_offset=5760 private: slice=3 offset=1 len=3583 disk_offset=1920 update: time=1260157705 seqno=0.222 ssb: actual_seqno=0.0 headers: 0 248 configs: count=1 len=2615 logs: count=1 len=396 Defined regions: config priv 000017-000247[000231]: copy=01 offset=000000 enabled config priv 000249-002632[002384]: copy=01 offset=000231 enabled log priv 002633-003028[000396]: copy=01 offset=000000 enabled Multipathing information: numpaths: 2 c6t500507630508853Ad8s2 state=enabled c4t500507630518853Ad8s2 state=enabled However via vxdmpadm I still see three paths and it now thinks two of them are disabled # vxdmpadm getdmpnode nodename=c6t500507630508853Ad8s2 NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME ========================================================================= c6t500507630508853Ad8s2 ENABLED IBM_DS8x00 3 1 2 IBM_DS8x000 # I've tried restarting vxconfigd, running 'vxdisk scandisks' and 'vxdctl initdmp' but with no luck. There are no extra devices lurking around under /dev/vx/dmp or /dev/vx/rdmp. The following is in /etc/vx/dmpevents.log. The events at 11:00 would be from when the cable was incorrectly added and then removed. Those at 15:00 will be from the action taken to try and tidy things up Fri Apr 9 11:17:06.000: Reconfiguration is in progress Fri Apr 9 11:17:06.000: Reconfiguration has finished Fri Apr 9 11:19:31.071: I/O error occured on Path c4t500507630513453Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:19:31.072: I/O error occured on Path c4t500507630513453Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:19:31.072: Disabled Path c4t500507630513453Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:19:31.072: I/O error occured on Path c4t500507630513453Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:19:31.072: I/O analysis done on Path c4t500507630513453Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:19:31.072: I/O error occured on Path c4t500507630513453Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:19:31.072: Disabled Path c4t500507630513453Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:19:31.072: Disabled Path c4t500507630513453Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:19:31.073: I/O analysis done on Path c4t500507630513453Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:19:31.073: Disabled Path c4t500507630513453Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:19:31.073: I/O analysis done on Path c4t500507630513453Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:19:31.074: I/O error occured on Path c4t500507630513453Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:19:31.074: I/O analysis done on Path c4t500507630513453Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:19:31.074: I/O error occured on Path c4t500507630513453Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:19:31.074: Disabled Path c4t500507630513453Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:19:31.074: I/O analysis done on Path c4t500507630513453Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:19:31.074: Disabled Path c4t500507630513453Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:19:31.075: I/O analysis done on Path c4t500507630513453Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:20:12.041: I/O error occured on Path c4t500507630513453Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:20:12.042: Disabled Path c4t500507630513453Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:20:12.042: I/O analysis done on Path c4t500507630513453Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:21:28.071: I/O error occured on Path c4t500507630518853Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:21:28.071: I/O error occured on Path c4t500507630518853Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:21:28.071: Disabled Path c4t500507630518853Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:21:28.072: I/O analysis done on Path c4t500507630518853Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:21:28.072: I/O error occured on Path c4t500507630518853Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:21:28.072: Disabled Path c4t500507630518853Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:21:28.072: Disabled Path c4t500507630518853Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:21:28.072: I/O error occured on Path c4t500507630518853Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:21:28.073: I/O analysis done on Path c4t500507630518853Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:21:28.073: Disabled Path c4t500507630518853Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:21:28.075: I/O error occured on Path c4t500507630518853Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:21:28.076: I/O analysis done on Path c4t500507630518853Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:21:28.076: Disabled Path c4t500507630518853Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:21:28.077: I/O analysis done on Path c4t500507630518853Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:21:28.077: I/O error occured on Path c4t500507630518853Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:21:28.077: Disabled Path c4t500507630518853Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:21:28.078: I/O analysis done on Path c4t500507630518853Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:21:28.078: I/O analysis done on Path c4t500507630518853Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:21:37.041: I/O error occured on Path c4t500507630518853Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:21:37.042: Disabled Path c4t500507630518853Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:21:37.042: I/O analysis done on Path c4t500507630518853Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:21:46.452: I/O error occured on Path c4t500507630518853Ad14s2 belonging to Dmpnode c6t500507630508853Ad14s2 Fri Apr 9 11:21:46.452: Disabled Path c4t500507630518853Ad14s2 belonging to Dmpnode c6t500507630508853Ad14s2 Fri Apr 9 11:21:46.453: I/O analysis done on Path c4t500507630518853Ad14s2 belonging to Dmpnode c6t500507630508853Ad14s2 Fri Apr 9 11:22:47.623: Disabled Path c4t500507630518853Ad18s2 belonging to Dmpnode c6t500507630508853Ad18s2 Fri Apr 9 11:22:47.623: Disabled Path c4t500507630518853Ad23s2 belonging to Dmpnode c6t500507630508853Ad23s2 Fri Apr 9 11:22:47.623: Disabled Path c4t500507630518853Ad7s2 belonging to Dmpnode c6t500507630508853Ad7s2 Fri Apr 9 11:22:47.623: Disabled Path c4t500507630518853Ad9s2 belonging to Dmpnode c6t500507630508853Ad9s2 Fri Apr 9 11:22:47.624: Disabled Path c4t500507630518853Ad22s2 belonging to Dmpnode c6t500507630508853Ad22s2 Fri Apr 9 11:22:47.624: Disabled Path c4t500507630518853Ad24s2 belonging to Dmpnode c6t500507630508853Ad24s2 Fri Apr 9 11:22:47.624: Disabled Path c4t500507630518853Ad12s2 belonging to Dmpnode c6t500507630508853Ad12s2 Fri Apr 9 11:27:47.624: Enabled Path c4t500507630518853Ad18s2 belonging to Dmpnode c6t500507630508853Ad18s2 Fri Apr 9 11:27:47.624: Enabled Path c4t500507630518853Ad23s2 belonging to Dmpnode c6t500507630508853Ad23s2 Fri Apr 9 11:27:47.624: Enabled Path c4t500507630518853Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 11:27:47.624: Enabled Path c4t500507630518853Ad7s2 belonging to Dmpnode c6t500507630508853Ad7s2 Fri Apr 9 11:27:47.625: Enabled Path c4t500507630518853Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 11:27:47.625: Enabled Path c4t500507630518853Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 11:27:47.625: Enabled Path c4t500507630518853Ad14s2 belonging to Dmpnode c6t500507630508853Ad14s2 Fri Apr 9 11:27:47.626: Enabled Path c4t500507630518853Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 11:27:47.626: Enabled Path c4t500507630518853Ad22s2 belonging to Dmpnode c6t500507630508853Ad22s2 Fri Apr 9 11:27:47.626: Enabled Path c4t500507630518853Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 11:27:47.626: Enabled Path c4t500507630518853Ad24s2 belonging to Dmpnode c6t500507630508853Ad24s2 Fri Apr 9 11:27:47.626: Enabled Path c4t500507630518853Ad12s2 belonging to Dmpnode c6t500507630508853Ad12s2 Fri Apr 9 11:27:47.627: Enabled Path c4t500507630518853Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 11:27:47.627: Enabled Path c4t500507630518853Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 11:27:47.628: Enabled Path c4t500507630518853Ad9s2 belonging to Dmpnode c6t500507630508853Ad9s2 Fri Apr 9 11:42:55.000: Reconfiguration is in progress Fri Apr 9 11:42:55.000: Reconfiguration has finished Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad16s2 belonging to Dmpnode c6t500507630508853Ad16s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad22s2 belonging to Dmpnode c6t500507630508853Ad22s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad1s2 belonging to Dmpnode c6t500507630508853Ad1s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad5s2 belonging to Dmpnode c6t500507630508853Ad5s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad9s2 belonging to Dmpnode c6t500507630508853Ad9s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad7s2 belonging to Dmpnode c6t500507630508853Ad7s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad21s2 belonging to Dmpnode c4t500507630518853Ad21s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad10s2 belonging to Dmpnode c6t500507630508853Ad10s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad18s2 belonging to Dmpnode c6t500507630508853Ad18s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad14s2 belonging to Dmpnode c6t500507630508853Ad14s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad2s2 belonging to Dmpnode c6t500507630508853Ad2s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad4s2 belonging to Dmpnode c6t500507630508853Ad4s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad24s2 belonging to Dmpnode c6t500507630508853Ad24s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad23s2 belonging to Dmpnode c6t500507630508853Ad23s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad12s2 belonging to Dmpnode c6t500507630508853Ad12s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad6s2 belonging to Dmpnode c6t500507630508853Ad6s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad13s2 belonging to Dmpnode c6t500507630508853Ad13s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad11s2 belonging to Dmpnode c6t500507630508853Ad11s2 Fri Apr 9 11:42:52.202: Disabled Path c4t500507630513453Ad17s2 belonging to Dmpnode c6t500507630508853Ad17s2 Fri Apr 9 15:48:32.873: Disabled Path c4t500507630513453Ad25s2 belonging to Dmpnode c6t500507630508853Ad25s2 Fri Apr 9 15:48:32.921: Disabled Path c4t500507630513453Ad20s2 belonging to Dmpnode c4t500507630518853Ad20s2 Fri Apr 9 15:48:32.968: Disabled Path c4t500507630513453Ad19s2 belonging to Dmpnode c6t500507630508853Ad19s2 Fri Apr 9 15:48:33.015: Disabled Path c4t500507630513453Ad15s2 belonging to Dmpnode c6t500507630508853Ad15s2 Fri Apr 9 15:48:33.063: Disabled Path c4t500507630513453Ad14s2 belonging to Dmpnode c6t500507630508853Ad14s2 Fri Apr 9 15:48:33.111: Disabled Path c4t500507630513453Ad8s2 belonging to Dmpnode c6t500507630508853Ad8s2 Fri Apr 9 15:48:33.160: Disabled Path c4t500507630513453Ad3s2 belonging to Dmpnode c6t500507630508853Ad3s2 Fri Apr 9 15:48:33.207: Disabled Path c4t500507630513453Ad0s2 belonging to Dmpnode c6t500507630508853Ad0s2 Fri Apr 9 15:50:54.000: Reconfiguration is in progress Fri Apr 9 15:50:54.000: Reconfiguration has finished Fri Apr 9 15:52:39.000: Reconfiguration is in progress Fri Apr 9 15:52:39.000: Reconfiguration has finished I'd prefer not to have to reboot so if anyone has any good ideas on how to get DMP to clear out what it knows properly and rescan it would be helpeful. Thanks in advance Phil.

1 ACCEPTED SOLUTION

Accepted Solutions

Phil_Coleman
Level 2
Thanks Dev.

A recon reboot worked fine, as did a normal reboot on a different server that's also got the same issue. I'll give the '-r reset' option a go on vxconfigd and see how that fares.

I think we have a way forward with the reboot, it would just be nice to tidy things up without having to resort to that.

Thanks again.

Cheers
Phil.

View solution in original post

6 REPLIES 6

Dev_Roy
Level 6
Accredited Certified

Reset the device tree using following method:
mv /etc/vx/array.info /etc/vx/array.info.old  
mv /etc/vx/disk.info /etc/vx/disk.info.old  
mv /etc/vx/jbod.info  /etc/vx/jbod.info.old
mv /etc/vx/dmppolicy.info /etc/vx/dmppolicy.info.old --> If this file exists else ignore.

Remove all the entries from the /dev/dsk, /dev/rdsk, /dev/vxdmp, and /dev/vx/rdmp directories except entires for bootdisk or bootmirror from the following directories.
rm /dev/vx/dmp/*  
rm /dev/vx/rdmp/*   
rm /dev/dsk/*  
rm /dev/rdsk/*  

devfsadm -Cv  

vxconfigd -k  -x cleartempdir ---> (if you have cluster then freeze HA cluster before running this command)
vxdctl initdmp
vxdctl enable
vxdctl init <hostname>

Above commands are safe to run and should not have any negative impact.
I hope above steps should help, if not, please consider a reconfigure reboot.

Regards,
Dev

Phil_Coleman
Level 2
Thanks Dev, will give that a go and let you know how it goes.

Cheers
Phil.

Phil_Coleman
Level 2
Dev,

No luck I'm afraid. Carried out the above steps but still see this extra path. The server is in an cluster so have tried this on both servers, but with differing results against the same disk...

Inquiry for /dev/rdsk/c4t500507630518853Ad2s2, evpd 0x0, page code 0x0
        Vendor id                        : IBM
        Product id                       : 2107900
        Revision                         : .220
        Serial Number                    : 75FGVD11D2B
fuj413:/root#fuj413:/root# vxdisk list c4t500507630518853Ad2s2
Device:    c4t500507630518853Ad2s2
devicetag: c4t500507630518853Ad2
type:      auto
hostid:    fuj413
disk:      name=b05_1D2B id=1188382829.17.fuj413
group:     name=l149dg00 id=1188383028.21.fuj413
info:      format=sliced,privoffset=1,pubslice=4,privslice=3
flags:     online ready private autoconfig noautoimport imported
pubpaths:  block=/dev/vx/dmp/c4t500507630518853Ad2s4 char=/dev/vx/rdmp/c4t500507630518853Ad2s4
privpaths: block=/dev/vx/dmp/c4t500507630518853Ad2s3 char=/dev/vx/rdmp/c4t500507630518853Ad2s3
version:   2.1
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=31447680 disk_offset=5760
private:   slice=3 offset=1 len=3583 disk_offset=1920
update:    time=1271069157 seqno=0.274
ssb:       actual_seqno=0.0
headers:   0 248
configs:   count=1 len=2615
logs:      count=1 len=396
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-002632[002384]: copy=01 offset=000231 enabled
 log      priv 002633-003028[000396]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   2
c4t500507630518853Ad2s2 state=enabled
c6t500507630508853Ad2s2 state=enabled
fuj413:/root#
fuj413:/root# vxdmpadm getdmpnode nodename=c4t500507630518853Ad2s2
NAME                 STATE     ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
=========================================================================
c4t500507630518853Ad2s2 ENABLED   IBM_DS8x00   3      1     2     IBM_DS8x001
fuj413:/root#
fuj413:/root# /etc/vx/diag.d/vxdmpinq /dev/rdsk/c4t500507630518853Ad2s2

fuj414:/root# vxdisk list c4t500507630518853Ad2s2
Device:    c4t500507630518853Ad2s2
devicetag: c4t500507630518853Ad2
type:      auto
hostid:    fuj413
disk:      name= id=1188382829.17.fuj413
group:     name=l149dg00 id=1188383028.21.fuj413
info:      format=sliced,privoffset=1,pubslice=4,privslice=3
flags:     online ready private autoconfig noautoimport
pubpaths:  block=/dev/vx/dmp/c4t500507630518853Ad2s4 char=/dev/vx/rdmp/c4t500507630518853Ad2s4
privpaths: block=/dev/vx/dmp/c4t500507630518853Ad2s3 char=/dev/vx/rdmp/c4t500507630518853Ad2s3
version:   2.1
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=31447680 disk_offset=5760
private:   slice=3 offset=1 len=3583 disk_offset=1920
update:    time=1271069157 seqno=0.274
ssb:       actual_seqno=0.0
headers:   0 248
configs:   count=1 len=2615
logs:      count=1 len=396
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-002632[002384]: copy=01 offset=000231 enabled
 log      priv 002633-003028[000396]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   2
c4t500507630518853Ad2s2 state=enabled
c6t500507630508853Ad2s2 state=enabled
fuj414:/root#
fuj414:/root# /usr/sbin/vxdmpadm getdmpnode nodename=c4t500507630518853Ad2s2
NAME                 STATE     ENCLR-TYPE   PATHS  ENBL  DSBL  ENCLR-NAME
=========================================================================
c4t500507630518853Ad2s2 ENABLED   IBM_DS8x00   3      2     1     IBM_DS8x001
fuj414:/root#
fuj414:/root# /etc/vx/diag.d/vxdmpinq /dev/rdsk/c4t500507630518853Ad2s2
Inquiry for /dev/rdsk/c4t500507630518853Ad2s2, evpd 0x0, page code 0x0
        Vendor id                        : IBM
        Product id                       : 2107900
        Revision                         : .220
        Serial Number                    : 75FGVD11D2B
fuj414:/root#

I get this on a disk that's in use in a volume group on one server, but visible to both because of the cluster, but also on a disk that's not in any disk group but visible to both servers.

I haven't tried a reconfigure boot yet as was hoping to avoid that if possible as there are other things in use on the servers outside of the cluster. If you've no other ideas then I'll go with that.

Thanks again for your help.

Cheers
Phil.
 

Dev_Roy
Level 6
Accredited Certified
Phil,

Seems a detailed investigation required, if reconfigure reboot also fails to get things fixed for you then please open a Support case with Symantec for further investigation.
I am optimistic about reconfigure reboot.

You may try reseting the volume manager daemon using following command:

vxconfigd -k -r reset  -->  (if you have cluster then freeze HA cluster before running this command)
This will flush out any stale info from OS kernel about vxconfigd. Not sure if it is going to fix the issue but worth a try.

Regards,
Dev

Phil_Coleman
Level 2
Thanks Dev.

A recon reboot worked fine, as did a normal reboot on a different server that's also got the same issue. I'll give the '-r reset' option a go on vxconfigd and see how that fares.

I think we have a way forward with the reboot, it would just be nice to tidy things up without having to resort to that.

Thanks again.

Cheers
Phil.

krishna_karthik
Level 2
Certified
Hi Phil,
Try to follow below steps.

1. mv /etc/vx/array.info /etc/vx/array.info.old  
2. mv /etc/vx/disk.info /etc/vx/disk.info.old  
3. mv /etc/vx/jbod.info  /etc/vx/jbod.info.old
4. rm -rf /dev/dsk/*
5. rm -rf /dev/rdsk/*
6. rm -rf /dev/vx/dmp/dsk/*
7. rm -rf /dev/vx/dmp/rdsk/*

8. Disable veritas dmppath one first
9. Do hard-reset in First HBA
If you have multiple HBA, repeat 8th,9th steps 
once complete all

devfsadm -C
vxconfigd -k -x syslog

Thanks & Regards

Krishna