Forum Discussion

Ramesh_Kasiredd's avatar
Ramesh_Kasiredd
Level 2
11 years ago

DMP I/O error is disabling the NETAPP paths via DMP

For everty roo boot the VXVM volume is going to DEPORT state.
I am importing it manually.
I saw the below intresting logs on "dmpevents.log"
 
 
 
Thu Oct 10 16:53:56.317: Enabled Disk array other_disks
Thu Oct 10 16:53:56.317: Enabled Disk array fas32200
Thu Oct 10 16:53:56.318: Added Dmpnode c0t1d0s2
Thu Oct 10 16:53:56.318: Added Dmpnode fas32200_0
Thu Oct 10 16:53:56.318: Dmpnode c0t0d0s2 has migrated from enclosure - to other_disks
Thu Oct 10 16:53:56.318: Disabled Disk array -
Thu Oct 10 16:53:56.367: I/O error occured on Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 16:53:56.370: I/O analysis done as DMP_PATH_OKAY on Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 16:53:56.370: I/O retry(4) on Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 16:53:56.373: I/O error occured on Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 16:53:56.374: SCSI error occured on Path c3t60A980004431702F643F44436B446B46d0s2: opcode=0x12 reported generic device failure (status=0x0, key=0x0, asc=0x50, ascq=0x50)
Thu Oct 10 16:53:56.374: I/O analysis done as DMP_PATH_FAILURE on Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 16:53:56.374: Disabled Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0 due to path failure
Thu Oct 10 16:53:56.374: Disabled Dmpnode fas32200_0
Thu Oct 10 16:53:56.374: Disabled Disk array fas32200
Thu Oct 10 16:53:56.374: I/O error occured (errno=0x6) on Dmpnode fas32200_0
Thu Oct 10 16:53:59.558: Enabled Disk array fas32200
Thu Oct 10 16:53:59.558: Enabled Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 16:53:59.558: Enabled Dmpnode fas32200_0
Thu Oct 10 16:53:59.559: Failover initiated for Dmpnode fas32200_0 without quiescing
Thu Oct 10 16:53:59.559: CURPRI set to primary for Dmpnode fas32200_0 without quiescing
Thu Oct 10 16:54:04.000: Initiated SAN topology discovery
Thu Oct 10 16:54:05.000: Completed SAN topology discovery
Thu Oct 10 22:06:11.460: Marked as idle Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 22:07:22.012: Unmarked as idle Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
Thu Oct 10 22:07:24.012: Lost 5120 DMP I/O statistics records
Thu Oct 10 22:07:25.012: Lost 2071 DMP I/O statistics records
Thu Oct 10 22:08:28.011: Marked as idle Path c3t60A980004431702F643F44436B446B46d0s2 belonging to Dmpnode fas32200_0
 
business continuity
Solaris
Storage Foundation
  • AlexeyL's avatar
    AlexeyL
    11 years ago

    It's possible that MPxIO and DMP interfere with each other. In my environment I tend to disable MPxIO for controllers/disks managed by DMP. Here is some info from Symantec about interoperability of MPxIO and DMP.

    http://www.symantec.com/business/support/index?page=content&id=TECH204060

    Also if you could provide the following:

    1) mpathadm list lu

    2) vxdmpadm getsubpaths

     

4 Replies

Sort By
  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    11 years ago

    Hello,

    It is clear from the errors that DMP/VxVM is trying to make IO to the device however is failing intermittently ..

    Questions:

    1. What version of VxVM is in use ?

    2. What Storage array is in use ?

    3. attach the output of # iostat -En

    4. paste output of # vxdmpadm listenclosure all

    5. Any analysis done from Storage / SAN switch side to know if there are any IO errors noticed ? I would expect the SAN switch logs should show some errors.

    6. Any other multipathing software underneath DMP ?  mpxio or powerpath ?

     

    G

  • Ramesh_Kasiredd's avatar
    Ramesh_Kasiredd
    Level 2
    11 years ago

    1. What version of VxVM is in use ?

                 VERSION:  5.1,REV=10.06.2009.21.48

    2. What Storage array is in use ?

                 NETAPP

    3. attach the output of # iostat -En

                  c0t0d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0

    Vendor: ATA      Product: ST31000524NS     Revision: SN12 Serial No:
    Size: 1000.20GB <1000204885504 bytes>
    Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
    Illegal Request: 49 Predictive Failure Analysis: 0
    c0t1d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
    Vendor: ATA      Product: ST31000524NS     Revision: SN12 Serial No:
    Size: 1000.20GB <1000204885504 bytes>
    Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
    Illegal Request: 29 Predictive Failure Analysis: 0
    c3t60A980004431702F643F44436B446B46d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
    Vendor: NETAPP   Product: LUN              Revision: 811a Serial No:
    Size: 536.95GB <536952700416 bytes>
    Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
    Illegal Request: 1 Predictive Failure Analysis: 0
     

    4 . paste output of # vxdmpadm listenclosure all

    ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT

    =======================================================================================
    fas32200          FAS3220        700000265539         CONNECTED    ALUA        1
    other_disks       OTHER_DISKS    OTHER_DISKS          CONNECTED    OTHER_DISKS   2
     

    5. Any analysis done from Storage / SAN switch side to know if there are any IO errors noticed ? I would expect the SAN switch logs should show some errors.

            A) Contacted SAN admin, every thing look fine.

    6 . Any other multipathing software underneath DMP ?  mpxio or powerpath ?

             A) mpxio

     

     

  • ursi's avatar
    ursi
    Level 4
    11 years ago

    hmm -- just my two cents:

     

    This looks like a SCSI error while trying to write to the disk c3t60A980004431702F643F44436B446B46d0s2 (nice name). From the ASC it says it is an "Write Append Error" but I could not find the additional ASCQ Qualifier "50" for that error code.

    If it is a SCSI error it seems unappropriate to disable the path since the path (to the broken (virtual) disk) is just fine.

     

  • AlexeyL's avatar
    AlexeyL
    Level 3
    11 years ago

    It's possible that MPxIO and DMP interfere with each other. In my environment I tend to disable MPxIO for controllers/disks managed by DMP. Here is some info from Symantec about interoperability of MPxIO and DMP.

    http://www.symantec.com/business/support/index?page=content&id=TECH204060

    Also if you could provide the following:

    1) mpathadm list lu

    2) vxdmpadm getsubpaths