Disk going to error state repeatedly
Hi friends, Everytime we switch the service group i.e the diskgroup resource is switched i.e dg is deported and importedwe are seeing below error during upgrade and one of mirror disk is going to error state one incident Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900008 Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048 Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048 Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048 Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-546 Disk disk3mirr in group appdg: Disk device not found ======================================== 2nd incident Sep 19 09:11:50 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900008 Sep 19 09:11:50 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048 Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not found Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabled Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabled Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2 Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-546 Disk disk3mirr in group appdg: Disk device not found Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16254 Disk group import of appdg succeeded. Sep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(4)::GPN_ID for D_ID=10d00 failed Sep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(4)::N_x Port with D_ID=10d00, PWWN=5001438026ece4ac disappeared from fabric Sep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(0)::GPN_ID for D_ID=10d00 failed Sep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(0)::N_x Port with D_ID=10d00, PWWN=5001438026ee33ac disappeared from fabric Sep 19 09:58:12 arm001 genunix: [ID 140958 kern.notice] LLT INFO V-14-1-10205 link 0 (net2) node 1 in trouble ======================================================= grep -i error dmpevents* | more dmpevents.log:Fri Sep 28 12:22:42.446: SCSI error occurred on Path c2t500601693DE039DFd9s2: opcode=0x5f reported reservation conflict (status=0x18, key=0x0, asc=0x0, as cq=0x0) dmpevents.log:Fri Sep 28 12:24:47.441: SCSI error occurred on Path c2t500601603DE039DFd8s2: opcode=0x5f reported reservation conflict (status=0x18, key=0x0, asc=0x0, as cq=0x0) dmpevents.log:Fri Sep 28 12:24:47.807: SCSI error occurred on Path c2t500601693DE039DFd9s2: opcode=0x5f reported reservation conflict (status=0x18, key=0x0, asc=0x0, as cq=0x0) dmpevents.log.0:Thu May 3 16:59:03.147: SCSI error occurred on Path c4t500601613DE039DFd8s2: opcode=0x28 reported device not ready (status=0x2, key=0x2, asc=0x4, ascq= 0x3) dmpevents.log.0:Thu May 3 16:59:09.356: SCSI error occurred on Pat Need suggestion as what could be the cause. We have initalised the disk and added into diskgroup and mirrored it again. However , we wnt to prevent this in futur. can anyone guide as what could be the cause ? Regards S