Disk going to error state repeatedly

Question

Hi friends,&nbsp;Everytime we switch the service group i.e the diskgroup resource is switched i.e&nbsp; dg is deported and importedwe are seeing below error during upgrade and&nbsp; one of mirror disk is going to error state&nbsp;one incident&nbsp;Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900008Sep 25 18:06:00 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048Sep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 25 18:06:03 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 25 18:06:04 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-546 Disk disk3mirr in group appdg: Disk device not found========================================&nbsp;2nd incident&nbsp;Sep 19 09:11:50 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900008Sep 19 09:11:50 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048Sep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 19 09:11:53 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-16751 import_start: disk 1530703976.348.jenarm002 (udid DGC%5FRAID%205%5FCKM00134500734%5F600601602AA036000B243B42CB02E411) not found, flags 0x900048Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.error] V-5-1-16253 Disk group import of appdg failed with error 183 - Disk for disk group not foundSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 : dg import with I/O fence enabledSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-11401 appdg: dg import with I/O fence enabledSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16765 Selecting configuration database copy from c2t500601603DE039DFd7s2 from disks: c2t500601603DE039DFd7s2Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16766 Trying to import the disk group appdg using configuration database copy from c2t500601603DE039DFd7s2Sep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.warning] V-5-1-546 Disk disk3mirr in group appdg: Disk device not foundSep 19 09:11:54 arm001 vxvm:vxconfigd: [ID 702911 daemon.notice] V-5-1-16254 Disk group import of appdg succeeded.Sep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(4)::GPN_ID for D_ID=10d00 failedSep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(4)::N_x Port with D_ID=10d00, PWWN=5001438026ece4ac disappeared from fabricSep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(0)::GPN_ID for D_ID=10d00 failedSep 19 09:58:12 arm001 fctl: [ID 517869 kern.warning] WARNING: fp(0)::N_x Port with D_ID=10d00, PWWN=5001438026ee33ac disappeared from fabricSep 19 09:58:12 arm001 genunix: [ID 140958 kern.notice] LLT INFO V-14-1-10205 link 0 (net2) node 1 in trouble=======================================================grep -i error dmpevents* | moredmpevents.log:Fri Sep 28 12:22:42.446: SCSI error occurred on Path c2t500601693DE039DFd9s2: opcode=0x5f reported reservation conflict (status=0x18, key=0x0, asc=0x0, ascq=0x0)dmpevents.log:Fri Sep 28 12:24:47.441: SCSI error occurred on Path c2t500601603DE039DFd8s2: opcode=0x5f reported reservation conflict (status=0x18, key=0x0, asc=0x0, ascq=0x0)dmpevents.log:Fri Sep 28 12:24:47.807: SCSI error occurred on Path c2t500601693DE039DFd9s2: opcode=0x5f reported reservation conflict (status=0x18, key=0x0, asc=0x0, ascq=0x0)dmpevents.log.0:Thu May&nbsp; 3 16:59:03.147: SCSI error occurred on Path c4t500601613DE039DFd8s2: opcode=0x28 reported device not ready (status=0x2, key=0x2, asc=0x4, ascq=0x3)dmpevents.log.0:Thu May&nbsp; 3 16:59:09.356: SCSI error occurred on Pat&nbsp;Need suggestion as what could be the cause. We have initalised the disk and added into diskgroup and mirrored it again.However , we wnt to prevent this in futur.can anyone guide as what could be the cause ?&nbsp;&nbsp;RegardsS&nbsp;

riaanbadenhorst · Answer

I would look at the Fibre Channel layer as you're seeing a lot os SCSI errors. It could be drivers, cables or firmware.
Veritas support should confirm the same.&nbsp;
I don't think its anything to do with your configuration.

frankgfan · Answer

an you please run the commands below on both the systems and post the outputs?

1. uname -a

2. vxdisk -o alldgs list

3. vxprint -ht | egrep -i "err|fail" | tail -5

4. grep -i scsi /etc/vx/dmpevents.log | egrep -i "fail|err" | wc -l

5. gabconfig -a

6. hastatus -sum

7. haclus -display } grep -i vers

plus

8. the details of how the disk groups are swicthed

Forum Discussion

Disk going to error state repeatedly

Related Content

AIX Media Server - Repeatedly 84 errors

Where did this link go?

Tape Drive Going OffiIne

Tape drive keeps going offline

Workflows not going down