cancel
Showing results for 
Search instead for 
Did you mean: 

Plex disabled stale while volume is enabled active

SaGu
Level 3
Certified

Hi All,

 

After a power outage i have a situation that the plexes went in disabled stale state while the volume is enabled active.

 

I've tried to fix clean the plex but then it says volume is enabled, anyways it's right because the volume is fine.

Any ideas how to get the plex enabled active again?

 

v  v_archlog    -            ENABLED  ACTIVE   184311808 SELECT   -        fsgen
pl v_archlog-01 v_archlog    DISABLED STALE    184312800 CONCAT   -        WO
sd mams06-dh-02 v_archlog-01 mams06-dh 20972160 41944320 0        HDS9980V0_10 ENA
sd mams11-dh-01 v_archlog-01 mams11-dh 0       71184960 41944320  HDS9980V0_0 ENA
sd mams11-lm-01 v_archlog-01 mams11-lm 0       71183520 113129280 HDS9980V1_0 ENA
pl v_archlog-02 v_archlog    ENABLED  ACTIVE   184312800 CONCAT   -        RW
sd mams06-lm-01 v_archlog-02 mams06-lm 0       41944320 0         HDS9980V1_10 ENA
sd mams12-dh-01 v_archlog-02 mams12-dh 0       71184960 41944320  HDS9980V0_1 ENA
sd mams12-lm-01 v_archlog-02 mams12-lm 0       71183520 113129280 HDS9980V1_1 ENA

1 ACCEPTED SOLUTION

Accepted Solutions

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hello,

one of the plex is disabled & its a mirrored volume, thats the reason volume is still in enabled state...

I would suggest first:

# vxrecover -bsE -g <diskgroup>

THis would try to recover the plex & sync again. In case if this doesn't works, I would suggest to detach the stale plex & attach it again.. you can do so by "vxplex det" & "vxplex att"

 

Gaurav

View solution in original post

10 REPLIES 10

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hello,

one of the plex is disabled & its a mirrored volume, thats the reason volume is still in enabled state...

I would suggest first:

# vxrecover -bsE -g <diskgroup>

THis would try to recover the plex & sync again. In case if this doesn't works, I would suggest to detach the stale plex & attach it again.. you can do so by "vxplex det" & "vxplex att"

 

Gaurav

SaGu
Level 3
Certified

Thanks for the reply!!

 

Tried vxplex att but it gives i/o error.

vxvm:vxplex: ERROR: Volume v_archlog, plex v_archlog-02, block 13329120: Plex read:
        Error: Read failure
vxvm:vxplex: ERROR: I/O error on volume v_archlog, plex v_archlog-01 not attached

 

Maybe something wrong with the disk or is it ust a block. Any ideas?

Gaurav_S
Moderator
Moderator
   VIP    Certified

well from the vxprint output, it looks disks good & enabled

pl v_archlog-01 v_archlog    DISABLED STALE    184312800 CONCAT   -        WO
sd mams06-dh-02 v_archlog-01 mams06-dh 20972160 41944320 0        HDS9980V0_10 ENA
sd mams11-dh-01 v_archlog-01 mams11-dh 0       71184960 41944320  HDS9980V0_0 ENA
sd mams11-lm-01 v_archlog-01 mams11-lm 0       71183520 113129280 HDS9980V1_0 ENA

 

Did the det command succeed ? If plex is detached, you tried to fix clean it outside the volume ? what is current vxprint output ?

Do you see any failed/failing disk in vxdisk list ?

what is vxvm version you are using ?

 

Gaurav

SaGu
Level 3
Certified

yes detach command went without error. After detaching i just tried to reattach it.

Below is the current o/p.

v  v_archlog    -            ENABLED  ACTIVE   184311808 SELECT   -        fsgen
pl v_archlog-01 v_archlog    DETACHED STALE    184312800 CONCAT   -        WO
sd mams06-dh-02 v_archlog-01 mams06-dh 20972160 41944320 0        HDS9980V0_10 ENA
sd mams11-dh-01 v_archlog-01 mams11-dh 0       71184960 41944320  HDS9980V0_0 ENA
sd mams11-lm-01 v_archlog-01 mams11-lm 0       71183520 113129280 HDS9980V1_0 ENA
pl v_archlog-02 v_archlog    ENABLED  ACTIVE   184312800 CONCAT   -        RW
sd mams06-lm-01 v_archlog-02 mams06-lm 0       41944320 0         HDS9980V1_10 ENA
sd mams12-dh-01 v_archlog-02 mams12-dh 0       71184960 41944320  HDS9980V0_1 ENA
sd mams12-lm-01 v_archlog-02 mams12-lm 0       71183520 113129280 HDS9980V1_1 ENA

 

Earlier all disk were fine but now mams06-lm disk which is in secondary active plex is now in disabled state. So that's also a worry now.

 

It's veritas 3.5!!

SaGu
Level 3
Certified

Correction mams06-lm disk is in failing state

Gaurav_S
Moderator
Moderator
   VIP    Certified

OK failing state indicates that there was some temporary iofailure on the public region of the disk (data area) .. It may or may not be a worry.... I would suggest to keep an eye on iostat counters & see if hard/transport errors are increasing or more... if yes, then you may need to think of replacing disk..

you can also turn off the failing flag on the disk, if it re-appears, you need to investigate...

# vxedit -g <diskgroup> set failing=off <disk>

Regarding the stale plex, I believe, you need to disassociate (vxplex dis) the plex (take out of volume), fix clean it & then associate it back.. but again make sure that all background issues with SAN are resolved & the three disks pertaining to bad plex are available & are not in failed/failing state...

Gaurav

g_lee
Level 6

SaGu,

The error you pasted earlier indicates mams06-lm has read errors / is now failing - this is why the detached plex could not be reattached, as VxVM encountered read errors on the "good" plex while trying to reattach.

ie: the attach command syntax is fine, it's just the "good" plex is now having errors, thus preventing re-attach.

Is mams06-lm actually seeing errors (on the underlying disk / in iostat)? If it's a transient condition, try the reattach again. If the disk still generates errors/still fails, would recommend ensuring you are backing up the data on this volume, as if the disk in the "good" plex is failing, and you are unable to reattach the plex before it fails, then you will lose the volume and will need to restore from backups once the disk(s) have been replaced.

Hope that helps,

Grace

SaGu
Level 3
Certified

How to clean a dissociated plex?

g_lee
Level 6

It's not possible to clean a dissociated plex, by definition:

--------------------
A plex is in a CLEAN state when it is known to contain a consistent copy (mirror) of the volume contents and an operation has disabled the volume. As a result, when all plexes of a volume are clean, no action is required to guarantee that the plexes are identical when that volume is started.
--------------------

ie: a dissociated plex is not attached/associated with any volume, so it cannot be clean

Per previous post, you have detached the plex, so now you need to reattach it (via vxplex att). However, you need to ensure the "good"/active plex is readable/not issuing any read errors, otherwise it won't be able to attach, as it won't be able to read the contents from the active plex to mirror it to the plex being re-attached.

SaGu
Level 3
Certified

Thanks Grace and Gaurav!!

 

Disk mams06-lm was the culprit. I did an analyze refresh on it in format and then removed the failing flag from it. After this the plex is now synching properly with vxplex att.

 

Thanks again!!