LUN Removal Process - Solaris 10 U8 / Storage Foundation 5.1 / PowerPath 5.3

[calgarypete]

Hello,

What is the correct procedure to remove a LUN which is no longer required?

Setup is a Solaris 10 server, dual pathed FC to EMC storage and managed with VxVM 5.1.

Once the LUN has been removed from Veritas (removed from DG then vxdisk rm <disk>)

At what point can I unmask the LUN on the storage side? Is it before or after I remove the disk with powermt? Do I even perform a powermt remove?

Once that is done, I would then remove from OS via cfgadm, then devfsadm.

What I am currently trying to do a powermt remove <emcpower#> after I do a vxdisk rm <emcpower#>, but it complains that the device is still in use.

 somebox01:# /etc/powermt remove dev=emcpower123a
Cannot remove device that is in use: emcpower123a
somebox01:#  

Thoughts appreciated.

Pete

[Gaurav_S]

Hello Pete,

If you are using powerpath, you are following the right steps, vxdmp layer is above powerpath..

1. remove device from DMP

2. remove device from vxvm  (vxdg remove disk,  vxdiskunsetup -C <disk> )

3. remove device from powerpath (powermt remove)

4. unconfigure device from OS

5. remove device from OS (luxadm)

as you are saying that you are getting above mentioned error, just wanted to confirm if you have removed the device from vxvm using step 2 above ?

Can you also confirm the vxvm version you are using ?

Gaurav

[calgarypete]

Thanks for your reply Gaurav,

I dont think I understand why you would remove device from DMP first? Isn't the vxvm layer  above the dmp layer?

vxvm

dmp

powerpath

os

Also, what command would be used to remove the device from dmp?

What ive done in the past (non powerpath) is just remove disk from VxVM, remove the disk from the SAN, then uncofigure the device (cfgadm -o unusable_FCP_dev -c unconfigure c:....) , then devfsadm. I've never done anything with DMP, or luxadm when removing a LUN.

vxvm is 5.1

Thank you.

[Gaurav_S]

sorry for the confusion, I meant cleaning up the device metanodes from /dev/vx/rdmp & /dev/vx/dmp structure.... though its not going to make much difference since device structure is already loaded in kernel but I always follow this to keep my device tree updated..

so coming back to error you are getting, have you removed the disk from diskgroup & unsetup the disk before running powermt remove ?

Gaurav

[calgarypete]

Once I remove the disk from the diskgroup, I vxdisk rm <disk> to get it out of VxVM control.

It is at this point that if I run powermt remove <disk>, I get the error stating that the disk is in use.

I'm reading up vxdiskunsetup as I've never needed to use that in the past. I would think vxdisk rm would suffice. Maybe things with powerpath are different.??

Pete

[Gaurav_S]

well I am not sure on internals of powerpath but just suspecting if powerpath is somehow able to detect that device is in use (may be because of private region of veritas lying on the disk)

Gaurav

[nmcherla]

Pete,

Before you remove the disk from powerpath, the disk is to be removed from the storage side. Follow the below procedure to remove LUN.

1. Remove the disk from veritas control using 'vxdisk rm <device> and vxdiskunsetup <device> . Then clean up the devices from /dev/vx/dmp and /dev/vx/rdmp.

2.  Remove the disk from storage side. 

Once the disk is removed from storage you will see the disk as <drive not available > in format.  

3. Clean powerpath

   #powermt check                     (remove the disk)

4. At this point you should be able to remove the disk  with " cfgadm" if the disk is in unusable state.  If its in 'failing' state use 'luxadm' to offline the disk. You cannot unconfigure a disk unless it is in 'unusable' state and you have to unconfigure both the paths of your disk.  If your disk is "c2t5006048ad52f4106d37" then

#cfgadm -al -o show_SCSI_LUN   |grep <disk>        

#luxadm -e offline /dev/dsk/<device>

#cfgadm -c unconfigure -o unusable_SCSI_LUN c2::5006048as52f4106

#cfgadm -c unconfigure -o unusable_SCSI_LUN c4::5006048as52f4109

4. devfsadm

Nandini  

[g_lee · Accepted solution]

These steps are not in the correct order, the procedure should be as follows:

1. Stop vxesd (vx event source daemon) as this may pick up disks again after you have removed them with vxdisk rm:

# vxddladm stop eventsource
# ps -ef |grep vxesd

2. remove device from vxvm  (vxdg -g <dg> rmdisk <disk>,  vxdiskunsetup -C <disk> )

3. remove device from DMP (rm /dev/vx/*dmp/<device>*   --- be careful to only pick up the device you're removing and not any devices that have similar names!)

4. remove device from powerpath (powermt remove dev=<device> [by default powermt remove removes *all* devices, so specify the device)

5. unconfigure device from OS (cfgadm/luxadm depending on age of platform. For Solaris 10 cfgadm should work. also run devfsadm -Cv after disks have been unpresented from storage)

Pete,

It sounds like it's possible that vxesd may have picked up the devices again after you removed them with vxdisk rm. Try stopping vxesd, removing the disks with vxdisk rm (and removing the dmp devices), then try the powermt remove again.

The fact that there is/was a private region on the disk should not impact the removal if the disk has been removed cleanly from VxVM/DMP (ie: and not picked up again by vxesd)

regards,

Grace

[calgarypete]

Thanks for the reply Nandini, but how does one vxdiskunsetup after you've already done a vxdisk rm? What's the purpose of vxdiskunset if I'm doing a vxdisk rm to take it out of vxvm control anyway?

Thanks,

Pete

[Gaurav_S]

Hi Pete,

you are right, you will not be able to do a vxdiskunsetup after doing a vxdisk rm, the order of steps should be reversed.

vxdisk rm will not be cleaning the private region from the disk however vxdiskunsetup will. So you need to run vxdiskunsetup first followed by vxdisk rm.

Gaurav

[nmcherla]

You should be able to remove the lun even if you don't run "vxdiskunsetup".  vxdiskunsetup will remove the private and public regions so if you want to clean up the slices, first vxdiskunsetup and then remove the disk from veritas.    The powerpath error you are seeing is because it is still active on storage side.  First try removing it from storage and then clean up the disk with powermt and cfgadm commands.