SAN disk group will not automatically re-import after reboots.

[mookyrooky]

OK here's the long version on what I believe is happening, and why the disk isn't automatically importing on 3 out of 4 of our exchange servers: there's several layers here I believe that have to be considered: the HBA, Veritas Volume Management, and the Windows Volume and the way they work together to import external disk. The significant difference is that the good server, EXCH04 is configured as a BASIC CLUSTER, while the other 3 servers are configured as DYNAMIC CLUSTERS, which we know aren't working since the SAN change from a Hitachi AMS500 to the Hitachi XP24000 . I do recall these were all configured this way even on the old SAN: exch04 being the only one that was a BASIC disk...not sure why unless there was a reason to make the other 3 dynamic in order to facilitate growth in the future. It doesn't really matter, but it sticks in my mind that I did wonder why they were dissimilar even before we moved to the new SAN. 

So the Veritas recommended fix I applied the other night would have reimported the disk as a Secondary cluster instead of a Dynamic Cluster, which in theory is meant to (a) delay the import until after all disk and dependency services start (like iSCSI, Veritas DG delay, etc...) to ensure a stable disk before importing it to Windows and making the volume visible, and (b) attempt to make them BASIC clusters, which defines them more like a delta block of disk rather than a dynamic group...this should have allowed the HBA's to just interpret them as simple local disk. This change from Dynamic to Secondary was a "baby steps" approach to make a small change to the disk cluster without affecting data. As we know, it didn't work. I didn't have the option to import as BASIC because when you convert the disk to basic GPT from dynamic, I was worried about deleting data. 

My theory is that when the SAN platform changed, the Emulex HBA's (and when we are talking about HBA's, we are talking about the Veritas Volume Manager software which manages them too) lost their ability to recognize the SAN disk as (essentially) local storage. I had hoped updating the firmware would have brought them up to date, but that had no effect either. 

So in order to get this functionality back, the disks must be deported then reconfigured as BASIC disk, so Veritas sees just one large disk and not a cluster it has to manage. This will make Veritas "think" it's local and imported at Windows startup, rather than waiting for Veritas to come online to manage it. Many of the forums I've visited over the course of troubleshooting this problem seem to confirm this. Nearly everyone has had to upgrade their servers to Server 2003 R2 or later to get this to work. 

The catch here is that converting to BASIC from DYNAMIC, which I strongly feel will resolve it, requires recreation of the cluster on the SAN - ie, deleting the data, blowing away the disk group, recreating the group in the BASIC format, then reimporting the disk to Windows and restoring data from backup. 

I'd like to submit this theory to Veritas for review and to tell me if I'm on the right track or full of it. But I'd like to be able to give them better specifics on what exactly was the change on the SAN.

[Wally_Heim · Accepted solution]

Hi Mookyrooky,

You are way over thinking this.

Clustered dynamic diskgroups are not automatically imported during boot time.  This is becase with clustered dynamic disk groups the cluster software should be controlling the import and deport of the disk group.

I'm not seeing any mention of what cluster product you are using.  It sounds like you have standalone nodes.  In which case you should be running secondary dynamic disk groups. 

Secondary diskgroups should be returned to the same import state that they were in during the last shutdown of the OS.  If the secondary disk group was imported when the server was rebooted then it should be imported when the server starts back up.  However, with some hardware confiugraitons the disks are not available during the boot time so they are not automaticaly imported.  In this case you need to set the delayed import for the diskgroup so that SFW knows to check for it after the normal boot time import operations and other os operations have finished.

I would try the following.

1. Ensure that the diskgroup is imported.

2. Test rebooting the server to see what happens to the diskgroup when the server restarts.

If the disk group does not import on reboot then try this.

1. Follow tech note TECH36108 to setup the delayed import service for the diskgroup.

http://www.symantec.com/business/support/index?page=content&id=TECH36108&actp=search&viewlocale=en_US&searchid=1314806450554

2. Ensure that the diskgroup is imported.

3. Test rebooting the server to see what happens to the diskgroup when the server restarts.

FYI - Your steps to convert the disk group to basic are correct.  It is a destructive process and requires that the data be backed up and restored.

Thanks,

Wally

[mookyrooky]

Thanks Wally. I had in fact already enabled the delayed import service, but hadn't tried from a CLI. When I did run the command, it errored out, and it struck me that the name of my disk group may be an issue. It's named Logs&Database, and I'm wondering if the ampersand isn't supported. I've schedueld a test window to take the disk group offline, rename it, bring it back online then test reboot.