Device Discovery in DMP

[Ameya]

The first basic function for a multi path (MP) driver is to identify the set of block devices that can be supported. The device discovery is generally triggered when the MP driver is loaded into the operating system (OS) as part of boot process. During early boot, not all devices connected to the system may be available. For example, the devices connected using the fibre channel (FC) cables may not be available during early boot time unless there is special support from the bootstrap, which is usually from NVRAM or EEPROM, to identify the fibre channel devices. In any case, it may be required for a MP driver to rescan the OS device tree at a later stage of boot process.

One of the biggest challenges in device discovery is the ability and skill of MP driver to discover disk devices and their attributes in a non-intrusive fashion such that the disk media is not touched. In other words, the device discovery has to be purely read-only operations to identify the disk and disk array attributes without touching the disk media. If disk media is touched, there is every possibility of undesirable effects such as trespass (for A/P arrays), system sluggishness as the target driver keeps the CPU busy trying to power up the disk media to read or write from specific location.

The early version of DMP (VM release 2.5.7), supported only on Solaris platform, had support for only direct attached SCSI disk devices and only a handful of A/A arrays were supported. The device discovery was entirely in kernel space and triggered when DMP driver was loaded into the Solaris kernel as part of boot process. Since it was initiated from the OS kernel and there were only a handful of devices, the overall procedure was quick. In course of time, as DMP was deployed with large number of disks and with disk arrays from different array vendors, the device discovery became a major limitation. Every new support required a change in the device discovery logic of DMP kernel which meant a major testing effort and, secondly, the system required a reboot to support a newly connected disk array.

Both the problems were alleviated by bringing the device discovery layer (DDL) in user space and the concept of array support library (ASL) was introduced. The ASL is a shared library that is invoked during vxconfigd start up process and, which identifies a disk to DDL during runtime. The basic assumption in DDL is that a disk will be recognized by only one ASL. When vxconfigd is started from system start up scripts or when a user explicitly invokes device reconfiguration using 'vxdisk scandisks', the DDL gathers a list of devices and consults each installed ASL to gather basic information such as

- Vendor ID

- Product ID

- Disk array name and its serial number

- Disk attributes such as unique serial number, primary, secondary

- SCSI version

- Whether the disk is controlled by a TPD

- Type of TPD (suppressing or non-suppressing type)

The DDL then compiles this information and downloads it to DMP. The ASL made it possible for DMP to support a new disk array dynamically and without a system reboot. As the reboot was avoided, the system downtimes improved and applications could continue uninterrupted. The success of the ASL program is mainly due to the simplicity of the SDK that is used by a vendor to write an ASL. Further, the vendor can continue with the ASL testing independently using the demonstration license. However, once the ASL is tested by the vendor, the vendor approaches Symantec to certify the ASL so that it can be deployed in the field. As part of the certification process, the vendor ASL is assigned a unique encoded key that is accessed by DDL to validate the ASL.

The ASL framework has undergone quite a number of improvements to help in using devices that previously could not be used. For example, the VM could not support hardware mirrors created from array console due to conflicting disk ID. With the improved ASL framework, it is now possible for an end user to selectively support a device with VM and the user has an option to use either the original device or the hardware mirror. The complete list of supported arrays can be found at http://clabwest.veritas.com

In the next article, I shall touch upon the APM framework that is consulted by the DMP kernel.

- Ameya P. Usgaonkar