problem with cvm/cfs
After install license properly, I began the testing of cvm/cfs.
I had already created a shared disk group, and activation mode was set to sw (shared write), while I tried to create the vxfs file system, it always reported error "cluster reservation failed for volume"
Does anyone know something about this? thanks.
slave:/ #vxdg list mydg
Group: mydg
dgid: 1168495239.16.slave
import-id: 33792.8
flags: shared cds ---> shared dg
version: 140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw ----> shared write
ssb: on
autotagging: on
detach-policy: global
dg-fail-policy: dgdisable
copies: nconfig=default nlog=default
config: seqno=0.1052 permlen=48144 free=48139 templen=3 loglen=7296
config disk c2t1d0s2 copy 1 len=48144 state=clean online
config disk c2t2d0s2 copy 1 len=48144 state=clean online
config disk c2t4d0s2 copy 1 len=48144 state=clean online
log disk c2t1d0s2 copy 1 len=7296
log disk c2t2d0s2 copy 1 len=7296
log disk c2t4d0s2 copy 1 len=7296
slave:/ #
slave:/ #mkfs -F vxfs /dev/vx/rdsk/mydg/vol01
UX:vxfs mkfs: ERROR: V-3-21018: cluster reservation failed for volume /dev/vx/rdsk/mydg/vol01
slave:/ #
problem is resolved.
Basically this is because the partial installation of cluster file system. Error "cluster reservation failed " is due to the abasence of glm module (group lock manager). After I noticed that VRTSglm is missing, I first tried manuall installation of this package but still couldn't open port f, that being used by cfs.
I deleted everything and re-installed storage foundation cluster file system and all went well afterwards.
Also as I am using 3 iscsi disks (they are created from one server of the cluster, so basically there is no real shared storeage), with dmp enabled, all 3 iscsi disks were probed into one disk with 3 paths, that is becuase the solaris soft iscsi disks don't provide serial number, as a workaround, I tried to disable dmp but failed to do so.
After applied MP3, this bug was resolved and after disabled dmp, 3 iscsi disks were presented to vxvm properly.
Now I got a cluster shared storage (3 iscsi disks actually), running on top of two server slave (master), test01 (slave)
slave:/ #vxdctl -c mode
mode: enabled: cluster active - MASTER
master: slave
slave:/ #df -h | grep shared
/dev/vx/dsk/mydg/vol01 2.0G 34M 1.8G 2% /sharedfs
slave:/ #
slave:/ #
slave:/ #
slave:/ #vxdctl -c mode
mode: enabled: cluster active - MASTER
master: slave
slave:/ #
slave:/ #vxdg list
NAME STATE ID
mydg enabled,shared,cds 1168567479.12.slave
slave:/ #vxdg list mydg
Group: mydg
dgid: 1168567479.12.slave
import-id: 33792.8
flags: shared cds
version: 140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw
ssb: on
autotagging: on
detach-policy: global
dg-fail-policy: dgdisable
copies: nconfig=default nlog=default
config: seqno=0.1044 permlen=48144 free=48140 templen=3 loglen=7296
config disk c2t1d0s2 copy 1 len=48144 state=clean online
log disk c2t1d0s2 copy 1 len=7296
slave:/ #
slave:/ #df -h
Filesystem size used avail capacity Mounted on
/dev/dsk/c0t1d0s0 19G 9.0G 10G 48% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 4.6G 1.6M 4.6G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
fd 0K 0K 0K 0% /dev/fd
swap 4.6G 120K 4.6G 1% /tmp
swap 4.6G 32K 4.6G 1% /var/run
swap 4.6G 0K 4.6G 0% /dev/vx/dmp
swap 4.6G 0K 4.6G 0% /dev/vx/rdmp
/dev/dsk/c0t1d0s3 19G 6.0G 13G 32% /data
/dev/odm 0K 0K 0K 0% /dev/odm
/dev/vx/dsk/mydg/vol01
2.0G 34M 1.8G 2% /sharedfs
slave:/ #grep shared /etc/mnttab
/dev/vx/dsk/mydg/vol01 /sharedfs vxfs rw,suid,delaylog,largefiles,qio,cluster,ioerror=mdisable,crw,dev=4f8b3b0 1168663491
slave:/ #test01:/ #vxdg list
NAME STATE ID
mydg enabled,shared,cds 1168567479.12.slave
test01:/ #vxdg list mydg
Group: mydg
dgid: 1168567479.12.slave
import-id: 33792.8
flags: shared cds
version: 140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw
ssb: on
autotagging: on
detach-policy: global
dg-fail-policy: dgdisable
copies: nconfig=default nlog=default
config: seqno=0.1044 permlen=0 free=0 templen=0 loglen=0
test01:/ #
test01:/ #df -h
Filesystem size used avail capacity Mounted on
/dev/dsk/c0t2d0s0 19G 6.2G 13G 33% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 4.3G 1.6M 4.3G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
fd 0K 0K 0K 0% /dev/fd
swap 4.3G 120K 4.3G 1% /tmp
swap 4.3G 32K 4.3G 1% /var/run
swap 4.3G 0K 4.3G 0% /dev/vx/dmp
swap 4.3G 0K 4.3G 0% /dev/vx/rdmp
/dev/odm 0K 0K 0K 0% /dev/odm
/dev/vx/dsk/mydg/vol01
2.0G 34M 1.8G 2% /sharedfs
test01:/ #grep sharedfs /etc/mnttab
/dev/vx/dsk/mydg/vol01 /sharedfs vxfs rw,suid,delaylog,largefiles,qio,cluster,ioerror=mdisable,crw,dev=4f8b3b0 1168663180
test01:/ #