cancel
Showing results for 
Search instead for 
Did you mean: 

problem with cvm/cfs

Eric_Gao
Level 4

After install license properly, I began the testing of cvm/cfs.

 

I had already created a shared disk group, and activation mode was set to sw (shared write),  while I tried to create the vxfs file system, it always reported error "cluster reservation failed for volume"

 

Does anyone know something about this?  thanks.

 

slave:/ #vxdg list mydg
Group:     mydg
dgid:      1168495239.16.slave
import-id: 33792.8
flags:     shared cds                                          ---> shared dg
version:   140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw              ----> shared write
ssb:            on
autotagging:    on
detach-policy: global
dg-fail-policy: dgdisable
copies:    nconfig=default nlog=default
config:    seqno=0.1052 permlen=48144 free=48139 templen=3 loglen=7296
config disk c2t1d0s2 copy 1 len=48144 state=clean online
config disk c2t2d0s2 copy 1 len=48144 state=clean online
config disk c2t4d0s2 copy 1 len=48144 state=clean online
log disk c2t1d0s2 copy 1 len=7296
log disk c2t2d0s2 copy 1 len=7296
log disk c2t4d0s2 copy 1 len=7296
slave:/ #
slave:/ #mkfs -F vxfs /dev/vx/rdsk/mydg/vol01
UX:vxfs mkfs: ERROR: V-3-21018: cluster reservation failed for volume /dev/vx/rdsk/mydg/vol01
slave:/ #

1 ACCEPTED SOLUTION

Accepted Solutions

Eric_Gao
Level 4

problem is resolved.

 

Basically this is because the partial installation of cluster file system.  Error "cluster reservation failed " is due to the abasence of glm module (group lock manager).  After I noticed that VRTSglm is missing, I first tried manuall installation of this package but still couldn't open port f, that being used by cfs.

 

I deleted everything and re-installed storage foundation cluster file system and all went well afterwards.

 

Also as I am using 3 iscsi disks (they are created from one server of the cluster, so basically there is no real shared storeage),  with dmp enabled, all 3 iscsi disks were probed into one disk with 3 paths,  that is becuase the solaris soft iscsi disks don't provide serial number, as a workaround, I tried to disable dmp but failed to do so. 

 

After applied MP3, this bug was resolved and after disabled dmp, 3 iscsi disks were presented to vxvm properly.

 

Now I got a cluster shared storage (3 iscsi disks actually), running on top of two server  slave (master), test01  (slave)

 

slave:/ #vxdctl -c mode
mode: enabled: cluster active - MASTER
master: slave
slave:/ #df -h | grep shared
/dev/vx/dsk/mydg/vol01   2.0G    34M   1.8G     2%    /sharedfs
slave:/ #
slave:/ #
slave:/ #
slave:/ #vxdctl -c mode
mode: enabled: cluster active - MASTER
master: slave
slave:/ #
slave:/ #vxdg list
NAME         STATE           ID
mydg         enabled,shared,cds   1168567479.12.slave
slave:/ #vxdg list mydg
Group:     mydg
dgid:      1168567479.12.slave
import-id: 33792.8
flags:     shared cds
version:   140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw
ssb:            on
autotagging:    on
detach-policy: global
dg-fail-policy: dgdisable
copies:    nconfig=default nlog=default
config:    seqno=0.1044 permlen=48144 free=48140 templen=3 loglen=7296
config disk c2t1d0s2 copy 1 len=48144 state=clean online
log disk c2t1d0s2 copy 1 len=7296
slave:/ #
slave:/ #df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t1d0s0       19G   9.0G    10G    48%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   4.6G   1.6M   4.6G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
fd                       0K     0K     0K     0%    /dev/fd
swap                   4.6G   120K   4.6G     1%    /tmp
swap                   4.6G    32K   4.6G     1%    /var/run
swap                   4.6G     0K   4.6G     0%    /dev/vx/dmp
swap                   4.6G     0K   4.6G     0%    /dev/vx/rdmp
/dev/dsk/c0t1d0s3       19G   6.0G    13G    32%    /data
/dev/odm                 0K     0K     0K     0%    /dev/odm
/dev/vx/dsk/mydg/vol01
                       2.0G    34M   1.8G     2%    /sharedfs
slave:/ #grep shared /etc/mnttab
/dev/vx/dsk/mydg/vol01  /sharedfs       vxfs    rw,suid,delaylog,largefiles,qio,cluster,ioerror=mdisable,crw,dev=4f8b3b0        1168663491
slave:/ #

 

test01:/ #vxdg list
NAME         STATE           ID
mydg         enabled,shared,cds   1168567479.12.slave
test01:/ #vxdg list mydg
Group:     mydg
dgid:      1168567479.12.slave
import-id: 33792.8
flags:     shared cds
version:   140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw
ssb:            on
autotagging:    on
detach-policy: global
dg-fail-policy: dgdisable
copies:    nconfig=default nlog=default
config:    seqno=0.1044 permlen=0 free=0 templen=0 loglen=0
test01:/ #
test01:/ #df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t2d0s0       19G   6.2G    13G    33%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   4.3G   1.6M   4.3G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
fd                       0K     0K     0K     0%    /dev/fd
swap                   4.3G   120K   4.3G     1%    /tmp
swap                   4.3G    32K   4.3G     1%    /var/run
swap                   4.3G     0K   4.3G     0%    /dev/vx/dmp
swap                   4.3G     0K   4.3G     0%    /dev/vx/rdmp
/dev/odm                 0K     0K     0K     0%    /dev/odm
/dev/vx/dsk/mydg/vol01
                       2.0G    34M   1.8G     2%    /sharedfs
test01:/ #grep sharedfs /etc/mnttab
/dev/vx/dsk/mydg/vol01  /sharedfs       vxfs    rw,suid,delaylog,largefiles,qio,cluster,ioerror=mdisable,crw,dev=4f8b3b0        1168663180
test01:/ #

 

 

View solution in original post

3 REPLIES 3

bsobek
Level 5

Hi,

 

you can only create vx-objectson the master-node:

 

Try this command:

# vxdctl -c mode
mode: enabled: cluster active - SLAVE
master: <SYS>

 

Greets

Bjoern

 

Eric_Gao
Level 4

yes of course I am operating on master node.

 

my cluster consists of two servers, slave and test01.  slave is master and test01 is slave.

Message Edited by Eric Gao on 02-24-2009 05:17 AM

Eric_Gao
Level 4

problem is resolved.

 

Basically this is because the partial installation of cluster file system.  Error "cluster reservation failed " is due to the abasence of glm module (group lock manager).  After I noticed that VRTSglm is missing, I first tried manuall installation of this package but still couldn't open port f, that being used by cfs.

 

I deleted everything and re-installed storage foundation cluster file system and all went well afterwards.

 

Also as I am using 3 iscsi disks (they are created from one server of the cluster, so basically there is no real shared storeage),  with dmp enabled, all 3 iscsi disks were probed into one disk with 3 paths,  that is becuase the solaris soft iscsi disks don't provide serial number, as a workaround, I tried to disable dmp but failed to do so. 

 

After applied MP3, this bug was resolved and after disabled dmp, 3 iscsi disks were presented to vxvm properly.

 

Now I got a cluster shared storage (3 iscsi disks actually), running on top of two server  slave (master), test01  (slave)

 

slave:/ #vxdctl -c mode
mode: enabled: cluster active - MASTER
master: slave
slave:/ #df -h | grep shared
/dev/vx/dsk/mydg/vol01   2.0G    34M   1.8G     2%    /sharedfs
slave:/ #
slave:/ #
slave:/ #
slave:/ #vxdctl -c mode
mode: enabled: cluster active - MASTER
master: slave
slave:/ #
slave:/ #vxdg list
NAME         STATE           ID
mydg         enabled,shared,cds   1168567479.12.slave
slave:/ #vxdg list mydg
Group:     mydg
dgid:      1168567479.12.slave
import-id: 33792.8
flags:     shared cds
version:   140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw
ssb:            on
autotagging:    on
detach-policy: global
dg-fail-policy: dgdisable
copies:    nconfig=default nlog=default
config:    seqno=0.1044 permlen=48144 free=48140 templen=3 loglen=7296
config disk c2t1d0s2 copy 1 len=48144 state=clean online
log disk c2t1d0s2 copy 1 len=7296
slave:/ #
slave:/ #df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t1d0s0       19G   9.0G    10G    48%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   4.6G   1.6M   4.6G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
fd                       0K     0K     0K     0%    /dev/fd
swap                   4.6G   120K   4.6G     1%    /tmp
swap                   4.6G    32K   4.6G     1%    /var/run
swap                   4.6G     0K   4.6G     0%    /dev/vx/dmp
swap                   4.6G     0K   4.6G     0%    /dev/vx/rdmp
/dev/dsk/c0t1d0s3       19G   6.0G    13G    32%    /data
/dev/odm                 0K     0K     0K     0%    /dev/odm
/dev/vx/dsk/mydg/vol01
                       2.0G    34M   1.8G     2%    /sharedfs
slave:/ #grep shared /etc/mnttab
/dev/vx/dsk/mydg/vol01  /sharedfs       vxfs    rw,suid,delaylog,largefiles,qio,cluster,ioerror=mdisable,crw,dev=4f8b3b0        1168663491
slave:/ #

 

test01:/ #vxdg list
NAME         STATE           ID
mydg         enabled,shared,cds   1168567479.12.slave
test01:/ #vxdg list mydg
Group:     mydg
dgid:      1168567479.12.slave
import-id: 33792.8
flags:     shared cds
version:   140
alignment: 8192 (bytes)
local-activation: shared-write
cluster-actv-modes: slave=sw test01=sw
ssb:            on
autotagging:    on
detach-policy: global
dg-fail-policy: dgdisable
copies:    nconfig=default nlog=default
config:    seqno=0.1044 permlen=0 free=0 templen=0 loglen=0
test01:/ #
test01:/ #df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t2d0s0       19G   6.2G    13G    33%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   4.3G   1.6M   4.3G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
fd                       0K     0K     0K     0%    /dev/fd
swap                   4.3G   120K   4.3G     1%    /tmp
swap                   4.3G    32K   4.3G     1%    /var/run
swap                   4.3G     0K   4.3G     0%    /dev/vx/dmp
swap                   4.3G     0K   4.3G     0%    /dev/vx/rdmp
/dev/odm                 0K     0K     0K     0%    /dev/odm
/dev/vx/dsk/mydg/vol01
                       2.0G    34M   1.8G     2%    /sharedfs
test01:/ #grep sharedfs /etc/mnttab
/dev/vx/dsk/mydg/vol01  /sharedfs       vxfs    rw,suid,delaylog,largefiles,qio,cluster,ioerror=mdisable,crw,dev=4f8b3b0        1168663180
test01:/ #