06-27-2013 07:36 PM
Hi,
I have a two node cluster. I am trying to create a simple service group with VxFS on VxVM. I have followed the steps outlined in http://www.symantec.com/business/support/index?page=content&id=TECH20539.
When I try to bring the service group online, the diskgroup would fail to import with "groupreserve option as scsi3pr is disabled". I've checked the diskgroup for scsi3-pr compliance with the vxfentsthdw utility. It is passing as shown below.
Can you please tell me how can I enable scsi3pr on the diskgroup so it can online successfully.
Thanks,
David
################### Diskgroup Service online Error ####################################
Jun 27 15:56:36 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-10001-1095 (sca-m532-504-gd10) DiskGroup:data_dg:online:Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled.
Jun 27 15:58:37 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13066 (sca-m532-504-gd10) Agent is calling clean for resource(data_dg) because the resource is not up even after online completed.
Jun 27 15:58:39 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-10001-1095 (sca-m532-504-gd10) DiskGroup:data_dg:online:Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled.
Jun 27 16:00:40 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13066 (sca-m532-504-gd10) Agent is calling clean for resource(data_dg) because the resource is not up even after online completed.
Jun 27 16:00:42 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-1-54031 Resource data_dg (Owner: Unspecified, Group: datagrp) is FAULTED on sys sca-m532-504-gd10
Jun 27 16:00:42 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-1-10205 Group datagrp is faulted on system sca-m532-504-gd10
################# SCSI3-PR Test of Diskgroup ######################################
sca-m532-504-gd9:/root
[rootD:]825> /opt/VRTS/bin/vxfentsthdw -n -c datadg
Veritas vxfentsthdw version 6.0.3 Solaris
The utility vxfentsthdw works on the two nodes of the cluster.
The utility verifies that the shared storage one intends to use is
configured to support I/O fencing. It issues a series of vxfenadm
commands to setup SCSI-3 registrations on the disk, verifies the
registrations on the disk, and removes the registrations from the disk.
The logfile generated for vxfentsthdw is /var/VRTSvcs/log/vxfen/vxfentsthdw.log.10882
Enter the first node of the cluster:
sca-m532-504-gd9
Enter the second node of the cluster:
sca-m532-504-gd10
********************************************
Testing sca-m532-504-gd9 /dev/vx/rdmp/zfs_71200_3 sca-m532-504-gd10 /dev/vx/rdmp/zfs_71200_3
Evaluate the disk before testing ........................ No Pre-existing keys
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_3 from node sca-m532-504-gd9 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd9 Passed
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_3 from node sca-m532-504-gd10 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd9 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd10 Passed
Preempt and abort key KeyA using key KeyB on node sca-m532-504-gd10 .... Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd9 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd10 Passed
Remove key KeyB on node sca-m532-504-gd10 .............................. Passed
Check to verify there are no keys from node sca-m532-504-gd9 ........... Passed
ALL tests on the disk /dev/vx/rdmp/zfs_71200_3 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd9
as a COORDINATOR DISK.
ALL tests on the disk /dev/vx/rdmp/zfs_71200_3 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd10
as a COORDINATOR DISK.
********************************************
Testing sca-m532-504-gd9 /dev/vx/rdmp/zfs_71200_4 sca-m532-504-gd10 /dev/vx/rdmp/zfs_71200_4
Evaluate the disk before testing ........................ No Pre-existing keys
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_4 from node sca-m532-504-gd9 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd9 Passed
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_4 from node sca-m532-504-gd10 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd9 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd10 Passed
Preempt and abort key KeyA using key KeyB on node sca-m532-504-gd10 .... Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd9 Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd10 Passed
Remove key KeyB on node sca-m532-504-gd10 .............................. Passed
Check to verify there are no keys from node sca-m532-504-gd9 ........... Passed
ALL tests on the disk /dev/vx/rdmp/zfs_71200_4 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd9
as a COORDINATOR DISK.
ALL tests on the disk /dev/vx/rdmp/zfs_71200_4 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd10
as a COORDINATOR DISK.
Removing test keys and temporary files, if any...
sca-m532-504-gd9:/root
[rootD:]826>
Solved! Go to Solution.
06-27-2013 10:51 PM
From the message, it loks like SCSI3-PR is disabled on the system
You can verify it from
# cat /etc/vx/volboot | grep scsi3_pr
If you find it ias off, enable SCSI3-PR with following command.
# vxdctl scsi3pr on
Now clear the fault and online the resource again.
Just for clarity,
After the Error message "Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled." another Info message should appear as follows.
Enable the scsi3pr option and retry online. To enable scsi3pr run the command [vxdctl scsi3pr on].
Do you have this message in your engine log?
06-27-2013 08:21 PM
Forgot to add my environment. I'm running with SF_HA 6.0.3 with Solaris 11.1.
06-27-2013 10:51 PM
From the message, it loks like SCSI3-PR is disabled on the system
You can verify it from
# cat /etc/vx/volboot | grep scsi3_pr
If you find it ias off, enable SCSI3-PR with following command.
# vxdctl scsi3pr on
Now clear the fault and online the resource again.
Just for clarity,
After the Error message "Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled." another Info message should appear as follows.
Enable the scsi3pr option and retry online. To enable scsi3pr run the command [vxdctl scsi3pr on].
Do you have this message in your engine log?
06-28-2013 12:59 AM
Thanks! My resources can come online after I enabled the scsi3-pr on the system with "vxdctl scsi3pr on"
06-28-2013 02:26 PM
Hi,
The solution shown above does not seem to persist across reboots. After I reboot the system, "scsi3_pr off" reappeared in /etc/vx/volboot. Is that a normal behavior? How can I make it persist across reboots?
Thanks in advance for you help.
07-01-2013 05:15 AM
Hi,
The scsi3pr value will be persistent in volboot file. Need to check if there is any script that is setting it to disabled after reboot. Is there any such command in /etc/vx/log/cmdlog that during boot time “vxdctl scsi3pr off”?
Also compare the /etc/vx/volboot file before and after setting it the scsi3pr to on, before reboot and after reboot to rule out any case where the updations is failing.
07-03-2013 11:29 AM
Hi Sajith,
The command log does shows that there are "vxdctl scsi3pr off" commands. Where can I look to find the startup scripts for VCS that disables this. Is there any commands to run that may remove this setting from the startup scripts?
Thanks!!! Really appreciate your help!!!
07-08-2013 12:28 AM
Hi,
VCS start up scripts do not disable scsi3pr settings.
I recommend to post this question on VxVM forum.