cancel
Showing results for 
Search instead for 
Did you mean: 

SCSI-3 or not to SCSI-3 ?!?

Fred2010
Level 6

Hi,

I am about to setup a new 3 node cluster, using Storage Foundation for Windows HA (SFW HA version 5.1 SP2 downloaded).

This Cluster is going to be a 3 Node Netbackup Masterserver (NBU 7.1 FA)

In the documentation I find remarks over using SCSI-3 Persistent Group Reservations, but it is not clear to me whether I should (Or even can!) use them or not.

My Environment:


 

  • 3 x Windows 2008 R2 x64 Enterprise
  • SFW HA 5.1 SP2
  • EMC Clariion CX3-80 (Flare 26 subversion 025) - Should be SCSI-3 Compliant (Right?!?)
  • PowerPath 5.5.0 x64 (So I am not using the Dynamic MultiPathing (DMP) from Veritas in this case)
     

My questions:

1) Can I use SCSI-3 Persistent Group Reservervations in this setup?
2) What is the advantage for me to use SCSI-3 PGR instead of using SCSI-2 (Which seems to be the default)
3) Are there any disadvantages/Gotcha's in using SCSI-3 angry

Hope someone has experience with this subject and can advise me!

Thank you in advance!

Fred

1 ACCEPTED SOLUTION

Accepted Solutions

rhanley
Level 4

Hi Fredster,

    I should be able to answer your questions here:

First off, your array: EMC Clariion CX3-80 (Flare 26 subversion 025)

You can check compatability from our 5.1 SP2 HCL which can be found here: http://www.symantec.com/docs/TECH138719

In the case of using this with PowerPath 5.5.0, this has not been tested, but this should work as stated in the HCL notes:

Note 1: The 3rd party multi-pathing support noted for this array or array family should be compatible, but has not been explicitly tested by Symantec with this particular array or array family at this time.

The 2nd note for (Note 5) also shows that support includeds the CX3-80 Array:

EMC CLARiiON CX Series arrays supported include the CX200, CX300, CX400, CX500, CX600 and CX700. EMC CLARiiON CX-3 Ultrascale Series arrays supported include the CX3-10, CX3-20, CX3-40 and CX3-80 arrays. EMC CLARiiON AX Series arrays supported include the AX100, AX150, AX4.

I have reached out to the team responsible for this testing requesting additional information (confirm that 5.5 is supported and if there are any plans to test this in the future). I will reply to this thread as soon as I have that detail.

1) Can I use SCSI-3 Persistent Group Reservervations in this setup?

Yes you can, as long as your array is configured to support this (which it should as you stated). You can use a utility we have (scsicmd) to run a SCSI3 Test to make certain this works properly before making the change. More information on this utility can be found here: http://www.symantec.com/business/support/index?page=content&id=TECH72086

Note: The SCSI3_Test command and example are the very last example shown in this technote. At the beginning of this technote is a url where the scsicmd utility can be downloaded (ftp.veritas.com/pub/support/vmtools - Filename: scsicmd.zip).

When running the command, just choose any of the harddisks from the array to perform the test on (you do not need to test each lun, just one from each array).

2) What is the advantage for me to use SCSI-3 PGR instead of using SCSI-2 (Which seems to be the default).

SCSI-3 is preferred because of the way it handles reserving/releasing reservations is more efficient than SCSI-2 so there may be slight performance increases from using SCSI-3 over SCSI-2. This is very minimal and in most configurations, changing between SCSI-2 and SCSI-3 would not be noticeable and both would function properly.

However, SCSI-3 is required in a cluster if Active/Active multipathing is being used. With Clariion arrays, you can use Active/Passive Concurrent which supports SCSI-2, but if you are using ALUA, then you will need to use SCSI-3.

3) Are there any disadvantages/Gotcha's in using SCSI-3 angry

As stated, SCSI-3 is more efficient in multipathing as all paths share the same reservation key. In an Active/Active multipathing configuration, all paths share the same reservation key so there is no checking, release, renewing reservations with path failovers. With SCSI-2, a reservation is path specfic so after sending I/O down Path1, Path 2 will check for a reservation, release it if one exists, then send a new reservation request, then send the I/O. Though these processes take miliseconds to complete, in large, I/O intesive environments, there may be performance improvement seen by moving from SCSI-2 to SCSI-3.

If your array passes the scsi3_test with the scsicmd utility, then I'd strongly advice using SCSI-3 for the reasons explained above and the fact that all arrays are moving to this (in fact, Windows Failover Cluster in 2008 already requires SCSI-3) and in the future, SCSI-2 will be phased out completely.

On a final note, when switching from SCSI-2 to SCSI-3, you will want to perform the following steps:

1. Take all cluster groups containing VMDg resources offline (or move them to another node in the cluster). If the Quorum is Dynamic (VMDg resource), move this to another node in the cluster.

2. Change from SCSI-2 to SCSI-3 from the VEA console (Control Panel > Storage Agent > System Settings > SCSI Support > SCSI-3 support)

3. From a command prompt, restart the Veritas Storage Agent by running the following 2 commands:
    - net stop vxvm
    - net start vxvm

4. Bring the cluster groups containing VMDg resources back online, or move them back to this node if they were failed over in Step 1.

5. You can then use the scsicmd scsi_reserved_status command to view/confirm the disks are reserved under SCSI-3.
 

Please let me know if you have any additional questions, and I'll also update the post as soon as I've heard back on my queries concerning support for PowerPath 5.5.0.

Cheers,
rhanley

View solution in original post

5 REPLIES 5

rhanley
Level 4

Hi Fredster,

    I should be able to answer your questions here:

First off, your array: EMC Clariion CX3-80 (Flare 26 subversion 025)

You can check compatability from our 5.1 SP2 HCL which can be found here: http://www.symantec.com/docs/TECH138719

In the case of using this with PowerPath 5.5.0, this has not been tested, but this should work as stated in the HCL notes:

Note 1: The 3rd party multi-pathing support noted for this array or array family should be compatible, but has not been explicitly tested by Symantec with this particular array or array family at this time.

The 2nd note for (Note 5) also shows that support includeds the CX3-80 Array:

EMC CLARiiON CX Series arrays supported include the CX200, CX300, CX400, CX500, CX600 and CX700. EMC CLARiiON CX-3 Ultrascale Series arrays supported include the CX3-10, CX3-20, CX3-40 and CX3-80 arrays. EMC CLARiiON AX Series arrays supported include the AX100, AX150, AX4.

I have reached out to the team responsible for this testing requesting additional information (confirm that 5.5 is supported and if there are any plans to test this in the future). I will reply to this thread as soon as I have that detail.

1) Can I use SCSI-3 Persistent Group Reservervations in this setup?

Yes you can, as long as your array is configured to support this (which it should as you stated). You can use a utility we have (scsicmd) to run a SCSI3 Test to make certain this works properly before making the change. More information on this utility can be found here: http://www.symantec.com/business/support/index?page=content&id=TECH72086

Note: The SCSI3_Test command and example are the very last example shown in this technote. At the beginning of this technote is a url where the scsicmd utility can be downloaded (ftp.veritas.com/pub/support/vmtools - Filename: scsicmd.zip).

When running the command, just choose any of the harddisks from the array to perform the test on (you do not need to test each lun, just one from each array).

2) What is the advantage for me to use SCSI-3 PGR instead of using SCSI-2 (Which seems to be the default).

SCSI-3 is preferred because of the way it handles reserving/releasing reservations is more efficient than SCSI-2 so there may be slight performance increases from using SCSI-3 over SCSI-2. This is very minimal and in most configurations, changing between SCSI-2 and SCSI-3 would not be noticeable and both would function properly.

However, SCSI-3 is required in a cluster if Active/Active multipathing is being used. With Clariion arrays, you can use Active/Passive Concurrent which supports SCSI-2, but if you are using ALUA, then you will need to use SCSI-3.

3) Are there any disadvantages/Gotcha's in using SCSI-3 angry

As stated, SCSI-3 is more efficient in multipathing as all paths share the same reservation key. In an Active/Active multipathing configuration, all paths share the same reservation key so there is no checking, release, renewing reservations with path failovers. With SCSI-2, a reservation is path specfic so after sending I/O down Path1, Path 2 will check for a reservation, release it if one exists, then send a new reservation request, then send the I/O. Though these processes take miliseconds to complete, in large, I/O intesive environments, there may be performance improvement seen by moving from SCSI-2 to SCSI-3.

If your array passes the scsi3_test with the scsicmd utility, then I'd strongly advice using SCSI-3 for the reasons explained above and the fact that all arrays are moving to this (in fact, Windows Failover Cluster in 2008 already requires SCSI-3) and in the future, SCSI-2 will be phased out completely.

On a final note, when switching from SCSI-2 to SCSI-3, you will want to perform the following steps:

1. Take all cluster groups containing VMDg resources offline (or move them to another node in the cluster). If the Quorum is Dynamic (VMDg resource), move this to another node in the cluster.

2. Change from SCSI-2 to SCSI-3 from the VEA console (Control Panel > Storage Agent > System Settings > SCSI Support > SCSI-3 support)

3. From a command prompt, restart the Veritas Storage Agent by running the following 2 commands:
    - net stop vxvm
    - net start vxvm

4. Bring the cluster groups containing VMDg resources back online, or move them back to this node if they were failed over in Step 1.

5. You can then use the scsicmd scsi_reserved_status command to view/confirm the disks are reserved under SCSI-3.
 

Please let me know if you have any additional questions, and I'll also update the post as soon as I've heard back on my queries concerning support for PowerPath 5.5.0.

Cheers,
rhanley

Fred2010
Level 6

Hi RHanley,

 

First of all for your very detailed and complete answer! Great yes

I am currently not using ALUA (Although the Flare code supports it), coz I really can't figure out what advantages and disadvantages it would give me. I understand the basic working, but just don't see why I would need it blush

As per your suggestion, I've downloaded the SCSIcmd software and will try the test you suggested (Will post results later)

I guess the bottom line of your story is:

- If your array supports SCSI3 PGR, use it (As there are no disadvantages and is future proof)
- If you want the use the array's ALUAfunction, you MUST use SCSI-3 PGR

So my goal will be to setup the cluster using the SCSI-3 mode yes

Thanks!

Fred

Fred2010
Level 6

Testing Disk 002 in our CX3-80 SAN using command 'SCSIcmd -d2 -sscsi3_test'

Looks like it won't be a problem cool

Fred

 

********************** SCSI-3 SUPPORT TEST ***********************************

* The SCSI-3 SUPPORT TEST performs a set of SCSI-3 PR on disk(s) specified in *

* -d option. *

* SCSICMD uses the predefined '1234567812345678' key in SCSI-3 support test. *

* *

* Make sure that the testing shouldn't be under the cluster resource. *

* *

* If SFW DMP (5.0 DDI-3/5.1 GA/5.1 DDI-1 or a newer DDI version) hasn't been *

* installed nor claim the disk, make sure that there is only one HBA path *

* connected to the disk(s) from the testing host before running SCSICMD tool. *

*******************************************************************************

 

Harddisk2

Scsi Address

------------------

Length : 0x8

PortNumber : 0x4

PathId : 0x0

TargetId : 0x0

Lun : 0x1

******** PERFORM SCSI-3 PR OPERATION TESTS ON Harddisk2 *************

 

===>Test #1: Clean up any SCSI-3 keys left on harddisk2

Test #1 - **** PASSED. There is no SCSI-3 key left on the disk: #2.

===> Test #2: Perform SCSI-3 Register action on harddisk2 with SCSI-3 key: 7856341278563412

Test #2 - Successfully issued SCSI-3 Register action.

Test #2 - Checking...There should be a registered key on harddisk2

SCSI3 registered keys on harddisk2

Key: 7856341278563412

Test #2 - *** PASSED. Successfully performed SCSI-3 Register Service Action.

===> Test #3: Perform SCSI-3 reservation action on harddisk2 with SCSI-3 key:7856341278563412

Test #3 - Successfully issued SCSI-3 reservation action.

Test #3 - Checking...Reading the SCSI-3 reservation holder

SCSI3 reservation holder key on harddisk2

Key: 7856341278563412

Test #3 - *** PASSED. Successfully performed SCSI-3 Reservation Action.

==> Test #4: Perform SCSI-3 release action on harddisk2 with SCSI-3 key: 7856341278563412

Test #4 - Successfully issued SCSI-3 release action.

Test #4 - Checking...There should be none SCSI-3 reservation holder left on the disk.

Test #4 - There is no SCSI-3 reservation holder left on harddisk2.

Test #4 - Checking...There should be a SCSI-3 registered key left on the disk.

SCSI3 registered keys on harddisk2

Key: 7856341278563412

Test #4 - *** PASSED. Successfully performed SCSI-3 Release Action.

==> Test #5: Perform a part of SCSI-3 preempt action test on harddisk2.

Test #5 - Testing the support of SCSI-3 preempt PR.

 

Test #5 - *** PASSED. Successfully performed a part of SCSI-3 Preempt Service Action test.

===> Test #6: Perform SCSI-3 Register Ignore Existing action on harddisk2.

Test #6 - Perform SCSI-3 Register Ignore Existing key action on harddisk2 with the new SCSI-3 key: 8344838F997E1DB9

Test #6 - Checking...There should be a new SCSI-3 registered on the disk. The registered key: 8344838F997E1DB9

SCSI3 registered keys on harddisk2

Key: 8344838F997E1DB9

Test #6 - *** PASSED. Successfully performed SCSI-3 Register Ignore Existing Service Action.

===> Test #7: Perform SCSI-3 Unregister Service Action on harddisk2.

Test #7 - Checking...There should be no SCSI-3 key left on the harddisk2.

Test #7 - *** PASSED. Successfully performed SCSI-3 UnRegister Service Action.

 

===> Test #8: Perform SCSI-3 Clear Service Action on harddisk2.

Test #8 - Performing SCSI-3 Register action on harddisk2 with SCSI-3 key: 7856341278563412

Test #8 - Performing SCSI-3 Reservation action on harddisk2 with SCSI-3 key: 7856341278563412

Test #8 - Performing SCSI-3 Clear Service Action

Test #8 - Checking...There should be no SCSI-3 key left on the disk

Test #8 - *** PASSED. Successfully performed SCSI-3 UnRegister Service Action

 

====> SCSI-3 PERSISTENT OPERATION TESTS: All tests are PASSED on harddisk2.

 

  

rhanley
Level 4

Hi Fredster,

    That's exactly what you want to see, so you should be good to move forward with SCSI-3 :).

Cheers,
rhanley
 

Fred2010
Level 6

Lol,

As it seems, we were already running on SCSI3 in the old environment (SFW 5.1 SP1), without really knowing why and if it was fully supported...

At least now I know the advantages and can actually test for SCSI3 compliance using the tool ;)

Thanks again!

Fred