cancel
Showing results for 
Search instead for 
Did you mean: 

CFS Health Check ?

Haniwa
Level 4
Partner Accredited Certified

Hi All,

We have recently built a 4-node Oracle single-instance CFS cluster, with more to on the way.

The cluster will host many databases, and all nodes will be active. The databases will migrate from non-clustered to CFS clustered servers, and we are concerned about what effect CFS may have on performance.

Concerns:

  - Suitability of our LLT links to handle intra-node communication generated by CFS ?

  - Location of CFS primary, for each filesytem, and how to determine if it is optimal ?

  - When multiple nodes write to the same filesystem simultaeously, would that be detrimental to performance ?

  - I am aware of GLM lock/release as one source of CFS intra-node traffic, but wondering if there are other significant sources, such as VxFS cache coherency ?

  - How to best monitor CFS overhead, intra-node traffic, and overall health ?


Thank your for reading !

Ken W

---------------------Configuration---------------------

Red Hat RHEL 6.5

SFCFSHA 6.1.1 with P200 patch

2 x 1 Gbit ethernet as LLT links, on dedicated VLANs

Oracle 11.2.0.3 and 12.1.0.2

Oracle code local to each node (outside of cluster control)

Oracle data files on CFS filesystem dedicated to only datases running on that node.

Oracle Redo on a single shared filesytem for all nodes.

-------------------------------------------------------------

 

 

 

 

2 ACCEPTED SOLUTIONS

Accepted Solutions

starflyfly
Level 6
Employee Accredited Certified
Hi, Ken In a summary, you can use common performance tools like : top, vmstat etc to monitor CFS system performance, cpu usage, memory usage, io busy. Several suggestions: 1. use fsadm -DE to check fragment report of file system, avoid fragmentation. 2. avoid too many small files under one directory. 3. multinode nodes write to a file system simultaeously, try to put data in different directory. More refer guide https://sort.symantec.com/public/documents/sfha/6.1/linux/productguides/html/sfcfs_admin/pt05.htm About your concern: - Suitability of our LLT links to handle intra-node communication generated by CFS ? <

View solution in original post

Haniwa
Level 4
Partner Accredited Certified

StarFly... Thank you for CFS best practice recommendation.

In addition, I have found...

1. Monitor LLT link utilization at the switch VLAN level using Cisco tools.

2. Very little performance degrade when CFS primary is located on an "off" node (non-writer).

3. CFS primary can be manually set after CFS is onlined (non-persistent).

         # fsclustadm setprimary /mnt    (sets to this node)

         # fsclustadm show primary /mnt    (report current primary)

4. CFS "Policy" attribute will set the primary automatically every time CFS is onlined, from a list of primaries. This is a persistent setting.

         # hares -modify <CFSMount_resource> Policy NodeB NodeA

5. Found 'glmstat' command but could not locate any use cases.

6. Recommended articles:

        How to set primary node for a cluster file system
        http://www.symantec.com/business/support/index?page=content&id=TECH39266

        Moving CFS Primary when using SFCFS for fast failover (mikebounds)
        https://www-secure.symantec.com/connect/forums/moving-cfs-primary-when-using-sfcfs-fast-failover

Thank You,

Ken W

 

 

 

 

 

View solution in original post

3 REPLIES 3

starflyfly
Level 6
Employee Accredited Certified
Hi, Ken In a summary, you can use common performance tools like : top, vmstat etc to monitor CFS system performance, cpu usage, memory usage, io busy. Several suggestions: 1. use fsadm -DE to check fragment report of file system, avoid fragmentation. 2. avoid too many small files under one directory. 3. multinode nodes write to a file system simultaeously, try to put data in different directory. More refer guide https://sort.symantec.com/public/documents/sfha/6.1/linux/productguides/html/sfcfs_admin/pt05.htm About your concern: - Suitability of our LLT links to handle intra-node communication generated by CFS ? <

Haniwa
Level 4
Partner Accredited Certified

StarFly... Thank you for CFS best practice recommendation.

In addition, I have found...

1. Monitor LLT link utilization at the switch VLAN level using Cisco tools.

2. Very little performance degrade when CFS primary is located on an "off" node (non-writer).

3. CFS primary can be manually set after CFS is onlined (non-persistent).

         # fsclustadm setprimary /mnt    (sets to this node)

         # fsclustadm show primary /mnt    (report current primary)

4. CFS "Policy" attribute will set the primary automatically every time CFS is onlined, from a list of primaries. This is a persistent setting.

         # hares -modify <CFSMount_resource> Policy NodeB NodeA

5. Found 'glmstat' command but could not locate any use cases.

6. Recommended articles:

        How to set primary node for a cluster file system
        http://www.symantec.com/business/support/index?page=content&id=TECH39266

        Moving CFS Primary when using SFCFS for fast failover (mikebounds)
        https://www-secure.symantec.com/connect/forums/moving-cfs-primary-when-using-sfcfs-fast-failover

Thank You,

Ken W

 

 

 

 

 

starflyfly
Level 6
Employee Accredited Certified
Very good adds.