cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Veritas.com
Support

NFS & share resource can't be online after configuring sfcfsha

yoyohimhim
Level 3

‎01-27-2010 02:29 AM

Hi,
    I installed and configured sfcfsha (SF5.1) on two SLES11(x86_64) with IO-fence disable mode.  Share group and share disks have been configured by vxvm. But after configuration everything, NFS and share resources can't be online.(Disks can be mounted on both nodes).
Can anyone help me? Thanks in advance.
1. Before configuring NFS:
arcx3455gbxv5:/opt/VRTS/bin # /opt/VRTS/bin/cfscluster  status

  Node             :  arcx3455frpv8
  Cluster Manager  :  running
  CVM state        :  running
  No mount point registered with cluster configuration


  Node             :  arcx3455gbxv5
  Cluster Manager  :  running
  CVM state        :  running
  No mount point registered with cluster configuration
*************************************************************
arcx3455gbxv5:/opt/VRTS/bin # vxdg list
NAME         STATE           ID
rootdg       enabled              1263890629.7.arcx3455gbxv5
cfs          enabled,shared,cds   1264561164.23.arcx3455gbxv5
qlogic       enabled,shared,cds   1264492611.18.arcx3455gbxv5
**************************************************************
arcx3455gbxv5:/opt/VRTS/bin # vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
san_vc0_1    auto:cdsdisk    -            -            online
san_vc0_2    auto:cdsdisk    -            -            online
san_vc0_3    auto:cdsdisk    -            -            online
san_vc0_4    auto:cdsdisk    qlogic01     qlogic       online shared
san_vc0_5    auto:cdsdisk    qlogic02     qlogic       online shared
san_vc0_6    auto:cdsdisk    qlogic03     qlogic       online shared
san_vc0_7    auto:cdsdisk    qlogic04     qlogic       online shared
san_vc0_8    auto:cdsdisk    cfs01        cfs          online shared
sda          auto:sliced     rootdisk     rootdg       online
(cfs group and cfs01 disk are for CNFS components)
2. Configure a VCS configuration for CFS/CVM
 arcx3455gbxv5:/opt/VRTS/bin # cfscluster config
  Warning: V-35-420: CVM Configuration already exists
  Warning: V-35-419: Cluster already configured
  cfscluster: CFS Cluster Configured Successfully
3.Configure the CNFS components
arcx3455gbxv5:/opt/VRTS/bin # cfsshare config cfs c01 /locks
  Mount Point is being added...
  /locks added to the cluster-configuration
  Mounting...
  [/dev/vx/dsk/cfs/c01] mounted successfully at /locks on arcx3455frpv8
  [/dev/vx/dsk/cfs/c01] mounted successfully at /locks on arcx3455gbxv5
But warning  and errormessages appear in /opt/VRTvcs/log/engineA.log

2010/01/27 03:05:25 VCS NOTICE V-16-1-10301 Initiating Online of Resource nfs (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:05:26 VCS WARNING V-16-10031-7002 (arcx3455frpv8) NFS:nfs:online:Sending SIGKILL signal to nfsd (11174 11175 11176 11177 11178 11179 11180 11181).
2010/01/27 03:05:26 VCS INFO V-16-2-13716 (arcx3455frpv8) Resource(nfs): Output of the completed operation (online)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:07:06 VCS INFO V-16-1-50135 User root fired command: hares -online nfs  arcx3455gbxv5  from localhost
2010/01/27 03:07:06 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455frpv8
2010/01/27 03:07:06 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455gbxv5
2010/01/27 03:07:06 VCS NOTICE V-16-1-10301 Initiating Online of Resource nfs (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:07:06 VCS WARNING V-16-10031-7002 (arcx3455gbxv5) NFS:nfs:online:Sending SIGKILL signal to nfsd (11626 11627 11628 11629 11630 11631 11632 11633).
2010/01/27 03:07:07 VCS INFO V-16-2-13716 (arcx3455gbxv5) Resource(nfs): Output of the completed operation (online)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:07:28 VCS ERROR V-16-2-13066 (arcx3455frpv8) Agent is calling clean for resource(nfs) because the resource is not up even after online completed.
2010/01/27 03:07:28 VCS WARNING V-16-10031-7002 (arcx3455frpv8) NFS:nfs:clean:Sending SIGKILL signal to nfsd (13067 13068 13069 13070 13071 13072 13073 13074).
2010/01/27 03:07:29 VCS INFO V-16-2-13716 (arcx3455frpv8) Resource(nfs): Output of the completed operation (clean)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:07:29 VCS INFO V-16-2-13068 (arcx3455frpv8) Resource(nfs) - clean completed successfully.
2010/01/27 03:07:29 VCS INFO V-16-2-13071 (arcx3455frpv8) Resource(nfs): reached OnlineRetryLimit(0).
2010/01/27 03:07:29 VCS ERROR V-16-1-10303 Resource nfs (Owner: unknown, Group: cfsnfssg) is FAULTED (timed out) on sys arcx3455frpv8
2010/01/27 03:07:29 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cfsnfs_locks (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:07:30 VCS INFO V-16-1-10305 Resource cfsnfs_locks (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (VCS initiated)
2010/01/27 03:07:30 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cvmvoldg1 (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:07:32 VCS INFO V-16-1-10305 Resource cvmvoldg1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (VCS initiated)
2010/01/27 03:07:32 VCS ERROR V-16-1-10205 Group cfsnfssg is faulted on system arcx3455frpv8
2010/01/27 03:07:32 VCS NOTICE V-16-1-10446 Group cfsnfssg is offline on system arcx3455frpv8
2010/01/27 03:07:32 VCS INFO V-16-6-0 (arcx3455frpv8) postoffline:Invoked with arg0=arcx3455frpv8, arg1=cfsnfssg
2010/01/27 03:07:32 VCS INFO V-16-6-15002 (arcx3455frpv8) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline arcx3455frpv8 cfsnfssg   successfully
2010/01/27 03:09:09 VCS ERROR V-16-2-13066 (arcx3455gbxv5) Agent is calling clean for resource(nfs) because the resource is not up even after online completed.
2010/01/27 03:09:09 VCS WARNING V-16-10031-7002 (arcx3455gbxv5) NFS:nfs:clean:Sending SIGKILL signal to nfsd (15200 15201 15202 15203 15204 15205 15206 15207).
2010/01/27 03:09:10 VCS INFO V-16-2-13716 (arcx3455gbxv5) Resource(nfs): Output of the completed operation (clean)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:09:10 VCS INFO V-16-2-13068 (arcx3455gbxv5) Resource(nfs) - clean completed successfully.
2010/01/27 03:09:10 VCS INFO V-16-2-13071 (arcx3455gbxv5) Resource(nfs): reached OnlineRetryLimit(0).
2010/01/27 03:09:10 VCS ERROR V-16-1-10303 Resource nfs (Owner: unknown, Group: cfsnfssg) is FAULTED (timed out) on sys arcx3455gbxv5
2010/01/27 03:09:10 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cfsnfs_locks (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:09:11 VCS INFO V-16-1-10305 Resource cfsnfs_locks (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (VCS initiated)
2010/01/27 03:09:11 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cvmvoldg1 (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:09:14 VCS INFO V-16-1-10305 Resource cvmvoldg1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (VCS initiated)
2010/01/27 03:09:14 VCS ERROR V-16-1-10205 Group cfsnfssg is faulted on system arcx3455gbxv5
2010/01/27 03:09:14 VCS NOTICE V-16-1-10446 Group cfsnfssg is offline on system arcx3455gbxv5
2010/01/27 03:09:14 VCS INFO V-16-6-0 (arcx3455gbxv5) postoffline:Invoked with arg0=arcx3455gbxv5, arg1=cfsnfssg
2010/01/27 03:09:14 VCS INFO V-16-6-15002 (arcx3455gbxv5) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline arcx3455gbxv5 cfsnfssg   successfully
********************************************
arcx3455gbxv5:/opt/VRTS/bin # /opt/VRTS/bin/cfscluster  status

  Node             :  arcx3455frpv8
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /locks         c01            cfs               NOT MOUNTED


  Node             :  arcx3455gbxv5
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /locks         c01            cfs               NOT MOUNTED
************************************************************************
4. Add and mount the NFS shared CFS file system to the VCS configuration
arcx3455gbxv5:/opt/VRTS/bin # cfsshare add qlogic q01 /mnt/q01 rw,no_root_squash all=rw                                                                                       
   Mount Point is being added...
  /mnt/q01 added to the cluster-configuration
  Mounting...
  [/dev/vx/dsk/qlogic/q01] mounted successfully at /mnt/q01 on arcx3455frpv8
  [/dev/vx/dsk/qlogic/q01] mounted successfully at /mnt/q01 on arcx3455gbxv5
  Warning: V-35-465: Resource [nfs] is not online on system [arcx3455frpv8].
  Warning: V-35-465: Resource [share1] is not online on system [arcx3455frpv8].
  Warning: V-35-465: Resource [nfs] is not online on system [arcx3455gbxv5].
  Warning: V-35-465: Resource [share1] is not online on system [arcx3455gbxv5].

EngineA.log will be added later.
5.arcx3455gbxv5:/opt/VRTS/bin # /opt/VRTS/bin/cfscluster  status

  Node             :  arcx3455frpv8
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /mnt/q01       q01            qlogic            NOT MOUNTED
  /locks         c01            cfs               NOT MOUNTED


  Node             :  arcx3455gbxv5
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /mnt/q01       q01            qlogic            NOT MOUNTED
  /locks         c01            cfs               NOT MOUNTED

arcx3455gbxv5:/opt/VRTS/bin # cfsshare display
  SHARE RESOURCE    MOUNTPOINT    SHARE OPTIONS
  share1           /mnt/q01   rw,no_root_squash
**************************************************************************
6. arcx3455gbxv5:/opt/VRTS/bin # cat /etc/VRTSvcs/conf/config/main.cf
include "OracleASMTypes.cf"
include "types.cf"
include "ApplicationNone.cf"
include "CFSTypes.cf"
include "CVMTypes.cf"
include "Db2udbTypes.cf"
include "OracleTypes.cf"
include "SybaseTypes.cf"

cluster dqq (
        UserNames = { admin = aHHpIBgGHoEChEIjIE }
        ClusterAddress = "9.11.111.66"
        Administrators = { admin }
        HacliUserLevel = COMMANDROOT
        )

system arcx3455frpv8 (
        )

system arcx3455gbxv5 (
        )

group ClusterService (
        SystemList = { arcx3455gbxv5 = 0, arcx3455frpv8 = 1 }
        AutoStartList = { arcx3455gbxv5, arcx3455frpv8 }
        OnlineRetryLimit = 3
        OnlineRetryInterval = 120
        )

        IP webip (
                Device = eth0
                Address = "9.11.111.66"
                NetMask = "255.255.254.0"
                )

        NIC csgnic (
                Device = eth0
                )

        webip requires csgnic


        // resource dependency tree
        //
        //      group ClusterService
        //      {
        //      IP webip
        //          {
        //          NIC csgnic
        //          }
        //      }


group cfsnfssg (
        SystemList = { arcx3455frpv8 = 0, arcx3455gbxv5 = 1 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { arcx3455frpv8, arcx3455gbxv5 }
        )

        ApplicationNone app (
                MonitorProgram = "/opt/VRTSvcs/bin/ApplicationNone/lockdstatdmon"
                )

        CFSMount cfsmount1 (
                Critical = 0
                MountPoint = "/mnt/q01"
                BlockDevice = "/dev/vx/dsk/qlogic/q01"
                MountOpt @arcx3455frpv8 = rw
                MountOpt @arcx3455gbxv5 = rw
                NodeList = { arcx3455frpv8, arcx3455gbxv5 }
                )

        CFSMount cfsnfs_locks (
                Critical = 0
                MountPoint = "/locks"
                BlockDevice = "/dev/vx/dsk/cfs/c01"
                NodeList = { arcx3455frpv8, arcx3455gbxv5 }
                )

        CVMVolDg cvmvoldg1 (
                Critical = 0
                CVMDiskGroup = cfs
                CVMVolume = { c01 }
                CVMActivation @arcx3455frpv8 = sw
                CVMActivation @arcx3455gbxv5 = sw
                )

        CVMVolDg cvmvoldg2 (
                Critical = 0
                CVMDiskGroup = qlogic
                CVMVolume = { q01 }
                CVMActivation @arcx3455frpv8 = sw
                CVMActivation @arcx3455gbxv5 = sw
                )

        NFS nfs (
                )

        Share share1 (
                PathName = "/mnt/q01"
                Options = "rw,no_root_squash"
                NFSRes = nfs
                )

        requires group cvm online local firm
        cfsmount1 requires cvmvoldg2
        cfsnfs_locks requires cvmvoldg1
        share1 requires cfsmount1
        share1 requires nfs


        // resource dependency tree
        //
        //      group cfsnfssg
        //      {
        //      ApplicationNone app
        //      CFSMount cfsnfs_locks
        //          {
        //          CVMVolDg cvmvoldg1
        //          }
        //      Share share1
        //          {
        //          NFS nfs
        //          CFSMount cfsmount1
        //              {
        //              CVMVolDg cvmvoldg2
        //              }
        //          }
        //      }


group cfsnfssg_dummy (
        SystemList = { arcx3455frpv8 = 0, arcx3455gbxv5 = 1 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { arcx3455frpv8, arcx3455gbxv5 }
        )

        requires group cvm online local firm


        // resource dependency tree
        //
        //      group cfsnfssg_dummy
        //      {
        //      }


group cvm (
        SystemList = { arcx3455frpv8 = 0, arcx3455gbxv5 = 1 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { arcx3455frpv8, arcx3455gbxv5 }
        )

        CFSfsckd vxfsckd (
                ActivationMode @arcx3455frpv8 = { cfs = sw, qlogic = sw }
                ActivationMode @arcx3455gbxv5 = { cfs = sw, qlogic = sw }
                )

        CVMCluster cvm_clus (
                CVMClustName = dqq
                CVMNodeId = { arcx3455frpv8 = 1, arcx3455gbxv5 = 0 }
                CVMTransport = gab
                CVMTimeout = 200
                )

        CVMVxconfigd cvm_vxconfigd (
                Critical = 0
                CVMVxconfigdArgs = { syslog }
                )

        cvm_clus requires cvm_vxconfigd
        vxfsckd requires cvm_clus


        // resource dependency tree
        //
        //      group cvm
        //      {
        //      CFSfsckd vxfsckd
        //          {
        //          CVMCluster cvm_clus
        //              {
        //              CVMVxconfigd cvm_vxconfigd
        //              }
        //          }
        //      }









   
      

0 Kudos
5 REPLIES 5

yoyohimhim
Level 3

‎01-27-2010 02:31 AM

engine_A.log when add and mount the NFS shared CFS file system to the VCS configuration
arcx3455gbxv5:/var/VRTSvcs/log # tail -f engine_A.log
 


2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: haconf -makerw from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -add cvmvoldg2  CVMVolDg  cfsnfssg  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -modify cvmvoldg2  CVMDiskGroup  qlogic  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -modify ... -add cvmvoldg2  CVMVolume  q01  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -local vxfsckd  ActivationMode  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -local cvmvoldg2  CVMActivation  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -modify cvmvoldg2  CVMActivation  sw  arcx3455frpv8  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -modify cvmvoldg2  CVMActivation  sw  arcx3455gbxv5  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -modify cvmvoldg2  Critical  0  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: hares -modify cvmvoldg2  Enabled  1  from localhost
2010/01/27 03:14:26 VCS INFO V-16-1-50135 User root fired command: haconf -dump -makero from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: haconf -makerw from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -add cfsmount1  CFSMount  cfsnfssg  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  MountPoint  /mnt/q01  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  BlockDevice  /dev/vx/dsk/qlogic/q01  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  MountOpt  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  RemountRes  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  ForceOff  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  SetPrimary  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  MountOpt  rw  arcx3455frpv8  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  NodeList  arcx3455frpv8  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  Critical  0  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -link cfsmount1  cvmvoldg2  0  0  from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  Enabled  1  from localhost
2010/01/27 03:14:27 VCS WARNING V-16-10031-1087 (arcx3455frpv8) CVMVolDg:cvmvoldg2:attr_changed:Resource cvmvoldg2 is not ONLINE on this node
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: haconf -dump -makero from localhost
2010/01/27 03:14:27 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  RemountRes  from localhost
2010/01/27 03:14:28 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  RemountRes  DONE  arcx3455gbxv5  from localhost
2010/01/27 03:14:28 VCS WARNING V-16-10031-1087 (arcx3455gbxv5) CVMVolDg:cvmvoldg2:attr_changed:Resource cvmvoldg2 is not ONLINE on this node
2010/01/27 03:14:28 VCS INFO V-16-1-50135 User root fired command: haconf -makerw from localhost
2010/01/27 03:14:28 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  MountOpt  from localhost
2010/01/27 03:14:28 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  MountOpt  rw  arcx3455gbxv5  from localhost
2010/01/27 03:14:28 VCS INFO V-16-1-50135 User root fired command: hares -modify ... -add cfsmount1  NodeList  arcx3455gbxv5  from localhost
2010/01/27 03:14:28 VCS INFO V-16-1-50135 User root fired command: haconf -dump -makero from localhost
2010/01/27 03:14:29 VCS INFO V-16-1-50135 User root fired command: MSG_RES_PROBE cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:14:29 VCS INFO V-16-1-50135 User root fired command: MSG_RES_PROBE cfsmount1  arcx3455gbxv5  from localhost
2010/01/27 03:14:29 VCS WARNING V-16-10031-1087 (arcx3455gbxv5) CVMVolDg:cvmvoldg2:attr_changed:Resource cvmvoldg2 is not ONLINE on this node
2010/01/27 03:14:29 VCS WARNING V-16-10031-1087 (arcx3455frpv8) CVMVolDg:cvmvoldg2:attr_changed:Resource cvmvoldg2 is not ONLINE on this node
2010/01/27 03:14:31 VCS INFO V-16-1-10304 Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (First probe)
2010/01/27 03:14:31 VCS INFO V-16-1-10304 Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (First probe)
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -clear cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -online cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -clear cfsmount1  arcx3455gbxv5  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -online cfsmount1  arcx3455gbxv5  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -online cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2  arcx3455frpv8  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -local cfsmount1  RemountRes  from localhost
2010/01/27 03:14:33 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  RemountRes  DONE  arcx3455gbxv5  from localhost
2010/01/27 03:14:35 VCS INFO V-16-1-50135 User root fired command: hares -online cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:14:35 VCS INFO V-16-1-10304 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (First probe)
2010/01/27 03:14:35 VCS INFO V-16-1-10306 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (Previous State = OFFLINE)
2010/01/27 03:14:35 VCS INFO V-16-1-10304 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (First probe)
2010/01/27 03:14:35 VCS INFO V-16-1-10306 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (Previous State = OFFLINE)
2010/01/27 03:14:37 VCS INFO V-16-1-50135 User root fired command: hares -online cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:14:37 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455frpv8
2010/01/27 03:14:37 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455gbxv5
2010/01/27 03:14:37 VCS NOTICE V-16-1-10301 Initiating Online of Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:14:39 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2  arcx3455frpv8  from localhost
2010/01/27 03:14:40 VCS INFO V-16-10031-1046 (arcx3455frpv8) CVMVolDg:cvmvoldg2:online:resource cvmvoldg2 is online
2010/01/27 03:15:11 VCS INFO V-16-1-10298 Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) is online on arcx3455frpv8 (VCS initiated)
2010/01/27 03:15:11 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:15:12 VCS INFO V-16-20011-5506 (arcx3455frpv8) CFSMount:cfsmount1:online:CFSMOUNT Onlined :MountOptions : -t vxfs -o cluster,rw,mntlock=VCS               Block Device /dev/vx/dsk/qlogic/q01 : MountPoint /mnt/q01
2010/01/27 03:15:13 VCS INFO V-16-1-10298 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is online on arcx3455frpv8 (VCS initiated)
2010/01/27 03:15:16 VCS INFO V-16-1-50135 User root fired command: hares -online cfsmount1  arcx3455gbxv5  from localhost
2010/01/27 03:15:16 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455frpv8
2010/01/27 03:15:16 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455gbxv5
2010/01/27 03:15:16 VCS NOTICE V-16-1-10301 Initiating Online of Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:15:19 VCS INFO V-16-10031-1046 (arcx3455gbxv5) CVMVolDg:cvmvoldg2:online:resource cvmvoldg2 is online
2010/01/27 03:15:21 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2  arcx3455gbxv5  from localhost
2010/01/27 03:15:51 VCS INFO V-16-1-10298 Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) is online on arcx3455gbxv5 (VCS initiated)
2010/01/27 03:15:51 VCS NOTICE V-16-1-10301 Initiating Online of Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:15:51 VCS INFO V-16-20011-5506 (arcx3455gbxv5) CFSMount:cfsmount1:online:CFSMOUNT Onlined :MountOptions : -t vxfs -o cluster,rw,mntlock=VCS               Block Device /dev/vx/dsk/qlogic/q01 : MountPoint /mnt/q01
2010/01/27 03:15:52 VCS INFO V-16-1-10298 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is online on arcx3455gbxv5 (VCS initiated)
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: haconf -makerw from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -add share1  Share  cfsnfssg  from localhost
2010/01/27 03:15:53 VCS NOTICE V-16-1-10016 Agent /opt/VRTSvcs/bin/Share/ShareAgent for resource type Share successfully started at Wed Jan 27 03:15:53 2010
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -modify share1  PathName  /mnt/q01  from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -modify share1  Options  rw,no_root_squash  from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -modify share1  NFSRes  nfs  from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -modify share1  Enabled  1  from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -link share1  nfs  0  0  from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-10304 Resource share1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (First probe)
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: hares -link share1  cfsmount1  0  0  from localhost
2010/01/27 03:15:53 VCS INFO V-16-1-10304 Resource share1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (First probe)
2010/01/27 03:15:53 VCS INFO V-16-1-50135 User root fired command: haconf -dump -makero from localhost
2010/01/27 03:15:54 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455frpv8  from localhost
2010/01/27 03:15:54 VCS INFO V-16-1-50135 User root fired command: hares -clear cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:15:54 VCS INFO V-16-1-50135 User root fired command: hares -clear nfs  arcx3455frpv8  from localhost
2010/01/27 03:15:54 VCS INFO V-16-1-10307 Resource nfs (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (Not initiated by VCS)
2010/01/27 03:15:54 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2  arcx3455frpv8  from localhost
2010/01/27 03:15:56 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455frpv8  from localhost
2010/01/27 03:15:56 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455frpv8
2010/01/27 03:15:56 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455gbxv5
2010/01/27 03:15:56 VCS NOTICE V-16-1-10301 Initiating Online of Resource nfs (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:15:56 VCS WARNING V-16-10031-7002 (arcx3455frpv8) NFS:nfs:online:Sending SIGKILL signal to nfsd (13419 13420 13421 13422 13423 13424 13425 13426).
2010/01/27 03:15:57 VCS INFO V-16-2-13716 (arcx3455frpv8) Resource(nfs): Output of the completed operation (online)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:15:58 VCS INFO V-16-1-50135 User root fired command: hares -clear cfsmount1  arcx3455frpv8  from localhost
2010/01/27 03:15:58 VCS INFO V-16-1-50135 User root fired command: hares -clear nfs  arcx3455frpv8  from localhost
2010/01/27 03:18:00 VCS ERROR V-16-2-13066 (arcx3455frpv8) Agent is calling clean for resource(nfs) because the resource is not up even after online completed.
2010/01/27 03:18:00 VCS WARNING V-16-10031-7002 (arcx3455frpv8) NFS:nfs:clean:Sending SIGKILL signal to nfsd (14906 14907 14908 14909 14910 14911 14912 14913).
2010/01/27 03:18:01 VCS INFO V-16-2-13716 (arcx3455frpv8) Resource(nfs): Output of the completed operation (clean)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:18:01 VCS INFO V-16-2-13068 (arcx3455frpv8) Resource(nfs) - clean completed successfully.
2010/01/27 03:18:01 VCS INFO V-16-2-13071 (arcx3455frpv8) Resource(nfs): reached OnlineRetryLimit(0).
2010/01/27 03:18:01 VCS ERROR V-16-1-10303 Resource nfs (Owner: unknown, Group: cfsnfssg) is FAULTED (timed out) on sys arcx3455frpv8
2010/01/27 03:18:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:18:02 VCS INFO V-16-1-10305 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (VCS initiated)
2010/01/27 03:18:02 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) on System arcx3455frpv8
2010/01/27 03:18:04 VCS INFO V-16-1-10305 Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455frpv8 (VCS initiated)
2010/01/27 03:18:04 VCS ERROR V-16-1-10205 Group cfsnfssg is faulted on system arcx3455frpv8
2010/01/27 03:18:04 VCS NOTICE V-16-1-10446 Group cfsnfssg is offline on system arcx3455frpv8
2010/01/27 03:18:04 VCS INFO V-16-6-0 (arcx3455frpv8) postoffline:Invoked with arg0=arcx3455frpv8, arg1=cfsnfssg
2010/01/27 03:18:04 VCS INFO V-16-6-15002 (arcx3455frpv8) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline arcx3455frpv8 cfsnfssg   successfully
2010/01/27 03:18:05 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455frpv8  from localhost
2010/01/27 03:18:07 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455frpv8  from localhost
.....
2010/01/27 03:18:52 VCS INFO V-16-1-50135 User root fired command: hares -modify cfsmount1  Primary  arcx3455gbxv5  from localhost
.....
2010/01/27 03:18:52 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455frpv8  from localhost
2010/01/27 03:19:40 VCS INFO V-16-1-50135 User root fired command: hares -clear cfsmount1  arcx3455gbxv5  from localhost
2010/01/27 03:19:40 VCS INFO V-16-1-50135 User root fired command: hares -clear nfs  arcx3455gbxv5  from localhost
2010/01/27 03:19:40 VCS INFO V-16-1-10307 Resource nfs (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (Not initiated by VCS)
2010/01/27 03:19:40 VCS INFO V-16-1-50135 User root fired command: hares -clear cvmvoldg2  arcx3455gbxv5  from localhost
2010/01/27 03:19:42 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455gbxv5  from localhost
2010/01/27 03:19:42 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455frpv8
2010/01/27 03:19:42 VCS NOTICE V-16-1-10232 Clearing Restart attribute for group cfsnfssg on node arcx3455gbxv5
2010/01/27 03:19:42 VCS NOTICE V-16-1-10301 Initiating Online of Resource nfs (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:19:43 VCS WARNING V-16-10031-7002 (arcx3455gbxv5) NFS:nfs:online:Sending SIGKILL signal to nfsd (15669 15670 15671 15672 15673 15674 15675 15676).
2010/01/27 03:19:43 VCS INFO V-16-2-13716 (arcx3455gbxv5) Resource(nfs): Output of the completed operation (online)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:19:44 VCS INFO V-16-1-50135 User root fired command: hares -clear cfsmount1  arcx3455gbxv5  from localhost
2010/01/27 03:19:44 VCS INFO V-16-1-50135 User root fired command: hares -clear nfs  arcx3455gbxv5  from localhost
2010/01/27 03:21:45 VCS ERROR V-16-2-13066 (arcx3455gbxv5) Agent is calling clean for resource(nfs) because the resource is not up even after online completed.
2010/01/27 03:21:45 VCS WARNING V-16-10031-7002 (arcx3455gbxv5) NFS:nfs:clean:Sending SIGKILL signal to nfsd (18915 18916 18917 18918 18919 18920 18921 18922).
2010/01/27 03:21:46 VCS INFO V-16-2-13716 (arcx3455gbxv5) Resource(nfs): Output of the completed operation (clean)
==============================================
mount: nfsd already mounted or /proc/fs/nfsd busy
mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd
fs.nfs.nlm_grace_period = 90
==============================================

2010/01/27 03:21:46 VCS INFO V-16-2-13068 (arcx3455gbxv5) Resource(nfs) - clean completed successfully.
2010/01/27 03:21:46 VCS INFO V-16-2-13071 (arcx3455gbxv5) Resource(nfs): reached OnlineRetryLimit(0).
2010/01/27 03:21:46 VCS ERROR V-16-1-10303 Resource nfs (Owner: unknown, Group: cfsnfssg) is FAULTED (timed out) on sys arcx3455gbxv5
2010/01/27 03:21:46 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:21:47 VCS INFO V-16-1-10305 Resource cfsmount1 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (VCS initiated)
2010/01/27 03:21:47 VCS NOTICE V-16-1-10300 Initiating Offline of Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) on System arcx3455gbxv5
2010/01/27 03:21:50 VCS INFO V-16-1-10305 Resource cvmvoldg2 (Owner: unknown, Group: cfsnfssg) is offline on arcx3455gbxv5 (VCS initiated)
2010/01/27 03:21:50 VCS ERROR V-16-1-10205 Group cfsnfssg is faulted on system arcx3455gbxv5
2010/01/27 03:21:50 VCS NOTICE V-16-1-10446 Group cfsnfssg is offline on system arcx3455gbxv5
2010/01/27 03:21:50 VCS INFO V-16-6-0 (arcx3455gbxv5) postoffline:Invoked with arg0=arcx3455gbxv5, arg1=cfsnfssg
2010/01/27 03:21:50 VCS INFO V-16-6-15002 (arcx3455gbxv5) hatrigger:hatrigger executed /opt/VRTSvcs/bin/triggers/postoffline arcx3455gbxv5 cfsnfssg   successfully
2010/01/27 03:21:52 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455gbxv5  from localhost
2010/01/27 03:21:54 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455gbxv5  from localhost
.....
2010/01/27 03:23:27 VCS INFO V-16-1-50135 User root fired command: hares -online share1  arcx3455gbxv5  from localhost

 



 

0 Kudos

Gaurav_S
Moderator
Moderator
   VIP    Certified

‎01-27-2010 11:25 PM

Hello,

Couple of quick suggestions:

a) Can you take out nfsd from OS control & allow VCS to start/stop nfsd ..
b) Not seen you have configured NFSRestart resource, configure one as defined in VCS Bundled Agents Guide..

Let me know if this helps..


Gaurav
0 Kudos

yoyohimhim
Level 3

‎01-28-2010 02:23 AM

Hi Gaurav,
   In SUSE, seems nfsd service is named nfsserver. I've checked its status after reboot, it is stopped.After configure CNFS component by running "cfsshare config cfs c01 /locks' it will be started.
  As for the second suggestion, since I'm not use the traditional VCS, I'm use the sfcfsha, the disks need to be used on two nodes synchronously, so the NFS Agent can't be used for the scenario.
  Do you have any other suggestion dear Gaurav? My colleagues used the same steps on Redhat 5,seems no problem.
   
  
0 Kudos

Gaurav_S
Moderator
Moderator
   VIP    Certified

‎01-28-2010 11:38 PM

Are you sure that nfsserver starts after running cfsshare command ?

Because what I think is, cfsshare command is nothing but executing a command to share a filesystem but on a CFS. command you mentioned above would configure /locks filesystem under CFS to share.... so really not sure if this will start your nfsserver process...

You have configured your NFS resource in cfsnfssg group which is a parallel group so obviously NFS resource should take care of NFS daemons on both the nodes... which is correct... also dependency of share resource is correct... however one thing which I see can be corrected is, your cfsnfssg group doesn't contain a NIC or proxy resource.....

so which NIC this traffic is flowing through ?

I see you have configured eth0 as csgnic in ClusterService group..... If this is the same NIC from which NFS traffic has to flow, then I suggest you to create a proxy resource in cfsnfssg service group & point it to csgnic in cluster service group... later, your NFS resource should depend on this proxy resource....

Regarding the dependency, check bundled agents guide, here you would also get pointers to use NFSRestart agent.....

One thing what I am confused of is, if we see bundled agents guide for NFS agent, everywhere it mentionds nfsd process, so I am wondering whether nfsserver or nfsd is the correct process.... plz chk once again if thers is any nfsd binary lying on server ?

let me know how this goes....

Gaurav
0 Kudos

yoyohimhim
Level 3

‎02-03-2010 11:34 PM

Hi Gaurav,
  I've found the reason. I deleted 127.0.0.1 localhost in /etc/hosts.
  Thank you for your help. ^_^
0 Kudos