cancel
Showing results for 
Search instead for 
Did you mean: 

NFS agent problems with starting nfsd

h2o666
Level 2

(Please excuse the kludged service group names ) I think I'm doing something wrong with NFSRestart Trying to run two NFS shares, one on each node. NFS agent fails to start nfsd or is very delayed

# showmount -e rpc mount export: RPC: Unable to receive; errno = Connection refused 2016/01/16 14:46:17 VCS INFO V-16-1-10298 Resource dCVS_NFSRestart

(Owner: unknown, Group: dCVS_Group) is online on cm-cvs-22 (VCS initiated) 2016/01/16 14:49:02 VCS ERROR V-16-2-13067 (cm-cvs-21) Agent is calling clean for resource(dCVS_NFS) because the resource became OFFLINE unexpectedly, on its own. 2016/01/16 14:49:03 VCS INFO V-16-2-13716 (cm-cvs-21) Resource(dCVS_NFS): Output of the completed operation (clean) ============================================== mount: nfsd already mounted or /proc/fs/nfsd busy mount: according to mtab, nfsd is already mounted on /proc/fs/nfsd fs.nfs.nlm_grace_period = 90 ============================================== 2016/01/16 14:49:03 VCS INFO V-16-2-13068 (cm-cvs-21) Resource(dCVS_NFS) - clean completed successfully. 2016/01/16 14:49:03 VCS ERROR V-16-2-13073 (cm-cvs-21) Resource(dCVS_NFS) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 1) the resource. 2016/01/16 14:49:04 VCS WARNING V-16-10031-7002 (cm-cvs-21) NFS:dCVS_NFS:online:Sending SIGKILL signal to nfsd (9674 9675 9676 9677 9678 9679 9680 9681 9682 9683 9684 9685 9686 9687 9688 9689). 2016/01/16 14:49:05 VCS INFO V-16-2-13716 (cm-cvs-21) Resource(dCVS_NFS): Output of the completed operation (online) #############main.cf################ include "types.cf" cluster cvsnew ( UserNames = { admin = hello } Administrators = { admin } ) system cm-cvs-21 ( ) system cm-cvs-22 ( ) group ClusterService ( SystemList = { cm-cvs-21 = 0, cm-cvs-22 = 1 } AutoStartList = { cm-cvs-21, cm-cvs-22 } OnlineRetryLimit = 3 OnlineRetryInterval = 120 ) IP webip ( Device = eth0 Address = "10.x.243.35" NetMask = "255.255.252.0" ) NIC csgnic ( Device = eth0 ) NotifierMngr ntfr ( SmtpServer = "mailhost.blah.blah.blah.com" SmtpRecipients = { root = Warning } ) ntfr requires csgnic webip requires csgnic // resource dependency tree // // group ClusterService // { // NotifierMngr ntfr // { // NIC csgnic // } // IP webip // { // NIC csgnic // } // } group dCVS_Group ( SystemList = { cm-cvs-22 = 0, cm-cvs-21 = 1 } AutoStartList = { cm-cvs-22, cm-cvs-21 } PreOnline @cm-cvs-22 = 1 PreOnline @cm-cvs-21 = 1 ) Apache dCVS_Apache ( Enabled = 0 httpdDir = "/usr/sbin" HostName = cvs ConfigFile = "/etc/httpd/conf/httpd.conf" ) DiskGroup dCVS_cvsdg ( DiskGroup = cvsdg01- ) IP dCVS_IP ( Device = eth0 Address = "10.x.243.79" NetMask = "255.255.252.0" ) Mount dCVS_Mount01 ( MountPoint = "/u/cvsdg01" BlockDevice = "/dev/vx/dsk/cvsdg01-/vol01" FSType = vxfs MountOpt = "ioerror=disable" FsckOpt = "-y" ) NFSRestart dCVS_NFSRestart ( NFSRes = dCVS_NFS ) NIC dCVS_NIC ( Device = eth0 ) Proxy sProxy_NFS ( TargetResName = dCVS_NFS ) Share dCVS_Share01 ( PathName = "/u/cvsdg01" Client = "*" Options = "rw,sync,insecure,no_root_squash" NFSRes = dCVS_NFS ) Volume dCVS_vol01 ( DiskGroup = cvsdg01- Volume = vol01 ) dCVS_IP requires dCVS_NIC dCVS_IP requires dCVS_Share01 dCVS_Mount01 requires dCVS_vol01 dCVS_NFSRestart requires dCVS_IP dCVS_NFSRestart requires dCVS_Mount01 dCVS_Share01 requires dCVS_Mount01 dCVS_Share01 requires sProxy_NFS dCVS_vol01 requires dCVS_cvsdg // resource dependency tree // // group dCVS_Group // { // Apache dCVS_Apache // NFSRestart dCVS_NFSRestart // { // Mount dCVS_Mount01 // { // Volume dCVS_vol01 // { // DiskGroup dCVS_cvsdg // } // } // IP dCVS_IP // { // Share dCVS_Share01 // { // Proxy sProxy_NFS // Mount dCVS_Mount01 // { // Volume dCVS_vol01 // { // DiskGroup dCVS_cvsdg // } // } // } // NIC dCVS_NIC // } // } // } group dCVS_Group-02 ( SystemList = { cm-cvs-22 = 0, cm-cvs-21 = 1 } AutoStartList = { cm-cvs-22, cm-cvs-21 } PreOnline @cm-cvs-22 = 1 PreOnline @cm-cvs-21 = 1 ) DiskGroup dCVS_cvsdg02 ( DiskGroup = cvsdg02- ) IP dCVS_IP02 ( Device = eth0 Address = "10.x.243.221" NetMask = "255.255.252.0" ) Mount dCVS_Mount0102 ( MountPoint = "/u/cvsdg02" BlockDevice = "/dev/vx/dsk/cvsdg02-/vol01" FSType = vxfs MountOpt = "ioerror=disable" FsckOpt = "-y" ) NFSRestart dCVS_NFSRestart02 ( NFSRes = dCVS_NFS ) NIC dCVS_NIC02 ( Device = eth0 ) Proxy sProxy_NFS02 ( TargetResName = dCVS_NFS ) Share dCVS_Share0102 ( PathName = "/u/cvsdg02" Client = "*" Options = "rw,sync,insecure,no_root_squash" NFSRes = dCVS_NFS ) Volume dCVS_vol0102 ( DiskGroup = cvsdg02- Volume = vol01 ) dCVS_IP02 requires dCVS_NIC02 dCVS_IP02 requires dCVS_Share0102 dCVS_Mount0102 requires dCVS_vol0102 dCVS_NFSRestart02 requires dCVS_IP02 dCVS_NFSRestart02 requires dCVS_Mount0102 dCVS_Share0102 requires dCVS_Mount0102 dCVS_Share0102 requires sProxy_NFS02 dCVS_vol0102 requires dCVS_cvsdg02 // resource dependency tree // // group dCVS_Group-02 // { // NFSRestart dCVS_NFSRestart02 // { // IP dCVS_IP02 // { // NIC dCVS_NIC02 // Share dCVS_Share0102 // { // Proxy sProxy_NFS02 // Mount dCVS_Mount0102 // { // Volume dCVS_vol0102 // { // DiskGroup dCVS_cvsdg02 // } // } // } // } // Mount dCVS_Mount0102 // { // Volume dCVS_vol0102 // { // DiskGroup dCVS_cvsdg02 // } // } // } // } group dNFS_Group ( SystemList = { cm-cvs-22 = 0, cm-cvs-21 = 1 } Parallel = 1 AutoStartList = { cm-cvs-22, cm-cvs-21 } ) NFS dCVS_NFS ( Nproc = 16 ) Phantom NFS_phantom ( Critical = 0 ) // resource dependency tree // // group dNFS_Group // { // Phantom NFS_phantom // NFS dCVS_NFS // }

3 REPLIES 3

h2o666
Level 2

Eventually nfsd will start so the remaining problem is that when both groups are on  node1 and one group is switched to the node2, the agent stops nfsd on node1, and then it has the same long delay as when the service first starts.

h2o666
Level 2

 

Let's see if this formats better

include "types.cf"

 

cluster cvsnew (

    UserNames = { admin = ghiAhcHeiDiiGqiChf }

    Administrators = { admin }

    )

 

system cm-cvs-21 (

    )

 

system cm-cvs-22 (

    )

 

group ClusterService (

    SystemList = { cm-cvs-21 = 0, cm-cvs-22 = 1 }

    AutoStartList = { cm-cvs-21, cm-cvs-22 }

    OnlineRetryLimit = 3

    OnlineRetryInterval = 120

    )

 

    IP webip (

        Device = eth0

        Address = "10.x.243.35"

        NetMask = "255.255.252.0"

        )

 

    NIC csgnic (

        Device = eth0

        )

 

    NotifierMngr ntfr (

        SmtpServer = "mailhost.blah.blah.blah.com"

        SmtpRecipients = { root = Warning }

        )

 

    ntfr requires csgnic

    webip requires csgnic

 

 

    // resource dependency tree

    //

    //    group ClusterService

    //    {

    //    NotifierMngr ntfr

    //        {

    //        NIC csgnic

    //        }

    //    IP webip

    //        {

    //        NIC csgnic

    //        }

    //    }

 

 

group dCVS_Group (

    SystemList = { cm-cvs-22 = 0, cm-cvs-21 = 1 }

    AutoStartList = { cm-cvs-22, cm-cvs-21 }

    PreOnline @cm-cvs-22 = 1

    PreOnline @cm-cvs-21 = 1

    )

 

    Apache dCVS_Apache (

        Enabled = 0

        httpdDir = "/usr/sbin"

        HostName = cvs

        ConfigFile = "/etc/httpd/conf/httpd.conf"

        )

 

    DiskGroup dCVS_cvsdg (

        DiskGroup = cvsdg01-

        )

 

    IP dCVS_IP (

        Device = eth0

        Address = "10.x.243.79"

        NetMask = "255.255.252.0"

        )

 

    Mount dCVS_Mount01 (

        MountPoint = "/u/cvsdg01"

        BlockDevice = "/dev/vx/dsk/cvsdg01-/vol01"

        FSType = vxfs

        MountOpt = "ioerror=disable"

        FsckOpt = "-y"

        )

 

    NFSRestart dCVS_NFSRestart (

        NFSRes = dCVS_NFS

        )

 

    NIC dCVS_NIC (

        Device = eth0

        )

 

    Proxy sProxy_NFS (

        TargetResName = dCVS_NFS

        )

 

    Share dCVS_Share01 (

        PathName = "/u/cvsdg01"

        Client = "*"

        Options = "rw,sync,insecure,no_root_squash"

        NFSRes = dCVS_NFS

        )

 

    Volume dCVS_vol01 (

        DiskGroup = cvsdg01-

        Volume = vol01

        )

 

    dCVS_IP requires dCVS_NIC

    dCVS_IP requires dCVS_Share01

    dCVS_Mount01 requires dCVS_vol01

    dCVS_NFSRestart requires dCVS_IP

    dCVS_NFSRestart requires dCVS_Mount01

    dCVS_Share01 requires dCVS_Mount01

    dCVS_Share01 requires sProxy_NFS

    dCVS_vol01 requires dCVS_cvsdg

 

 

    // resource dependency tree

    //

    //    group dCVS_Group

    //    {

    //    Apache dCVS_Apache

    //    NFSRestart dCVS_NFSRestart

    //        {

    //        Mount dCVS_Mount01

    //            {

    //            Volume dCVS_vol01

    //                {

    //                DiskGroup dCVS_cvsdg

    //                }

    //            }

    //        IP dCVS_IP

    //            {

    //            Share dCVS_Share01

    //                {

    //                Proxy sProxy_NFS

    //                Mount dCVS_Mount01

    //                    {

    //                    Volume dCVS_vol01

    //                        {

    //                        DiskGroup dCVS_cvsdg

    //                        }

    //                    }

    //                }

    //            NIC dCVS_NIC

    //            }

    //        }

    //    }

 

 

group dCVS_Group-02 (

    SystemList = { cm-cvs-22 = 0, cm-cvs-21 = 1 }

    AutoStartList = { cm-cvs-22, cm-cvs-21 }

    PreOnline @cm-cvs-22 = 1

    PreOnline @cm-cvs-21 = 1

    )

 

    DiskGroup dCVS_cvsdg02 (

        DiskGroup = cvsdg02-

        )

 

    IP dCVS_IP02 (

        Device = eth0

        Address = "10.x.243.221"

        NetMask = "255.255.252.0"

        )

 

    Mount dCVS_Mount0102 (

        MountPoint = "/u/cvsdg02"

        BlockDevice = "/dev/vx/dsk/cvsdg02-/vol01"

        FSType = vxfs

        MountOpt = "ioerror=disable"

        FsckOpt = "-y"

        )

 

    NFSRestart dCVS_NFSRestart02 (

        NFSRes = dCVS_NFS

        )

 

    NIC dCVS_NIC02 (

        Device = eth0

        )

 

    Proxy sProxy_NFS02 (

        TargetResName = dCVS_NFS

        )

 

    Share dCVS_Share0102 (

        PathName = "/u/cvsdg02"

        Client = "*"

        Options = "rw,sync,insecure,no_root_squash"

        NFSRes = dCVS_NFS

        )

 

    Volume dCVS_vol0102 (

        DiskGroup = cvsdg02-

        Volume = vol01

        )

 

    dCVS_IP02 requires dCVS_NIC02

    dCVS_IP02 requires dCVS_Share0102

    dCVS_Mount0102 requires dCVS_vol0102

    dCVS_NFSRestart02 requires dCVS_IP02

    dCVS_NFSRestart02 requires dCVS_Mount0102

    dCVS_Share0102 requires dCVS_Mount0102

    dCVS_Share0102 requires sProxy_NFS02

    dCVS_vol0102 requires dCVS_cvsdg02

 

 

    // resource dependency tree

    //

    //    group dCVS_Group-02

    //    {

    //    NFSRestart dCVS_NFSRestart02

    //        {

    //        IP dCVS_IP02

    //            {

    //            NIC dCVS_NIC02

    //            Share dCVS_Share0102

    //                {

    //                Proxy sProxy_NFS02

    //                Mount dCVS_Mount0102

    //                    {

    //                    Volume dCVS_vol0102

    //                        {

    //                        DiskGroup dCVS_cvsdg02

    //                        }

    //                    }

    //                }

    //            }

    //        Mount dCVS_Mount0102

    //            {

    //            Volume dCVS_vol0102

    //                {

    //                DiskGroup dCVS_cvsdg02

    //                }

    //            }

    //        }

    //    }

 

 

group dNFS_Group (

    SystemList = { cm-cvs-22 = 0, cm-cvs-21 = 1 }

    Parallel = 1

    AutoStartList = { cm-cvs-22, cm-cvs-21 }

    )

 

    NFS dCVS_NFS (

        Nproc = 16

        )

 

    Phantom NFS_phantom (

        Critical = 0

        )

 

 

 

    // resource dependency tree

    //

    //    group dNFS_Group

    //    {

    //    Phantom NFS_phantom

    //    NFS dCVS_NFS

    //    }

 

 

sudhir_h
Level 4
Employee

Hi.

 

Can you kindly provide which VCS and OS version are you using?

 

Also the VCS configuration may need modification based on the the VCS version. In the recent VCS releases we configure two NFSRestart resources in each service group. On one of the NFSRestart resource we set the Lower attribute to 1.

https://sort.symantec.com/public/documents/sfha/6.2/linux/productguides/html/vcs_bundled_agents/ch04s03s04.htm

 

Also you gan go through https://www.veritas.com/community/articles/how-make-nfs-shares-highly-available-using-atleast-resource-dependency post to give you more info.

 

When you switch the service group the nfsd daemon is stopped by NFSRestart agent and then re-started again.

 

Regards,

Sudhir