Forum Discussion

UnixFan's avatar
UnixFan
Level 3
15 years ago

df: cannot statvfs Permission denied on cluster file system


Hi ,

Does any one got any idea why I am getting the permission denied error when I do the   df -k  on a 3 node cluster with a shared Cluster file systems (CVM)

I can do the df -k on  the master server as a normal user  ,however ,I can get the df -k on other two nodes ,

[abcadmin@sol100]: /> sudo vxdctl -c mode
mode: enabled: cluster active - MASTER
master: sol100

[abcadmin@sol100]: /> df -kh
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          15G   8.4G   6.3G    58%    /
/devices                 0K     0K     0K     0%    /devices
/dev/odm                 0K     0K     0K     0%    /dev/odm
/dev/vx/dsk/shareddg/sharedvol
                        93G    14G    74G    16%    /opt/fwire/Shared
                      ---- this is the shared cluster file system   works fine on master server with the df -k as normal user


[abcadmin@sol300]: ~> sudo vxdctl -c mode
mode: enabled: cluster active - SLAVE
master: sol100

[abcadmin@sol300]: ~> df -kh
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          14G   4.1G    10G    29%    /
/dev/odm                 0K     0K     0K     0%    /dev/odm
df: cannot statvfs /opt/fwire/Shared: Permission denied                 --- shared file systems get permission denied when I try with normal user ,I can get this if I do a sudo or root.

all three servers got same permission  for the mount points ...

Really appreciate if any one can show som lights on this , I just wanted this for a monitoring script to run as a normal user than root ..

Thanks in advance ..






  • That seems to have no relation to CFS or CVM ..... From google it looks to be purely a solaris issue...

    Did you had a run through any of these:

    http://www.sunmanagers.org/archives/1997/0147.html

    http://software.allfaq.org/forums/t/134896.aspx

    The user is contained in a group, does that group have a atleast read access to the filesystem ?

    Gaurav





  • Gaurav,

    the permissions are exactly same on all three nodes ,

    [abcdmin@sol100]: ~> ls -ld /opt/fwire/Shared
    drwxrwxr-x  26 delfw01  fwire       1024 Aug 13 15:25 /opt/fwire/Shared

    [abcadmin@sol300]: ~> sudo ls -ld /opt/fwire/Shared
    drwxrwxr-x  26 delfw01  fwire       1024 Aug 13 15:25 /opt/fwire/Shared

    it has 775 permission on those folders , which should allow all other users to just read though its not part of that group correct ..?

    I just forgot to mention the volume permission on this which is set to 770   "perms:    user=delfw01 group=fwire mode=0770"

    but I dont understand why I can read that from the master server  ..? and not on other nodes.

    Thanks
  • hmm makes sense that it should allow the read access..... by any chance have you setup solaris auditing ?

    Gaurav
  • Yep  ,we got auditing enabled

    online         Apr_23   svc:/system/auditd:default



  • looking for it, in the mean time have a look at this:

    http://support.bb4.com/archive/200608/msg00024.html


    Not able to remember exactly, there was something around auditing which use to cause this.... trying to find right doc for this...



    Gaurav

  • Can you also check the activation mode for the cvm dg?
    Could you please provide us with the following command output from master node?
    1.> vxdctl -c mode
    2.> vxdg list <cvm dg>
    3.> cfscluster status
    4.>/etc/vx/bin/vxclustadm nidmap  --> on all nodes
    5.>/etc/vx/bin/vxclustadm -v nodestate  --> on all nodes
    6.> hastatus -sum
    7.> cfscluster config
    8.> modinfo | grep -i vx
    9.> ps -ef | grep -i vxconf
    10.> uname -a

    Has this setup worked fine in the past?

  • Are you able to write to the cfs mount point?
    Have you tried "sudo df -k" ?

  • Roy,

    If you look at my initial posting I have pasted those info already .. I dont think we need any cluster config details here , as I mentioned earlier I can do it by using the sudo . I want to do it as normal user , which I can do it on master server ,not on other nodes in the cluster

    sol100# /etc/vx/bin/vxclustadm nidmap
    Name                             CVM Nid    CM Nid     State
    sol100                        1          0          Joined: Master
    sol200                        0          1          Joined: Slave
    sol300                        2          2          Joined: Slave
    sol100# /etc/vx/bin/vxclustadm -v nodestate
    state: cluster member
            nodeId=0
            masterId=0
            neighborId=2
            members=0x7
            joiners=0x0
            leavers=0x0
            reconfig_seqnum=0x7ba82c
            vxfen=on

    ---
    vscecd300# /etc/vx/bin/vxclustadm nidmap
    Name                             CVM Nid    CM Nid     State
    sol100                        1          0          Joined: Master
    sol200                        0          1          Joined: Slave
    sol300                        2          2          Joined: Slave
    sol300# /etc/vx/bin/vxclustadm -v nodestate
    state: cluster member
            nodeId=2
            masterId=0
            neighborId=1
            members=0x7
            joiners=0x0
            leavers=0x0
            reconfig_seqnum=0x7ba82c
            vxfen=on


    sol300# uname -a
    SunOS sol300 5.10 Generic_142900-03 sun4v sparc SUNW,SPARC-Enterprise-T5220


    sol300# ps -ef | grep -i vxconf
        root    71     1   0   Apr 19 ?           0:37 vxconfigd -x syslog -m boot
        root 24009 23543   0 09:16:03 pts/7       0:00 grep -i vxconf
        root  3150     1   0   Apr 19 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd
        root  4166     1   0   Apr 19 ?         169:44 /opt/VRTSvcs/bin/CVMVxconfigd/CVMVxconfigdAgent -type CVMVxconfigd
        root  3807  3150   0   Apr 19 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd


    sol100#  ps -ef | grep -i vxconf
        root    71     1   0   Mar 23 ?           1:04 vxconfigd -x syslog -m boot
        root  4372     1   0   Mar 23 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd
        root  5036  4372   0   Mar 23 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd
        root  5483     1   0   Mar 23 ?         214:39 /opt/VRTSvcs/bin/CVMVxconfigd/CVMVxconfigdAgent -type CVMVxconfigd
        root 10225  9503   0 09:16:22 pts/12      0:00 grep -i vxconf
    sol100# uname -a
    SunOS sol100 5.10 Generic_141444-09 sun4v sparc SUNW,SPARC-Enterprise-T5220



    sol100# cfscluster status

      Node             : sol100
      Cluster Manager  :  running
      CVM state        :  running
      MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
      /opt/fwire/Shared sharedvol      shareddg          MOUNTED


      Node             :  sol200
      Cluster Manager  :  running
      CVM state        :  running
      MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
      /opt/fwire/Shared sharedvol      shareddg          MOUNTED


      Node             :  sol300
      Cluster Manager  :  running
      CVM state        :  running
      MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
      /opt/fwire/Shared sharedvol      shareddg          MOUNTED


    Thanks


  • I would advise you to check with Sun Solaris Forums, as this appears more over a OS / user / shell / access / permission related issue.

    regards
    Rajesh Regunta