cancel
Showing results for 
Search instead for 
Did you mean: 

df: cannot statvfs Permission denied on cluster file system

UnixFan
Level 3

Hi ,

Does any one got any idea why I am getting the permission denied error when I do the   df -k  on a 3 node cluster with a shared Cluster file systems (CVM)

I can do the df -k on  the master server as a normal user  ,however ,I can get the df -k on other two nodes ,

[abcadmin@sol100]: /> sudo vxdctl -c mode
mode: enabled: cluster active - MASTER
master: sol100

[abcadmin@sol100]: /> df -kh
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          15G   8.4G   6.3G    58%    /
/devices                 0K     0K     0K     0%    /devices
/dev/odm                 0K     0K     0K     0%    /dev/odm
/dev/vx/dsk/shareddg/sharedvol
                        93G    14G    74G    16%    /opt/fwire/Shared
                      ---- this is the shared cluster file system   works fine on master server with the df -k as normal user


[abcadmin@sol300]: ~> sudo vxdctl -c mode
mode: enabled: cluster active - SLAVE
master: sol100

[abcadmin@sol300]: ~> df -kh
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          14G   4.1G    10G    29%    /
/dev/odm                 0K     0K     0K     0%    /dev/odm
df: cannot statvfs /opt/fwire/Shared: Permission denied                 --- shared file systems get permission denied when I try with normal user ,I can get this if I do a sudo or root.

all three servers got same permission  for the mount points ...

Really appreciate if any one can show som lights on this , I just wanted this for a monitoring script to run as a normal user than root ..

Thanks in advance ..





9 REPLIES 9

Gaurav_S
Moderator
Moderator
   VIP    Certified

That seems to have no relation to CFS or CVM ..... From google it looks to be purely a solaris issue...

Did you had a run through any of these:

http://www.sunmanagers.org/archives/1997/0147.html

http://software.allfaq.org/forums/t/134896.aspx

The user is contained in a group, does that group have a atleast read access to the filesystem ?

Gaurav




UnixFan
Level 3

Gaurav,

the permissions are exactly same on all three nodes ,

[abcdmin@sol100]: ~> ls -ld /opt/fwire/Shared
drwxrwxr-x  26 delfw01  fwire       1024 Aug 13 15:25 /opt/fwire/Shared

[abcadmin@sol300]: ~> sudo ls -ld /opt/fwire/Shared
drwxrwxr-x  26 delfw01  fwire       1024 Aug 13 15:25 /opt/fwire/Shared

it has 775 permission on those folders , which should allow all other users to just read though its not part of that group correct ..?

I just forgot to mention the volume permission on this which is set to 770   "perms:    user=delfw01 group=fwire mode=0770"

but I dont understand why I can read that from the master server  ..? and not on other nodes.

Thanks

Gaurav_S
Moderator
Moderator
   VIP    Certified
hmm makes sense that it should allow the read access..... by any chance have you setup solaris auditing ?

Gaurav

UnixFan
Level 3
Yep  ,we got auditing enabled

online         Apr_23   svc:/system/auditd:default


Gaurav_S
Moderator
Moderator
   VIP    Certified

looking for it, in the mean time have a look at this:

http://support.bb4.com/archive/200608/msg00024.html


Not able to remember exactly, there was something around auditing which use to cause this.... trying to find right doc for this...



Gaurav

Dev_Roy
Level 6
Accredited Certified

Can you also check the activation mode for the cvm dg?
Could you please provide us with the following command output from master node?
1.> vxdctl -c mode
2.> vxdg list <cvm dg>
3.> cfscluster status
4.>/etc/vx/bin/vxclustadm nidmap  --> on all nodes
5.>/etc/vx/bin/vxclustadm -v nodestate  --> on all nodes
6.> hastatus -sum
7.> cfscluster config
8.> modinfo | grep -i vx
9.> ps -ef | grep -i vxconf
10.> uname -a

Has this setup worked fine in the past?

Dev_Roy
Level 6
Accredited Certified

Are you able to write to the cfs mount point?
Have you tried "sudo df -k" ?

UnixFan
Level 3

Roy,

If you look at my initial posting I have pasted those info already .. I dont think we need any cluster config details here , as I mentioned earlier I can do it by using the sudo . I want to do it as normal user , which I can do it on master server ,not on other nodes in the cluster

sol100# /etc/vx/bin/vxclustadm nidmap
Name                             CVM Nid    CM Nid     State
sol100                        1          0          Joined: Master
sol200                        0          1          Joined: Slave
sol300                        2          2          Joined: Slave
sol100# /etc/vx/bin/vxclustadm -v nodestate
state: cluster member
        nodeId=0
        masterId=0
        neighborId=2
        members=0x7
        joiners=0x0
        leavers=0x0
        reconfig_seqnum=0x7ba82c
        vxfen=on

---
vscecd300# /etc/vx/bin/vxclustadm nidmap
Name                             CVM Nid    CM Nid     State
sol100                        1          0          Joined: Master
sol200                        0          1          Joined: Slave
sol300                        2          2          Joined: Slave
sol300# /etc/vx/bin/vxclustadm -v nodestate
state: cluster member
        nodeId=2
        masterId=0
        neighborId=1
        members=0x7
        joiners=0x0
        leavers=0x0
        reconfig_seqnum=0x7ba82c
        vxfen=on


sol300# uname -a
SunOS sol300 5.10 Generic_142900-03 sun4v sparc SUNW,SPARC-Enterprise-T5220


sol300# ps -ef | grep -i vxconf
    root    71     1   0   Apr 19 ?           0:37 vxconfigd -x syslog -m boot
    root 24009 23543   0 09:16:03 pts/7       0:00 grep -i vxconf
    root  3150     1   0   Apr 19 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd
    root  4166     1   0   Apr 19 ?         169:44 /opt/VRTSvcs/bin/CVMVxconfigd/CVMVxconfigdAgent -type CVMVxconfigd
    root  3807  3150   0   Apr 19 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd


sol100#  ps -ef | grep -i vxconf
    root    71     1   0   Mar 23 ?           1:04 vxconfigd -x syslog -m boot
    root  4372     1   0   Mar 23 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd
    root  5036  4372   0   Mar 23 ?           0:00 /sbin/sh - /usr/lib/vxvm/bin/vxconfigbackupd
    root  5483     1   0   Mar 23 ?         214:39 /opt/VRTSvcs/bin/CVMVxconfigd/CVMVxconfigdAgent -type CVMVxconfigd
    root 10225  9503   0 09:16:22 pts/12      0:00 grep -i vxconf
sol100# uname -a
SunOS sol100 5.10 Generic_141444-09 sun4v sparc SUNW,SPARC-Enterprise-T5220



sol100# cfscluster status

  Node             : sol100
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /opt/fwire/Shared sharedvol      shareddg          MOUNTED


  Node             :  sol200
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /opt/fwire/Shared sharedvol      shareddg          MOUNTED


  Node             :  sol300
  Cluster Manager  :  running
  CVM state        :  running
  MOUNT POINT    SHARED VOLUME  DISK GROUP        STATUS
  /opt/fwire/Shared sharedvol      shareddg          MOUNTED


Thanks


rregunta
Level 4
I would advise you to check with Sun Solaris Forums, as this appears more over a OS / user / shell / access / permission related issue.

regards
Rajesh Regunta