chown(1) and username spaces under NFSv3 and NFSv4

A colleague came to me with an interesting question. They had a share on a server which allowed root access and could not get chown(1) to work via NFS. They wanted to know why our NFS server implementation denied it.

I immediately ran the test and it worked right off the bat! So now the question was what was different on the two systems? First of all, I tried it with a Linux client:

root@silent:~# mount slammer:/export/slammer /slammer/slammer/
root@silent:~# cd /slammer/slammer/
root@silent:/slammer/slammer# touch sil
root@silent:/slammer/slammer# chown thud sil
root@silent:/slammer/slammer# ls -la sil
-rw-r--r--+ 1 thud root 0 2009-11-11 12:40 sil

To eliminate that as the culprit, I tried an OpenSolaris client:

[root@silver tdh]> mount slammer:/export/slammer /slammer/slammer/
[root@silver tdh]> cd /slammer/slammer/
[root@silver slammer]> touch ver
[root@silver slammer]> chown thud ver
chown: ver: Permission denied

While it appears to be an issue between OpenSolaris and Linux, it is far from the case. The issue is at the NFS protocol layer and I can easily show it by flipping the success on each client!

Working OpenSolaris:

[root@silver slammer]> cd
[root@silver ~]> umount /slammer/slammer/
[root@silver ~]> mount -o vers=3 slammer:/export/slammer /slammer/slammer/
[root@silver ~]> cd /slammer/slammer/
[root@silver slammer]> touch ver_3
[root@silver slammer]> chown thud ver_3
[root@silver slammer]> ls -la ver_3
-rw-r--r--   1 thud     root           0 Nov 11 12:50 ver_3

Broken Linux:

root@silent:~# mount -t nfs4 slammer:/export/slammer /slammer/slammer/
root@silent:~# cd /slammer/slammer/
root@silent:/slammer/slammer# touch sil_4
root@silent:/slammer/slammer# chown thud sil_4
root@silent:/slammer/slammer# ls -la sil_4
-rw-r--r-- 1 thud 4294967294 0 2009-11-11 12:53 sil_4

But wait, you are saying it worked for Linux. Well, no, it did not. If we look at this on the server, we see:

slammer# ls -la sil\* ver\*
-rw-r--r--   1 2008     root           0 Nov 11 18:40 sil
-rw-r--r--   1 nobody   root           0 Nov 11 18:53 sil_4
-rw-r--r--   1 nobody   nobody         0 Nov 11 18:44 ver
-rw-r--r--   1 2008     root           0 Nov 11 18:50 ver_3

So while the Linux client thinks that it worked, it did not. We can confirm that by remounting with NFSv3:

root@silent:/slammer/slammer# ls -la sil\* ver\*
-rw-r--r--+ 1 thud       root       0 2009-11-11 12:40 sil
-rw-r--r--+ 1 4294967294 root       0 2009-11-11 12:53 sil_4
-rw-r--r--+ 1 4294967294 4294967294 0 2009-11-11 12:44 ver
-rw-r--r--+ 1 thud       root       0 2009-11-11 12:50 ver_3

And for fun, if we remount with NFSv4:

root@silent:/slammer/slammer# cd
root@silent:~# umount /slammer/slammer/
root@silent:~# mount -t nfs4 slammer:/export/slammer /slammer/slammer/
root@silent:~# cd /slammer/slammer/
root@silent:/slammer/slammer# ls -la sil\* ver\*
-rw-r--r-- 1 4294967294 4294967294 0 2009-11-11 12:40 sil
-rw-r--r-- 1 4294967294 4294967294 0 2009-11-11 12:53 sil_4
-rw-r--r-- 1 4294967294 4294967294 0 2009-11-11 12:44 ver
-rw-r--r-- 1 4294967294 4294967294 0 2009-11-11 12:50 ver_3

The first thing we normally check in such cases is the mapid domain:

slammer# cat /var/run/nfs4_domain
internal.excfb.com
[root@silver slammer]> cat /var/run/nfs4_domain
internal.excfb.com
root@silent:/slammer/slammer# cat /proc/net/rpc/nfs4.idtoname/content
#domain type id [name]

So I need to get the Linux box in the right domain! Actually, I need to modify '/etc/default/nfs-common' on this Ubuntu system and tell it to run mapid!

Much better:

root@silent:/slammer/slammer# touch sil_4_2
root@silent:/slammer/slammer# chown thud sil_4_2
chown: changing ownership of `sil_4_2': Invalid argument

Now the Linux and OpenSolaris clients are agreeing to behave the same way with the server! What is the issue here?

In NFSv4, we change the owner with a SETATTR compound op:

NFS:  Op = 34 (SETATTR)
NFS:  State ID hash = SPECIAL_0
NFS:      len = 12    val = 000000000000000000000000
NFS:      State ID Sequence ID = 00000000
NFS:    0x00
NFS:    0x00
NFS:    0x00
NFS:    0x00
NFS:    0x10   OWNER
NFS:    0x00
NFS:    0x00
NFS:    0x00
NFS:  Owner = thud@internal.excfb.com

And in NFSv3, we try with this:

NFS:  Proc = 2 (Set file attributes)
NFS:  File handle = [86D5]
NFS:   A76A323408831F040A001500000000003F0C00000A000300000000002D000000
NFS:  Mode = (not set)
NFS:  User ID = 2008
NFS:  Group ID = (not set)
NFS:  Size = (not set)
NFS:  Access time = (do not set)
NFS:  Modification time = (do not set)
NFS:

So in the NFSv4 case, we need to be able to map 'thud@internal.excfb.com' to an user id. If that mapping does not exist, then we disallow the action. In NFSv3, we already have the user id, and we don't care if it is undefined on the server:

slammer# grep thud /etc/passwd
slammer# ls -la sil\* ver\*
-rw-r--r--   1 2008     root           0 Nov 11 18:40 sil
-rw-r--r--   1 nobody   root           0 Nov 11 18:53 sil_4
-rw-r--r--   1 root     root           0 Nov 11 19:18 sil_4_2
-rw-r--r--   1 nobody   nobody         0 Nov 11 18:44 ver
-rw-r--r--   1 2008     root           0 Nov 11 18:50 ver_3

If we add the user id of thud, we see:

slammer# useradd -u 2009 -g 10 -d /export/slammer thud
slammer# ls -la sil\* ver\*
-rw-r--r--   1 2008     root           0 Nov 11 18:40 sil
-rw-r--r--   1 nobody   root           0 Nov 11 18:53 sil_4
-rw-r--r--   1 root     root           0 Nov 11 19:18 sil_4_2
-rw-r--r--   1 nobody   nobody         0 Nov 11 18:44 ver
-rw-r--r--   1 2008     root           0 Nov 11 18:50 ver_3

Yes, I did an off by one error on purpose! What happens if we try to chown(1) via NFSv4 now:

root@silent:/slammer/slammer# chown thud base
root@silent:/slammer/slammer# ls -la base sil\* ver\*
-rw-r--r-- 1 thud   root    0 2009-11-11 13:29 base
-rw-r--r-- 1 nobody root    0 2009-11-11 12:40 sil
-rw-r--r-- 1 nobody root    0 2009-11-11 12:53 sil_4
-rw-r--r-- 1 root   root    0 2009-11-11 13:18 sil_4_2
-rw-r--r-- 1 nobody nogroup 0 2009-11-11 12:44 ver
-rw-r--r-- 1 nobody root    0 2009-11-11 12:50 ver_3

Those 'nobody's should be 'thud' and that 'thud' should not map! What does NFSv3 say:

root@silent:/slammer/slammer# ls -la base sil\* ver\*
-rw-r--r--+ 1       2009 root       0 2009-11-11 13:29 base
-rw-r--r--+ 1 thud       root       0 2009-11-11 12:40 sil
-rw-r--r--+ 1 4294967294 root       0 2009-11-11 12:53 sil_4
-rw-r--r--+ 1 root       root       0 2009-11-11 13:18 sil_4_2
-rw-r--r--+ 1 4294967294 4294967294 0 2009-11-11 12:44 ver
-rw-r--r--+ 1 thud       root       0 2009-11-11 12:50 ver_3

The NFSv4 listing shows you the username space of the server and the NFSv3 listing shows you the username space of the client.

The big difference is that NFSv4 enforces that both systems understand the name. It doesn't mandate that the names have the same userid, just that they have a mapping.

To get everything in sync, to have both NFSv3 and NFSv4 work the same, use either NIS or LDAP to maintain your user mappings.


Originally posted on Kool Aid Served Daily
Copyright (C) 2009, Kool Aid Served Daily
Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

tdh

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today