By nico on Nov 06, 2007
"Fully integrated" implies many good things:
- proper interaction of SMB share modes, file locks and oplocks with NFSv4equivalents
- proper integration with Solaris administration utilities (e.g., sharemgr(1M))
- support for case- and Unicode normalization-insensitive but case- and normalization preserving filesystems (yes, we now have Unicode normalization code in the kernel!)
- integration of the Solaris and Windows identity models
- filesystem support for the integrated identity model, as well as extended ACLs to support Windows ACL features
That is a very significant list! A lot of work went into this project and related sub-projects.
I'll be blogging about the integration of the Solaris and Windows identity models, both in this post and subsequent ones.
Solaris has distinct, small, flat user and group identity namespaces (POSIX UIDs and GIDs). Windows has a unified, practically unlimited, and non-flat namespace for user and group identities (SIDs). There's a very high impedance mismatch there!
We knew we'd need to map between these two models, so we started a project to do that. We needed to be able to map any valid SID in an AD forest to a Unix UID and/or GID, as needed. And we needed such a system to be low-configuration, easy to use, and safe. Mapping between these models isn't hard, it's the other requirements that were challenging to tackle.
Initially we pursued a notion of persistent dynamic mappings within each Unix (NIS/NIS+/native LDAP) domain, but Mike Shapiro helped us simplify things greatly with an outside-the-box idea: use the heretofore unused "negative" UID/GID namespaces for ephemeral dynamic ID mapping, thus removing two big problems with our earlier design (the need to configure a pool of IDs and the reliability issues associated with having to persistently store important mappings).
"Negative UID/GID namespaces", you ask? Until now uid_t and gid_t have been signed 32-bit integers in Solaris, but the relevant standards (POSIX, SUS) require UIDs and GIDs to be positive integers, which means that we wasted almost half of the uid_t/gid_t namespace. Mike's insight was that we could use that wasted ID namespace as a pool of IDs that we can dynamically allocate IDs from, resetting the pool at boot time, and that this wouldn't be too expensive in terms of incompatibility (more on that below). So we changed the uid_t and gid_t types, and we reserved the 2\^31..2\^32-2 ID namespace for Solaris-driven allocation (i.e., customers cannot assign these IDs directly).
ID mapping then works as follows:
- there is an ID mapping service, svc:/system/idmap:default
- the idmap service is accessed via RPC over doors only (i.e., it's a local service)
- by default the idmap service validates SIDs and maps them to the next available 'ephemeral' UID or GID, and this mapping persists until the system reboots (more on this below)
- the mapping service also offers name-based ID mapping, where you can map Windows domain users and groups to Unix users and groups by name
- the consumers are: the SMB server in the kernel and in user-land utilities, the NFSv4 user-land nfsmapid daemon, the kernel ksid\*() functions (which are called from cr\*() kernel functions that deal with cred_t), and the idmap(1M) utility
Now, using ephemeral IDs in the erstwhile negative ID space has some implications. First and foremost: ephemeral IDs must not be persistently stored anywhere, including in filesystem objects. Because that is far too restrictive the Solaris VFS and one filesystem, ZFS, have been modified to support storing SIDs instead of ephemeral IDs (the other filesystems simply reject any attempt to store an ephemeral ID). You read that right: ZFS can now use SIDs in ACL entries! Most applications will already do the Right Thing -- either reject or pass through ephemeral IDs -- and those core Solaris apps that needed modification have been modified. C++ mangled symbols for methods that take uid_t or gid_t arguments will change on recompile (this was deemed acceptable). For more information you should see the ARC case that covers ephemeral IDs (which will be available soon, as I understand it).
By the way, I think Solaris may now be the first non-Windows implementation of NFSv4 that supports the use of user/group names from many domains on the wire!
Next up: current limitations of ID mapping, ongoing sub-projects, and a guided tour of the source. The impatient can start by looking at:
- the local door RPC protocol used between the kernel/utilities/nfsmapid and idmapd
- the idmap kernel module
- the idmap daemon
- the idmap(1M) utility
- kernel code related to SIDs and cred_t
- and related header files in $SRC/uts/common/sys