Tuesday Jan 18, 2005

Integrated Locking

Here's an interesting entry on a faq for sqlite:

The locking mechanism used to control simultaneous access might not work correctly if the database file is kept on an NFS filesystem. This is because file locking is broken on some NFS implementations. You should avoid putting SQLite database files on NFS if multiple processes might try to access the file at the same time.

The protocol behind locking (NLM) for NFSv2/v3 is not broken\*, but rather some of the implementations were broken - especially early on (people spent time on making the NFS protocol work, but NLM was more complex/an after thought). What is interesting is that we find there really is no difference between perception and reality. People perceive locking over NFS to be broken, so (as demonstrated above) they will not use it. Running a Solaris client against a Solaris server will find that NFSv3 + NLM works great, and is not broken, but it doesn't matter - it is perceived to be broken.

Thankfully, NFSv4 fixed this by integrating locking into the one and only protocol. Now, locking has to work or you don't have a NFSv4 implementation. Don't do locking and you have NFSvhalf-ass-effort.

When people refer to NFSv4 as a stateful protocol, this is the main reason. State has to be kept around to ensure proper locking. NLM had to keep track of state, so does NFSv4.

\*Ok, there are a couple very edge cases in NLM that got sorted out in NFSv4. The easiest example is recovery of a client crash. Upon coming back up, the client's status monitor (statd) will contact the server's statd indicating that the client no longer needs any of its locks. So what happens if the client never comes back up, and another client needs to lock the same file? You need administrative intervention at this point. Since NFSv4's state is leased based, no intervention is necessary -- if the client doesn't reclaim its state within the alloted time, the locks are lost.

Monday Dec 13, 2004

Linux Automounter - improved

So the NFS group at Sun has run into interesting situations where a customer has deployed both Solaris and Linux clients, but can't use the more advanced Solaris automounter features simply because they've never been implemented on Linux.

Enter Mike Waychison and his efforts. Oh yeah, this is GPL.

This is good for Linux and this is good for NFS.

So what features are being added?

  • Direct Mounts
  • /net (--hosts access)
  • Lazy mounting of hierarchal / multimounts
  • Lazy unmounting of hierarchal / multimounts
  • Browsing

    Currently this is the 0.4 release of AutofsNG.

    BK trees at: http://autofsng.bkbits.net

    Tarballs are available by FTP:


    - -------

    0.3.1 -> 0.4
  • Kernel automount trapping is now interruptible
  • Expiry fixes for multimounts with > ~1024 entries
  • Agent cleanup, less likelyhood of the agent leaving garbage in /tmp
  • Moved MOUNTPROG xdr generation into the build so we don't rely on librpcsvc.a
  • MOUNTFD is now closed before exec()ing mount(8): killing the agent alone now cancels an automount trap and lookup for userspace.
  • xprt sharing added: Multiple NFS mounts to the same server use a single tcp/udp connection. (allows for > 800 nfs mounts)
  • The kernel mapcache used for browsing now uses it's own slab cache to save a couple bytes.
  • Memory leaks plugged in the kernel browse mapcache.

    - ------------

  • Default mount options in auto.master are currently ignored.
  • Default timeout / timeout on the command line are ignored (all mounts timeout after 10 seconds).
  • The parser may be a little too strict for some maps that use non-nfs(ipv4) mounts.
  • Many applications are broken because they don't understand what a direct mount is (and thus trample into them).
  • The kernel code is hard-coded to use '/sbin/autofsng' to perform the mounts. (installing it anywhere else won't work).
  • Only x86 has syscall table entries at present.

    (patches welcome :)

  • Wednesday Dec 08, 2004

    NFSv4 in Solaris 10

    One new feature of Solaris 10 that has slipped under the radar is NFSv4. I work on the client side for Solaris. You can find the rfc here and the community website here. Original Design Considerations.

    So what's the big deal of NFSv4 anyways?

  • NFSv4 makes security mandatory. NFS IS SECURE!
  • Sun gave up control of NFS to the IETF.
  • A common ACL model (Unix and Windows working together before Gates and Papadopulos made it popular).
  • File level delegation to reduce network latency and server workload for a class of applications.
  • COMPOUND procedure to allow for flexible request construction and reduction of network latency and server workload.
  • Built in minor versioning to extend the protocol through the auspices of the IETF protocol process.
  • Integrated locking (no need of a separate protocol - NLM, and we work with Windows locking semanctics).
  • One protocol and one port (2049) for all functionality.

    So who else is implementing NFSv4?

  • University of Michigan/CITI has putback to Linux 2.6. Back in 1999/2000, this is where I spent my last year in school working.
  • Netapp has already released their implementation.
  • IBM too.
  • Have Windows in your environment? Hummingbird can hook you up today.
  • Rick at the University of Guelph is implementing a server for BSD.

    I'll go into details for some of the features of NFSv4 in future posts.

    We hold regular interopability-fests about every 4 months via Connectathon and (NFSv4 only) bake-a-thons ( last one).
  • About



    « July 2016