Encrypting ZFS pools using lofi crypto
By Darren Moffat-Oracle on Jun 01, 2009
I'm running OpenSolaris 2009.06 on my laptop, soon I'll be running my own development bits of ZFS Crypto but I couldn't do that because OpenSolaris 2009.06 is based on build 111 but the ZFS crypto source is already at build 116. Once the /dev repository catches up to 116 (or later) then I can put ZFS Crypto onto my laptop and run it "real world" rather than just pounding on it with the test script.
In the mean time I had a need to store some confidential and also some highly personal data on my laptop - not something I generally do as it is mostly used for remote access or the sources I have on it are all open source.
I also really wanted to take advantage of the ZFS auto-snapshot service for this data. So I really needed ZFS Crypto but I was also constrained to 2009.06. I could have gone back and used an older build of ZFS crypto that was in sync with 111 but that would have taken me a few days to backport my current sources - not something that I'd consider useful work given the only person that would benefit from this was me and for a short period of time.
It was also only a relatively small amount of data and performance of access wasn't critical. I should also mention that storing it on a USB mass storage device of any kind was a complete non starter given where I was going with this laptop at the time - lets just say one can't take usb disks there.
I wrote previously about the crypto capability we have in the lofi(7D) driver - this is "block" device based crypto and knows nothing about filesystems. As I mentioned before it has a number of weaknesses but by running ZFS on top of it some of those are mitigated for me in the situation I outlined above. Particularly since I'd end up with "ZFS - lofi - ZFS".
What I decided to do was use lofi as a "shim" and create a ZFS pool on a lofi device which was backed by a ZVOL. As I mentioned above the reason for using ZFS as the end filesystem was to take advantage of the auto-snapshot feature, using ZFS below lofi means that I can expand the size of the encrypted area by adjusting the size of the ZVOL if necessary.
darrenm$ export PVOL=rpool/export/home/darrenm/pvol
# zfs create -V 1g $PVOL
darrenm$ pktool genkey keystore=pkcs11 label=$PVOL keylen=256 keytype=aes
Enter PIN for Sun Software PKCS#11 softtoken:
darrenm$ pfexec lofiadm -a /dev/zvol/rdsk/$PVOL -T:::$PVOL -c aes-256-cbc
# zpool create darrenm -O canmount=off -O checksum=sha256 -O mountpoint=/export/home/darrenm darrenm /dev/lofi/1
# zfs allow darrenm create,destroy,mount darrenm
darrenm$ zfs create -o canmount=off darrenm/Documents
darrenm$ zfs create darrenm/Documents/Private
That sets things up the first time. This won't be automatically available after a reboot, infact the "darrenm" pool will be known about but will be in an FAULTED state since its devices won't be present. To make it available again after reboot do this:
darrenm$ pfexec lofiadm -a /dev/zvol/rdsk/$PVOL -T :::$PVOL -c aes-256-cbc darrenm$ pfexec zpool import -d /dev/lofi darrenm
darrenm$ zfs mount -a
A few notes on the choice of naming. I deliberately made the pool named after my usename and set the mountpoint of the top level dataset to match where my home dir is mounted (NOT the automounted /home/ entry but the actual underlying one) and set that as canmount=off so that my default home dir is still unencrypted but I can have subdirs below it that are actually ZFS datasets mounted encrypted from the encrypted pool.
The lofi encryption key is stored (encrypted) in my keystore managed by pkcs11_softtoken. (As it happens my latop has a TPM so I could have stored it in there - expect for the fact that the TPM driver stack isn't fully operational until after build 111). I chose to name the key using the name of the ZVOL just to keep things simple and to remind me what it is for, the label is arbitary but using the ZVOL name will help when this is scripted.
I'm planning on polishing this off a bit and making some scripts available, but first I want to try out some more complex use cases including having the "guest" pool be mirrored. I also want to get the appropriate ZFS delegation and RBAC details correct. For how though the above should be enough to show what is possible here until the "real" ZFS encrypted dataset support appears. This isn't designed to be a replacement for the ZFS crypto project but a, for me at least, a usable workaround using what we have in 2009.06.