OpenStack Cinder Volume encryption with ZFS
By Darren Moffat-Oracle on Jul 31, 2014
In an OpenStack deployment the VMs is provided by the Cinder service. In the case of a Solaris instance these VMs are either Kernel Zones or non global zones configured for ZOSS (Zones On Shared Storage). When Solaris 11.1 came out I wrote about using ZFS to encrypt zones.
The Cinder volume service for OpenStack can be provided by ZFS using ZVOLs. So it shouldn't be surprising that we get to benefit from ZFS features such as compression, encryption and deduplication.
When deploying a simple OpenStack configuration using the 'solaris.zfs.ZFSVolumeDriver' we create ZVOLs in the dataset specified by the 'zfs_volume_base' variable in /etc/cinder/cinder.conf. If the dataset specified by 'zfs_volume_base' doesn't already exist then the SMF service 'svc:/application/openstack/cinder/cinder-volume:setup' will create it for you and sets the file system permissions and zfs allow delegations for the 'cinder' user appropriately.
If we pre-create the ZFS dataset that zfs_volume_base points to all the ZVOLs that are created by cinder below that are automatically encrypted.
For example if I'm using a ZFS pool called 'cloudstore' and I set 'cloudstore/cinder' as 'zfs_volume_base' I can do this:
# zfs create -o encryption=on -o keysource=passphrase,https://keys.example.com/cinder cloudstore/cinder
In the above example I'm assuming we have an ad-hoc key manager available already that is providing keys/passphrases over https, you could also use a raw file, PKCS#11 keystore or interactively prompt; see the ZFS Encryption documentation for more guidance.
Now restart the cinder-volume:setup service and we are ready to use our transparent encryption of Cinder volumes:
# svcadm restart cinder-volume:setup
If we look at the ZFS datasets that are created after we have launched a VM instance and the cinder volume for it was created we see this:
$ zfs get -r encryption cloudstore/cinder zfs-bugs NAME PROPERTY VALUE SOURCE cloudstore/cinder encryption on local cloudstore/cinder/volume-8ae498b7-5866-60da-85f6-d22d6bc932e9 encryption on inherited from cloudstore/cinder
Using the above method neither Cinder or Nova are aware of the encryption of the volumes nor are they involved in the key management.
We are investigating what will be required to extend the Solaris ZFS drivers for Cinder so that Cinder is involved in or at least aware of ZFS encryption and then eventuall the key management since Cinder has some support for this already and a future OpenStack release will be extending this via the Barbican project.