How to Treat an NFS File As a Block Storage Device
By Lenz Grimmer on Jan 09, 2013
What is dm-nfs?
The dm-nfs kernel module provides a device-mapper target that allows you to treat an NFS file as a block device. It provides loopback-style emulation of a block device using a regular file as backing storage. The backing file resides on a remote system and is accessed via the NFS protocol.
The general idea is to have a more-efficient-than-loop access to files on NFS. The device mapper module directly converts requests to the dm device into NFS RPC calls.
dm-nfs is used transparently by Oracle VM's Dom0 when mounting NFS-backed virtual disks. It essentially allows for asynchronous and direct I/O to an NFS-backed block device, which is a lot faster than normal NFS for virtual disks. The Xen block hotplug script has been modified on OVM to look for files which are on NFS filesystems. If the file is on NFS, OVM uses dm-nfs automatically, otherwise it falls back to using the regular (but slower) loop mount method.
The original dm-nfs module was written by Chuck Lever. It has been supported and used by Oracle VM since version 2.2 and is also included in the Unbreakable Enterprise Kernel for Oracle Linux.
Why this feature matters
This feature creates virtual disk devices (LUNs) where the data is stored in an NFS file instead of on local storage. Managed networked storage has many benefits over keeping virtual devices on a disk local to the physical host.
A sample use case is the fast migration of guest VMs for load balancing or if a physical host requires maintenance. This functionality is also possible using iSCSI LUNs, but the advantage of dm-nfs is that you can manage new virtual drives on a local host system, rather than requiring a storage administrator to initialize new LUNs on the storage subsystem. Host administrators can handle their own virtual disk provisioning.
For durability and performance, dm-nfs uses asynchronous and direct I/O so all I/O operations are performed efficiently and coherently. Guest disk data is not double cached on the underlying host. If the underlying host crashes, there's a lower probability of data corruption. If the guest is frozen, a clean backup can be taken of the virtual disk, as you can be certain that its data has been fully written out.
How to use it
You use dm-nfs by first loading the kernel module, then using dmsetup to create a device mapper device on your file. The syntax is very similar to the dm-linear module.
The following sample code demonstrates how to use dmsetup to create a mapped device (/dev/mapper/$dm_nfsdev) for the file $filename that is accessible on a mounted NFS file system:
nblks=`stat -c '%s' $filename` echo -n "0 $nblks nfs $filename 0" | dmsetup create $dm_nfsdev
Now you can mount /dev/mapper/$dm_nfsdev like any other filesystem image.
- Lenz Grimmer (Oracle Linux Blog)