How to Treat an NFS File As a Block Storage Device

source

Wim actually beat me in blogging about this feature while I was on vacation, but I'd like to add a little more background about dm-nfs, which I gathered from our kernel developers:

What is dm-nfs?

The dm-nfs kernel module provides a device-mapper target that allows you to treat an NFS file as a block device. It provides loopback-style emulation of a block device using a regular file as backing storage. The backing file resides on a remote system and is accessed via the NFS protocol.

The general idea is to have a more-efficient-than-loop access to files on NFS. The device mapper module directly converts requests to the dm device into NFS RPC calls.

dm-nfs is used transparently by Oracle VM's Dom0 when mounting NFS-backed virtual disks. It essentially allows for asynchronous and direct I/O to an NFS-backed block device, which is a lot faster than normal NFS for virtual disks. The Xen block hotplug script has been modified on OVM to look for files which are on NFS filesystems. If the file is on NFS, OVM uses dm-nfs automatically, otherwise it falls back to using the regular (but slower) loop mount method.

The original dm-nfs module was written by Chuck Lever. It has been supported and used by Oracle VM since version 2.2 and is also included in the Unbreakable Enterprise Kernel for Oracle Linux.

Why this feature matters

This feature creates virtual disk devices (LUNs) where the data is stored in an NFS file instead of on local storage. Managed networked storage has many benefits over keeping virtual devices on a disk local to the physical host.

A sample use case is the fast migration of guest VMs for load balancing or if a physical host requires maintenance. This functionality is also possible using iSCSI LUNs, but the advantage of dm-nfs is that you can manage new virtual drives on a local host system, rather than requiring a storage administrator to initialize new LUNs on the storage subsystem. Host administrators can handle their own virtual disk provisioning.

For durability and performance, dm-nfs uses asynchronous and direct I/O so all I/O operations are performed efficiently and coherently. Guest disk data is not double cached on the underlying host. If the underlying host crashes, there's a lower probability of data corruption. If the guest is frozen, a clean backup can be taken of the virtual disk, as you can be certain that its data has been fully written out.

How to use it

You use dm-nfs by first loading the kernel module, then using dmsetup to create a device mapper device on your file. The syntax is very similar to the dm-linear module.

The following sample code demonstrates how to use dmsetup to create a mapped device (/dev/mapper/$dm_nfsdev) for the file $filename that is accessible on a mounted NFS file system:

nblks=`stat -c '%s' $filename`
echo -n "0 $nblks nfs $filename 0" | dmsetup create $dm_nfsdev

Now you can mount /dev/mapper/$dm_nfsdev like any other filesystem image.

- Lenz Grimmer (Oracle Linux Blog)

Website Newsletter Facebook Twitter
Comments:

You say - "It essentially allows for asynchronous and direct I/O to an NFS-backed block device, which is a lot faster than normal NFS for virtual disks."

Can you please be a bit more verbose on how and why dm-nfs is faster than using the NFS file directly ?

Posted by guest on January 10, 2013 at 02:04 AM MST #

Let me quote the developer's comments from the source code, this explains the benefits in more detail:

This driver is separate from dm-loop for several reasons.

1. Provide good data integrity and reasonable performance given
the write delaying behavior of the NFS client.

2. Reduce or eliminate double caching. Data for the target is
already cached above the emulated block device; caching the
backing file's data will pollute the page cache, risk exposing
the data to others who can view pages in the cache, and risk
data integrity when the backing file is accessed by multiple
targets (eg. offline backup).

3. Local file-based targets require extra logic that is not
needed for NFS file-based targets. Extent management is
entirely unnecessary for NFS files, for example.

4. There is no need to protect against file truncation. In
the dm-loop case, truncation could result in writes into
unallocated blocks or blocks allocated to other files. For
NFS files, this is entirely the NFS server's problem.

Posted by guest on January 10, 2013 at 09:01 AM MST #

Can backing files be resized on-line using dm-nfs? I believe the previous loop-file method didn't allow for online resize.

Posted by guest on January 15, 2013 at 11:23 AM MST #

Can backing files be resized on-line using dm-nfs? I believe the previous loop-file method didn't allow for online resize.

Posted by guest on January 15, 2013 at 11:24 AM MST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Contributors:
Rick Ramsey
Kemer Thomson
and members of the OTN community

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
12
13
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Blogs We Like