Oracle VM clustering relies on the OCFS2 clusterstack/filesystem that is native in the kernel (uek2/2.6.39-x). When we create an HA-enabled pool, we create, what we call, a pool filesystem. That filesystem contains an ocfs2 volume so that we can store cluster-wide data. In particular we store shared database files that are needed by the Oracle VM agents on the nodes for HA. It contains info on pool membership, which VMs are in HA mode, what the pool IP is etc...
When the user provides an nfs filesystem for the pool, we do the following :
If someone wants to try out something that relies on block-based shared storage devices, such as ocfs2, but does not have iSCSI or SAN storage, using nfs is an alternative and dm nfs just makes it really easy.
To do this yourself, the following commands will do it for you :
mount mynfsserver:/mountpoint /mnt
dd if=/dev/zero of=/mnt/myvolume.img bs=1M count=2000
dmsetup create myvolume --table "0 4096000 nfs /mnt/myvolume.img 0"
myvolume = the name of the /dev/mapper device. Here we end up with /dev/mapper/myvolume
table = start (normally always 0), number of blocks/length, this is in 512byte blocks, so you double the number,nfs since this is on nfs, filename of the nfs based file,
offset (normally always 0)
So now you have /dev/mapper/myvolume, it acts like a normal block device. If you do this on multiple servers, you can actually create an ocfs2 filesystem on this block device and it will be consistent across the servers.
Credits go to Chuck Lever for writing dm nfs in the first place, thanks Chuck :)
The code for dm nfs is here.