Well, now that my VMWare Server 2 environment is no more and I purchased 5 1TB disks for my RAID 5 array, Avi Miller had the excellent idea of using OpenFiler to manage access to the array. However I had some difficulty in finding the right information to set this up, but the helpful people on the internal mailing list and pulling together some pieces from other forums, IRC and such I was able to get this working. I must admit, I didn't take a very efficient route but it worked for me and well... thats all that matters.
I should also give credit to Belgarath who was immense help in the #openfiler IRC channel on freenode.net. His understanding of both Xen and OpenFiler got me over quite a few bumps.
Before I go into my method of getting the environment working. I initially tried creating the image by building a .img from scratch, partitioning it, extracting the OpenFiler Xen tar into it and then copying in the boot files. I had problems doing this, most likely due to my lack of Linux knowledge. My second attempt was to build a base Oracle Enterprise Linux install, mount the .img via loopback devices, then extract the OpenFiler Xen tar onto that. This however partially worked, I had a log of errors like the following;
/sbin/MAKEDEV: /lib/tls/libc.so.6: version 'GLIBC_2.4' not found (required by /lib/libselinux.so.1)
The image did start, but I ran out of patience trying to understand the issue with these libraries, OpenFiler ships with 2.3.1 and obviously there were files from the underlying Enterprise Linux install that had 2.4 requirements. In the end I went down the following path, like I say, not quite best practice, but it actually had some nice advantages.
Building an OpenFiler VMWare image
So I figured I would first build a VMWare image on my Windows workstation using VMWare Workstation. This let me very quickly get an instance of OpenFiler running and sort out networking, have a play with it before deciding to deploy to OVM. It was relatively simple;- Download the OpenFiler ISO installer image.
- Create a new VMWare image with an 8GB SCSI single virtual file hard disk, 512MB RAM, single CPU and bridged network interface.
- Boot the VMWare guest and install OpenFiler
Convert to Oracle VM (Xen) image
Once I was happy with the setup, I cloned it to give me a simple set of files with no snapshots to transfer over to the OVM server. I copied the directory over into my /OVS/running_pool/ and using the Oracle VM Manager interface I was then able to import the image and convert to a Xen compatible image. The steps to doing the import are;
|  |
Modify image for running the Xen kernel
So after the conversion you'll notice that the vm.cfg file that has been created in the same dir the VMWare images are is setup to run the OpenFiler guest in HVM mode. However OpenFiler have already compiled a kernel that supports paravirtualization (PVM). We need to mount the .img file so we can go in, add the files for this Xen PVM kernel and configure the OS to boot using this kernel.- First create a loopback device for the image. Note you may well get a different number for loop0.
[root@ovm-big OPF_OpenFiler]# losetup -fv system.img
Loop device is /dev/loop0
- Now expose the partitions on this device.
[root@ovm-big openfiler]# kpartx -av /dev/loop0
add map loop0p1 : 0 208782 linear /dev/loop0 63
add map loop0p2 : 0 14474565 linear /dev/loop0 208845
add map loop0p3 : 0 2088450 linear /dev/loop0 14683410
- Now we need to mount these partitions so we can copy over the data from the Xen tar. You will need to now extract a copy of the openfiler-2.3-x86.tar.gz to the filesystem so we can get access to the files we need. Once you done this, mount the partitions like so.
[root@ovm-big openfiler]# mkdir /mnt/of
[root@ovm-big openfiler]# mount /dev/mapper/loop0p2 /mnt/of
[root@ovm-big openfiler]# mount /dev/mapper/loop0p1 /mnt/of/boot
- Now copy into the /mnt/of/boot folder the files for the Xen kernel from the extracted tar directory. I did a tar zxvf openfiler-2.3-x86.tar.gz -C /install/openfiler-2.3-x86.
cp /install/openfiler-2.3-x86/boot/*.domU* /mnt/of/boot
cp -Rv /install/openfiler-2.3-x86/lib/modules/2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU /mnt/of/lib/modules
- Finally we need to add an entry into the grub bootloader config.
vi /mnt/of/boot/grub/grub.conf
And add the folllowing lines to the end of the file.title OpenFiler NSA Xen (2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU)
root (hd0,0)
kernel /vmlinuz-2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU ro root=LABEL=/ quiet
initrd /initrd-2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU.img
Also change the line default=0 to default=1 so that it boots automatically into the Xen kernel. Save out the file and now we need to unmount this image so we can start it up.
- Run the following to unmount all the pieces of the image.
[root@ovm-big /]# umount /mnt/of/boot
[root@ovm-big /]# umount /mnt/of
[root@ovm-big /]# kpartx -d /dev/loop0
[root@ovm-big /]# losetup -d /dev/loop0
Prepare space for use via OpenFiler
Now that we have OpenFiler working, we need to setup some space to expose to the VM. I have my3 terrabyte array and two single 500gb disks which will be used. First create valid partitions on all three;
Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 8e Linux LVM
Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 60801 488384001 8e Linux LVM
WARNING: GPT (GUID Partition Table) detected on '/dev/sdd'! The util fdisk doesn't support GPT. Use GNU Parted.
Disk /dev/sdd: 2999.9 GB, 2999967547392 bytes
255 heads, 63 sectors/track, 364725 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 267350 2147483647+ ee EFI GPT
Note that for the 3TB array you must used parted using a GPT partition table and not fdisk due to limitations with the msdos partition type. Now create a physical and then a logical volume on each partition. Below you can see the output of me doing this with one of the 500GB drives.
[root@ovm-big ~]# pvcreate /dev/sdb1
Physical volume "/dev/sdb1" successfully created
[root@ovm-big ~]# vgcreate -s 16M Data0 /dev/sdb1
Volume group "Data0" successfully created
[root@ovm-big 103_contentserver]# vgchange -a z
1 logical volume(s) in volume group "Data0" now active
Configure space in OpenFiler
The final step is to configure the volumes in OpenFiler. First make sure the guest is not running, we need to edit the vm.cfg to expose these new volumes to the OpenFiler guest.- In your shell change to the directory where the OpenFiler guest lives, cd /OVS/running_pool/openfiler/
- vi vm.cfg and edit the disk setting so it looks something like.
disk = ['file:/OVS/running_pool/openfiler/OpenFiler.img,hda,w',
'phy:/dev/sdd1,sdb1,w',
'phy:/dev/sdb1,sdb2,w',
'phy:/dev/sdc1,sdb3,w',]
- Save the file.
Now unfortunately even though we have converted the HVM guest to PVM, Oracle VM Manager doesn't realise this. So to startup the image we have to use the command line.
[root@ovm-big /]# cd /OVS/running_pool/openfiler/
[root@ovm-big openfiler]# xm create vm.cfg
Using config file "./vm.cfg".
Started domain openfiler
OpenFiler should now be running! First I checked this by VNC'ing to the console of the guest. This let me watch OpenFiler boot. Once it was up I could login as root, with no password (unless you changed this during the VM build). I changed the root pass which enabled me to then SSH in remotely and check everything was ok. Finally it was time to load up the web based UI. This is accessed via https://filer.host.name:446 and the default username and password is openfiler and password. From here I could continue configuring the volumes, add shares etc. Once I have my OID and OVD install up and running, that will serve as the LDAP server behind this system for authentication...
]]>
]]>