Running OpenFiler 2.3 on Oracle VM (Would work in any Xen server)

OpenFiler-Logo.gif

Well, now that my VMWare Server 2 environment is no more and I purchased 5 1TB disks for my RAID 5 array, Avi Miller had the excellent idea of using OpenFiler to manage access to the array. However I had some difficulty in finding the right information to set this up, but the helpful people on the internal mailing list and pulling together some pieces from other forums, IRC and such I was able to get this working. I must admit, I didn't take a very efficient route but it worked for me and well... thats all that matters.

I should also give credit to Belgarath who was immense help in the #openfiler IRC channel on freenode.net. His understanding of both Xen and OpenFiler got me over quite a few bumps.

Before I go into my method of getting the environment working. I initially tried creating the image by building a .img from scratch, partitioning it, extracting the OpenFiler Xen tar into it and then copying in the boot files. I had problems doing this, most likely due to my lack of Linux knowledge. My second attempt was to build a base Oracle Enterprise Linux install, mount the .img via loopback devices, then extract the OpenFiler Xen tar onto that. This however partially worked, I had a log of errors like the following;

/sbin/MAKEDEV: /lib/tls/libc.so.6: version 'GLIBC_2.4' not found (required by /lib/libselinux.so.1)

The image did start, but I ran out of patience trying to understand the issue with these libraries, OpenFiler ships with 2.3.1 and obviously there were files from the underlying Enterprise Linux install that had 2.4 requirements. In the end I went down the following path, like I say, not quite best practice, but it actually had some nice advantages.

Building an OpenFiler VMWare image


So I figured I would first build a VMWare image on my Windows workstation using VMWare Workstation. This let me very quickly get an instance of OpenFiler running and sort out networking, have a play with it before deciding to deploy to OVM. It was relatively simple;

  • Download the OpenFiler ISO installer image.
  • Create a new VMWare image with an 8GB SCSI single virtual file hard disk, 512MB RAM, single CPU and bridged network interface.
  • Boot the VMWare guest and install OpenFiler

Using VMWare I could now easily create a snapshot of the initial install and play with some of the settings without worry.

Convert to Oracle VM (Xen) image


Once I was happy with the setup, I cloned it to give me a simple set of files with no snapshots to transfer over to the OVM server. I copied the directory over into my /OVS/running_pool/ and using the Oracle VM Manager interface I was then able to import the image and convert to a Xen compatible image. The steps to doing the import are;



  • Copy the VMWare image files into /OVS/running_pool/openfiler
  • Log into the Oracle VM Manager website and switch to the Resources tab.
  • Select the Virtual Machine Images sub tab and hit the Import button.
  • Choose Select from Server Pool (Discover and register), this will look in the running_pool for images not already registered with the manager.
  • On the next page select the pool the OVM server is a member of and should then be able to choose the VMWare image you've just copied over.
  • For operating system choose other, enter a console password and click next where you can import the image.
  • The imported image then next needs to be approved, select the image and hit approve. At this point Oracle VM server will convert this image from VMWare to a Xen based one plus a config file.
  • Once converted, don't start the guest, we need to switch this to a paravirtualised guest first.


OpenFiler-OVM01.gif

Modify image for running the Xen kernel


So after the conversion you'll notice that the vm.cfg file that has been created in the same dir the VMWare images are is setup to run the OpenFiler guest in HVM mode. However OpenFiler have already compiled a kernel that supports paravirtualization (PVM). We need to mount the .img file so we can go in, add the files for this Xen PVM kernel and configure the OS to boot using this kernel.


  • First create a loopback device for the image. Note you may well get a different number for loop0.



    [root@ovm-big OPF_OpenFiler]# losetup -fv system.img

    Loop device is /dev/loop0


  • Now expose the partitions on this device.


    [root@ovm-big openfiler]# kpartx -av /dev/loop0

    add map loop0p1 : 0 208782 linear /dev/loop0 63

    add map loop0p2 : 0 14474565 linear /dev/loop0 208845

    add map loop0p3 : 0 2088450 linear /dev/loop0 14683410



  • Now we need to mount these partitions so we can copy over the data from the Xen tar. You will need to now extract a copy of the openfiler-2.3-x86.tar.gz to the filesystem so we can get access to the files we need. Once you done this, mount the partitions like so.


    [root@ovm-big openfiler]# mkdir /mnt/of

    [root@ovm-big openfiler]# mount /dev/mapper/loop0p2 /mnt/of

    [root@ovm-big openfiler]# mount /dev/mapper/loop0p1 /mnt/of/boot



  • Now copy into the /mnt/of/boot folder the files for the Xen kernel from the extracted tar directory. I did a tar zxvf openfiler-2.3-x86.tar.gz -C /install/openfiler-2.3-x86.


    cp /install/openfiler-2.3-x86/boot/*.domU* /mnt/of/boot

    cp -Rv /install/openfiler-2.3-x86/lib/modules/2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU /mnt/of/lib/modules



  • Finally we need to add an entry into the grub bootloader config.


    vi /mnt/of/boot/grub/grub.conf


    And add the folllowing lines to the end of the file.

    title OpenFiler NSA Xen (2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU)

       root (hd0,0)

       kernel /vmlinuz-2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU ro root=LABEL=/ quiet

       initrd /initrd-2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU.img



    Also change the line default=0 to default=1 so that it boots automatically into the Xen kernel. Save out the file and now we need to unmount this image so we can start it up.


  • Run the following to unmount all the pieces of the image.


    [root@ovm-big /]# umount /mnt/of/boot

    [root@ovm-big /]# umount /mnt/of

    [root@ovm-big /]# kpartx -d /dev/loop0

    [root@ovm-big /]# losetup -d /dev/loop0




Prepare space for use via OpenFiler


Now that we have OpenFiler working, we need to setup some space to expose to the VM. I have my3 terrabyte array and two single 500gb disks which will be used. First create valid partitions on all three;



Disk /dev/sdb: 500.1 GB, 500107862016 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdb1 1 60801 488384001 8e Linux LVM


Disk /dev/sdc: 500.1 GB, 500107862016 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdc1 1 60801 488384001 8e Linux LVM


WARNING: GPT (GUID Partition Table) detected on '/dev/sdd'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdd: 2999.9 GB, 2999967547392 bytes

255 heads, 63 sectors/track, 364725 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes


Device Boot Start End Blocks Id System

/dev/sdd1 1 267350 2147483647+ ee EFI GPT



Note that for the 3TB array you must used parted using a GPT partition table and not fdisk due to limitations with the msdos partition type. Now create a physical and then a logical volume on each partition. Below you can see the output of me doing this with one of the 500GB drives.


[root@ovm-big ~]# pvcreate /dev/sdb1

Physical volume "/dev/sdb1" successfully created

[root@ovm-big ~]# vgcreate -s 16M Data0 /dev/sdb1

Volume group "Data0" successfully created

[root@ovm-big 103_contentserver]# vgchange -a z

1 logical volume(s) in volume group "Data0" now active

Configure space in OpenFiler


The final step is to configure the volumes in OpenFiler. First make sure the guest is not running, we need to edit the vm.cfg to expose these new volumes to the OpenFiler guest.

  • In your shell change to the directory where the OpenFiler guest lives, cd /OVS/running_pool/openfiler/
  • vi vm.cfg and edit the disk setting so it looks something like.


    disk = ['file:/OVS/running_pool/openfiler/OpenFiler.img,hda,w',

    'phy:/dev/sdd1,sdb1,w',

    'phy:/dev/sdb1,sdb2,w',

    'phy:/dev/sdc1,sdb3,w',]


  • Save the file.

Now unfortunately even though we have converted the HVM guest to PVM, Oracle VM Manager doesn't realise this. So to startup the image we have to use the command line.

[root@ovm-big /]# cd /OVS/running_pool/openfiler/
[root@ovm-big openfiler]# xm create vm.cfg
Using config file "./vm.cfg".
Started domain openfiler

OpenFiler-OVM02.gif

OpenFiler should now be running! First I checked this by VNC'ing to the console of the guest. This let me watch OpenFiler boot. Once it was up I could login as root, with no password (unless you changed this during the VM build). I changed the root pass which enabled me to then SSH in remotely and check everything was ok. Finally it was time to load up the web based UI. This is accessed via https://filer.host.name:446 and the default username and password is openfiler and password. From here I could continue configuring the volumes, add shares etc. Once I have my OID and OVD install up and running, that will serve as the LDAP server behind this system for authentication...

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

Simon Thorpe, senior consultant at Oracle, blogs about simple and useful tips when working with Oracle technology.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today