Exalogic Virtual - Creating a vServer using LVM2 for the root file-system

For a customer Proof of Concept - demonstrating the benefits of Oracle Fusion Middleware on Exalogic Virtual - we decided to build a custom base template. All vServers created by using this custom base template enable some customer specific configuration settings, activation of additional services and yields flexibility for the file-system layout.

We decided to use Logical Volume Management LVM2 to achieve the goal of file-system flexibility. This can easily be implemented, as the Base vServer Template, delivered by Oracle, already has all the necessary rpm's installed. The Default Gemini and Navstar vServer Templates come with a fixed partition and file-system layout. To keep a simple layout we went for one logical volume for the root file-system and another volume for swap. By using an ext3 file-system on top of the logical volume for root we are now able to re-size it as needed during normal operation. So whenever logs are filling up /var, or pre-req checks fail due to a lack of space in /tmp we can easily correct this.
In this first blog I will describe how to build a vServer using LVM2. As I said before, for the PoC we planned to use this vServer as a new custom base template. In a second blog post I will show the steps needed to create a vServer template out of this vServer.
This is very similar to what my colleague Andrew Hopkinson has already blogged about here, but needs a few extra steps because of the LVM2 usage.

Here is what I did:

Using EMOC I created a custom vServer Type with:

  • Memmory Size: 4GB
  • Storage Size: >=10GB
  • vCPU's: 2

Using EMOC I created a vServer:

  • In the create vServer wizard I:
    • Selected the Default Navstar vServer Template.
    • Chose the vServer Type created above.
    • Configured EoIB client and shared-storage network.

I used ssh to login to newly created vServer

  • First of all I inspected the original partition setup and file system usage.
  • [root@lvm-tvm ~]# fdisk -l /dev/xvda
    Disk /dev/xvda: 10.7 GB, 10737418240 bytes
    255 heads, 63 sectors/track, 1305 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
        Device Boot      Start         End      Blocks   Id  System
    /dev/xvda1   *           1          13      104391   83  Linux
    /dev/xvda2              14         615     4835565   83  Linux
    /dev/xvda3             616         679      514080   82  Linux swap / Solaris
    [root@lvm-tvm ~]# df
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /dev/xvda2             4687500   3453556   1040524  77% /
    /dev/xvda1              101086     21727     74140  23% /boot
    tmpfs                  2024748         0   2024748   0% /dev/shm
  • Deactivate swap space on xvda3.
  • swapoff /dev/xvda3
  • Change partitions for LVM2 usage by using the below HERE script for fdisk.
  • fdisk  /dev/xvda <<EOF
  • Show new partition layout.
  • [root@lvm-tvm ~]# fdisk -l /dev/xvda
    Disk /dev/xvda: 10.7 GB, 10737418240 bytes
    255 heads, 63 sectors/track, 1305 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
        Device Boot      Start         End      Blocks   Id  System
    /dev/xvda1   *           1          13      104391   83  Linux
    /dev/xvda2              14         615     4835565   83  Linux
    /dev/xvda3             616        1305     5542425   8e  Linux LVM
  • Use partprobe to inform the running OS of the changed partition table. 
  • [root@lvm-tvm ~]# partprobe /dev/xvda
  • Initialize partition xvda3 for LVM2 use.
  • [root@lvm-tvm ~]# pvcreate /dev/xvda3
    WARNING: swap signature detected on /dev/xvda3. Wipe it? [y/n] y
      Wiping swap signature on /dev/xvda3.
      Physical volume "/dev/xvda3" successfully created
  • Create LVM2 volume groups, logical volumes and create and mount file-systems.
  • vgcreate VolGroup00 /dev/xvda3
    lvcreate -L 1G -n swapVol VolGroup00
    lvcreate -L 4G -n rootVol VolGroup00
    mkfs.ext3 -L/root /dev/mapper/VolGroup00-rootVol
    mkdir /mnt/rootfs
    mount -t ext3 /dev/mapper/VolGroup00-rootVol /mnt/rootfs
    mkdir /mnt/rootfs/boot
    mount -t ext3 /dev/xvda1 /mnt/rootfs/boot
  • Copy old root file system content to LVM2 based root file-systems.
  • [root@lvm-tvm ~]# find / -xdev | cpio -pmd /mnt/rootfs
  • Mount special devices and do a chroot.
  • mount -o bind /dev /mnt/rootfs/dev
    chroot /mnt/rootfs
    mount -t proc /proc /proc
    mount -t sysfs /sys /sys
  • Update fstab to reflect new labels, initial ram disk and grub config. Note that I correct the swap setting later.
  • vi /etc/fstab
    LABEL=/root    /                       ext3    defaults        1 1
    LABEL=/boot    /boot                   ext3    defaults        1 2
    tmpfs          /dev/shm                tmpfs   defaults        0 0
    devpts         /dev/pts                devpts  gid=5,mode=620  0 0
    sysfs          /sys                    sysfs   defaults        0 0
    proc           /proc                   proc    defaults        0 0
    LABEL=swap     swap                    swap    defaults        0 0
    cd /boot
    mv initrd-2.6.32-200.21.2.el5uek.img \
    mkinitrd -v --builtin=ehci-hcd --builtin=ohci-hcd \
         --builtin=uhci-hcd initrd-$(uname -r).img $(uname -r)
    vi /boot/grub/grub.conf
    title Oracle Linux Server (2.6.32-200.21.2.el5uek)
            root (hd0,0)
            kernel /vmlinuz-2.6.32-200.21.2.el5uek ro root=LABEL=/root \
                   rhgb tsc=reliable
               nohpet nopmtimer hda=noprobe hdb=noprobe ide0=noprobe \
                   numa=off console=tty0 console=ttyS0,19200n8
            initrd /initrd-2.6.32-200.21.2.el5uek.img

Reuse now free space from former root partition xvda2.

  • Use ssh to login to lvm-tvm again.
  • Inspect current partition and file-system layout.
    [root@lvm-tvm ~]# fdisk -l /dev/xvda
    Disk /dev/xvda: 10.7 GB, 10737418240 bytes
    255 heads, 63 sectors/track, 1305 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
        Device Boot      Start        End      Blocks   Id  System
    /dev/xvda1   *           1         13      104391   83  Linux
    /dev/xvda2              14        615     4835565   83  Linux
    /dev/xvda3             616       1305     5542425   8e  Linux LVM
    [root@lvm-tvm ~]# df
    Filesystem           1K-blocks     Used Available Use% Mounted on
                           4128448   3521732    397004  90% /
    /dev/xvda1              101086     25356     70511  27% /boot
    tmpfs                  2024748         0   2024748   0% /dev/shm

  • Extend Volume Group VolGroup00.

    [root@lvm-tvm ~]# vgextend VolGroup00 /dev/xvda2
      No physical volume label read from /dev/xvda2
      Physical volume "/dev/xvda2" successfully created
      Volume group "VolGroup00" successfully extended

  • Extend root and swap volumes.

    [root@lvm-tvm ~]# lvextend -L+2.89G /dev/VolGroup00/rootVol
      Rounding up size to full physical extent 2.89 GB
      Extending logical volume rootVol to 6.89 GB
      Logical volume rootVol successfully resized
    [root@lvm-tvm ~]# lvextend -L+2G /dev/VolGroup00/swapVol
      Extending logical volume swapVol to 3.00 GB
      Logical volume swapVol successfully resized

  • Resize root file-system.

    [root@lvm-tvm ~]# resize2fs /dev/VolGroup00/rootVol 6G
    resize2fs 1.39 (29-May-2006)
    Filesystem at /dev/VolGroup00/rootVol is mounted on /; on-line resizing required
    Performing an on-line resize of /dev/VolGroup00/rootVol to 1572864 (4k) blocks.
    The filesystem on /dev/VolGroup00/rootVol is now 1572864 blocks long.

  • Verify new root file-system size.

    [root@lvm-tvm ~]# df
    Filesystem           1K-blocks      Used Available Use% Mounted on
                           6192704   3521756   2356404  60% /
    /dev/xvda1              101086     25356     70511  27% /boot
    tmpfs                  2024748         0   2024748   0% /dev/shm

  • Extend swap space

    [root@lvm-tvm ~]# mkswap -L swap /dev/VolGroup00/swapVol
    Setting up swapspace version 1, size = 3221221 kB
    LABEL=swap, no uuid
    [root@lvm-tvm ~]# swapon /dev/VolGroup00/swapVol

  • Verify new swap size

    [root@lvm-tvm ~]# free -m
                 total       used       free     shared    buffers    cached
    Mem:          3954        479       3475          0         42       116
    -/+ buffers/cache:        320       3634
    Swap:         3071          0       3071

we are done!


Post a Comment:
  • HTML Syntax: NOT allowed

The primary contributors to this blog are comprised of the Exalogic and Cloud Application Foundation contingent of Oracle's Fusion Middleware Architecture Team, fondly known as the A-Team. As part of the Oracle development organization, The A-Team supports some of Oracle's largest and most strategic customers worldwide. Our mission is to provide deep technical expertise to support various Oracle field organizations and customers deploying Oracle Fusion Middleware related products. And to collect real world feedback to continuously improve the products we support. In this blog, our experts and guest experts will focus on Exalogic, WebLogic, Coherence, Tuxedo/mainframe migration, Enterprise Manager and JDK/JRockIT performance tuning. It is our way to share some of our experiences with Oracle community. We hope our followers took away something of value from our experiences. Thank you for visiting and please come back soon.


« April 2014