Friday Dec 06, 2013

Oracle Linux containers continued

More on Linux containers... the use of btrfs in particular and being able to easily create clones/snapshots of container images. To get started : have an Oracle Linux 6.5 installation with UEKr3 and lxc installed and configured.

lxc by default uses /container as the directory to store container images and metadata. /container/[containername]/rootfs and /container/[containername]/config. You can specify an alternative pathname using -P. To make it easy I added an extra disk to my VM that I use to try out containers (xvdc) and then just mount that volume under /container.

- Create btrfs volume

If not yet installed, install btrfs-progs (yum install btrfs-progs)

# mkfs.btrfs /dev/xvdc1

# mount /dev/xvdc1 /container 
You can auto-mount this at startup by adding a line to /etc/fstab

/dev/xvdc1		/container		btrfs   defaults 0 0

- Create a container

# lxc-create -n OracleLinux59 -t oracle -- -R 5.9
This creates a btrfs subvolume /container/OracleLinux59/rootfs

Use the following command to verify :

# btrfs subvolume list /container/
ID 260 gen 33 top level 5 path OracleLinux59/rootfs

- Start/Stop container

# lxc-start -n OracleLinux59

This starts the container but without extra options your current shell becomes the console of the container.
Add -c [file] and -d for the container to log console output to a file and return control to the shell after starting the container.

# lxc-start -n OracleLinux59 -d -c /tmp/OL59console

# lxc-stop -n OracleLinux59

- Clone a container using btrfs's snapshot feature which is built into lxc

# lxc-clone -o OracleLinux59 -n OracleLinux59-dev1 -s
Tweaking configuration
Copying rootfs...
Create a snapshot of '/container/OracleLinux59/rootfs' in '/container/OracleLinux59-dev1/rootfs'
Updating rootfs...
'OracleLinux59-dev1' created

# btrfs subvolume list /container/
ID 260 gen 34 top level 5 path OracleLinux59/rootfs
ID 263 gen 34 top level 5 path OracleLinux59-dev1/rootfs

This snapshot clone is instantaneous and is a copy on write snapshot.
You can test space usage like this :

# btrfs filesystem df /container
Data: total=1.01GB, used=335.17MB
System: total=4.00MB, used=4.00KB
Metadata: total=264.00MB, used=25.25MB

# lxc-clone -o OracleLinux59 -n OracleLinux59-dev2 -s
Tweaking configuration
Copying rootfs...
Create a snapshot of '/container/OracleLinux59/rootfs' in '/container/OracleLinux59-dev2/rootfs'
Updating rootfs...
'OracleLinux59-dev2' created

# btrfs filesystem df /container
Data: total=1.01GB, used=335.17MB
System: total=4.00MB, used=4.00KB
Metadata: total=264.00MB, used=25.29MB

- Adding Oracle Linux 6.5

# lxc-create -n OracleLinux65 -t oracle -- -R 6.5

lxc-create: No config file specified, using the default config /etc/lxc/default.conf
Host is OracleServer 6.5
Create configuration file /container/OracleLinux65/config
Downloading release 6.5 for x86_64
Configuring container for Oracle Linux 6.5
Added container user:oracle password:oracle
Added container user:root password:root
Container : /container/OracleLinux65/rootfs
Config    : /container/OracleLinux65/config
Network   : eth0 (veth) on virbr0
'oracle' template installed
'OracleLinux65' created

- Install an RPM in a running container

# lxc-attach -n OracleLinux59-dev1 -- yum install mysql
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package mysql.i386 0:5.0.95-3.el5 set to be updated

This connects to the container and executes # yum install mysql inside the container.

- Modify container resource usage

# lxc-cgroup -n OracleLinux59-dev1 memory.limit_in_bytes 53687091

# lxc-cgroup -n OracleLinux59-dev1 cpuset.cpus

# lxc-cgroup -n OracleLinux59-dev1 cpuset.cpus 0,1

Assigns cores 0 and 1. You can also use a range 0-2,...

# lxc-cgroup -n OracleLinux59-dev1 cpu.shares

# lxc-cgroup -n OracleLinux59-dev1 cpu.shares 100

# lxc-cgroup -n OracleLinux59-dev1 cpu.shares

# lxc-cgroup -n OracleLinux59-dev1 blkio.weight

# lxc-cgroup -n OracleLinux59-dev1 blkio.weight 20

A list of resource control parameters :

Lenz has created a Hands-on lab which you can find here :

Wednesday Dec 04, 2013

Oracle Linux containers

So I played a bit with docker yesterday (really cool) and as I mentioned, it uses lxc (linux containers) underneath the covers. To create an image based on OL6, I used febootstrap, which works fine but Dwight Engen pointed out that I should just use lxc-create since it does all the work for you.

Dwight's one of the major contributors to lxc. One of the things he did a while back, was adding support in lxc-create to understand how to create Oracle Linux images. All you have to do is provide a version number and it will figure out which yum repos to connect to on and download the required rpms and install them in a local subdirectory. This is of course superconvenient and incredibly fast. So... I played with that briefly this morning and here's the very short summary.

Start out with a standard Oracle Linux 6.5 install and uek3. Make sure to add/install lxc if it's not yet there (yum install lxc) and you're good to go.

*note - you also have to create /container for lxc - so also do mkdir /container after you install lxc, thank Tony for pointing this out.

# lxc-create -n ol65 -t oracle -- -R 6.5.

That's it. lxc-create will know this is an Oracle Linux container, using OL6.5's repository to create the container named ol65.

lxc-create automatically connects to public-yum, figures out which repos to use for 6.5, downloads all required rpms and generates the container. At the end you will see :

Configuring container for Oracle Linux 6.5
Added container user:oracle password:oracle
Added container user:root password:root
Container : /container/ol65/rootfs
Config    : /container/ol65/config
Network   : eth0 (veth) on virbr0
'oracle' template installed
'ol65' created

Now all you need to do is :

lxc-start --name ol65

And you are up and running with a new container. Very fast, very easy.

If you want an OL5.9 container (or so) just do lxc-create -n ol59 -t oracle -- -R 5.9. Done. lxc has tons of very cool features, which I will get into more later. You can use this model to import images into docker as well, instead of using febootstrap.

#  lxc-create -n ol65 -t oracle -- -R 6.5
#  tar --numeric-owner -jcp -C /container/ol65/rootfs . | \
    docker import - ol6.5
#  lxc-destroy -n ol65

Tuesday Dec 03, 2013

Oracle Linux 6.5 and Docker

I have been following the Docker project with great interest for a little while now but never got to actually try it out at all. I found a little bit of time tonight to at least try hello world.

Since docker relies on cgroups and lxc, it should be easy with uek3. We provide official support for lxc, we are in fact a big contributor to the lxc project (shout out to Dwight Engen) and the docker website says that you need to be on 3.8 for it to just work. So, OL6.5 + UEK3 seems like the perfect combination to start out with.

Here are the steps to do few very simple things:

- Install Oracle Linux 6.5 (with the default UEK3 kernel (3.8.13))

- To quickly play with docker you can just use their example

(*) if you are behind a firewall, set your HTTP_PROXY

-> If you start from a Basic Oracle Linux 6.5 installation, install lxc first. Your out-of-the-box OL should be configured to access the public-yum repositories.

# yum install lxc

-> ensure you mount the cgroups fs

# mkdir -p /cgroup ; mount none -t cgroup /cgroup

-> grab the docker binary

# wget -O docker
# chmod 755 docker

-> start the daemon

(*) again, if you are behind a firewall, set your HTTP_PROXY setting (http_proxy won't work with docker)

# ./docker -d &
-> you can verify if it works

# ./docker version
Client version: 0.7.0
Go version (client): go1.2rc5
Git commit (client): 0d078b6
Server version: 0.7.0
Git commit (server): 0d078b6
Go version (server): go1.2rc5

-> now you can try to download an example using ubuntu (we will have to get OL up there :))

# ./docker run -i -t ubuntu /bin/bash

this will go and pull in the ubuntu template and run bash inside

# ./docker run -i -t ubuntu /bin/bash
WARNING: IPv4 forwarding is disabled.

and now I have a shell inside ubuntu!

-> ok so now on to playing with OL6. Let's create and import a small OL6 image.

-> first install febootstrap so that we can create an image

# yum install febootstrap

-> now you have to point to a place where you have the repoxml file and the packages on an http server. I copied my ISO content over to a place

I will install some basic packages in the subdirectory ol6 (it will create an OL installed image - this is based on what folks did for centos so it works the same (

# febootstrap -i bash -i coreutils -i tar -i bzip2 -i gzip \
-i vim-minimal -i wget -i patch -i diffutils -i iproute -i yum ol6 ol6 http://wcoekaer-srv/ol/

# touch ol6/etc/resolv.conf
# touch ol6/sbin/init

-> tar it up and import it

# tar --numeric-owner -jcpf ol6.tar.gz -C ol6 .
# cat ol6.tar.gz | ./docker import - ol6


List the image

# ./docker images

# ./docker images
REPOSITORY          TAG                 IMAGE ID      
      CREATED             SIZE
ol6                 latest              d389ed8db59d    
      8 minutes ago       322.7 MB (virtual 322.7 MB)
ubuntu              12.04               8dbd9e392a96     
      7 months ago        128 MB (virtual 128 MB)

And now I have a docker image with ol6 that I can play with!

# ./docker run -i -t ol6 ps aux
WARNING: IPv4 forwarding is disabled.
root         1  1.0  0.0  11264   656 ?        R+   23:58   0:00 ps aux

Way more to do but this all just worked out of the box!

# ./docker run ol6 /bin/echo hello world
WARNING: IPv4 forwarding is disabled.
hello world

That's it for now. Next time, I will try to create a mysql/ol6 image and various other things.

This really shows the power of containers on Linux and Linux itself. We have all these various Linux distributions but inside lxc (or docker) you can run ubuntu, debian, gentoo, yourowncustomcrazything and it will just run, old versions of OL, newer versions of OL, all on the same host kernel.

I can run OL6.5 and create OL4, OL5, OL6 containers or docker images but I can also run any old debian or slackware images at the same time.


Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

You can follow him on Twitter at @wimcoekaerts


  • Oracle
« December 2013 »