Sunday Jan 19, 2014

OpenSCAP distributed with Oracle VM Server for x86

Security Compliance : true

We recently released Oracle VM Server for x86 3.2.7. For more information you can go here. In addition we also recently released Oracle Linux 6.5. Find the press release here and the link to the release notes here.

You will notice that for Oracle Linux we have updated the version of OpenSCAP to use the NIST SCAP 1.2 specification.

We have also decided to distribute OpenSCAP with Oracle VM Server for x86 so you will be able to use the same utility for security compliance checks that you may use with Oracle Linux and Oracle Solaris. Initially, the OpenSCAP package we are distributing with Oracle VM Server for x86 is available on the Oracle Public Yum Server, so you may start by using the oscap(8) - OpenSCAP command line tool after you've installed the openscap-utils RPM on your Dom0 test environment. If you are working on the technical security controls that are required by your organization for the approval to operate Oracle VM Server for x86, then you should understand that OpenSCAP is an effective tool to demonstrate security compliance to your authorizing official. However, you should carefully examine your organizations SCAP content and the implementation details such as the use of OVAL for compliance checks.

We typically recommend that you do not directly execute additional utilities within the Oracle VM Server management domain (i.e. the Dom0 domain), but checking security compliance requires careful limited access by your authorized administrators to produce the reports. The Oracle VM Security Guide for Release 3 explains the philosophy of protection for the installation of the Oracle VM Server using a small footprint:

"Oracle VM Server runs a lightweight, optimized version of Oracle Linux. It is based upon an updated version of the Xen hypervisor technology and includes Oracle VM Agent. The installation of Oracle VM Server in itself is secure: it has no unused packages or applications and no services listening on any ports except for those required for the operation of the Oracle VM environment."

Please note that you should report any potential security vulnerabilities in Oracle products following the instructions found here.

We posted some helpful details about Oracle Linux Errata and CVE information this time last year and you may also review the notifications of Oracle VM errata here. For the examples we are reviewing now, the use of OVAL checks is a part the traditional ways you would show that your servers are all compliant (locked-down or hardened) with relevant security settings in your checklists that reference the product security guides.

The Oracle Software Security Assurance Secure Configuration Initiative has established Oracle product security goals for both Secure Configuration and Security Guides. We have built in the security features with Oracle VM Server for x86 and you should expect that the default installation follows the software security assurance guidelines. Using OpenSCAP for security compliance checks may help you to show that the Oracle VM Server for x86 configuration is up to date with the latest details documented in the security guides for operating systems and server virtualization.

A standardized approach to security compliance is a goal that many organizations are working toward and includes a broad set of security controls typically found within a complete Risk Management Framework provided by the NIST RMF and other standards bodies within the international IT security community. When you begin to use OpenSCAP you will find that the standard SCAP content contains product specific technical security controls that are expected to be unique and have version dependencies as well. You will notice the standard SCAP content used with OpenSCAP on Oracle VM Server for x86 can produce valid securty compliance reports, but you must still understand the technical nuances for measuring compliance that show results for each test:

    Not Applicable  
    Not Evaluated

Advantages to using a standardized approach for security compliance include considerations of "what is measured" and "how it is measured" to improve the precision, accuracy and ultimate effectiveness required to mitigate risks. The initial results that are produced using OpenSCAP for security compliance checks must be further examined to truly understand the meaning of 'true' or 'false' so that you can demonstrate the rationalization for applying any fixes to re-mediate a verifiable problem. The effectiveness of OpenSCAP depends on the thorough understanding of all the technical details at the early stages of your testing, so you will benefit by the complete coverage that may be repeated for all of your production Oracle VM Servers.

Automating system administration activities is a fundamental objective for on-premise and cloud computing architectures and we are working to standardize as much of the enterprise infrastructure components as possible to produce the most cost effective solutions using Oracle VM Server. The security compliance requirements of many organizations have increased reporting cycles that must be continuously monitored. With careful planning, OpenSCAP may be an effective tool for reporting your organizations IT security controls, but we want to review some of the basic concepts that you should be aware of.

We noted earlier that Dom0 is a special purpose management domain that is based on Xen built with Oracle Linux. The Oracle Linux and Oracle Solaris configurations share a common set of technical security controls that are useful to measure consistently with Oracle VM Server. However, the results you analyse requires historic perspectives and current insight to determine the relevance and criticality that is important to convey to the decision makers or authorizing officials in your organization.

One random example of a security compliance check that illustrates a number of considerations is related to CWE-264: Permissions, Privileges, and Access Controls. More specifically, as an exercise, we want to drill down to both CWE-275: Permission Issues and CWE-426: Untrusted Search Path potential problems.

To demonstrate how OpenSCAP can be used to report the results of a check related to CWE-275 and CWE-426 we can start by viewing the Red Hat 5 STIG Benchmark, Version 1, Release 4 from DISA:

[root@ovm327 ~]# wget

For brevity, we have extracted out the OVAL compliance item for 'STIG ID: GEN000960' that we show using the DISA STIG Viewer:

If you also want to test this, here is the raw XML

This looks simple enough, so let's see the result using OpenSCAP on Oracle VM Server for x86:

[root@ovm327 ~]# oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: true
Evaluation done.
[root@ovm327 ~]#

We think we understand the result but let's view this differently just to be sure:

[root@ovm327 ~]# ls -ldL `echo $PATH | tr ':' '\n'`
ls: /root/bin: No such file or directory
drwxr-xr-x 2 root root  4096 Jan  2 12:45 /bin
drwxr-xr-x 2 root root  4096 Jan  2 12:45 /sbin
drwxr-xr-x 3 root root 16384 Jan  2 12:45 /usr/bin
drwxr-xr-x 2 root root  4096 Feb 16  2010 /usr/local/bin
drwxr-xr-x 2 root root  4096 Feb 16  2010 /usr/local/sbin
drwxr-xr-x 2 root root 12288 Jan  2 12:45 /usr/sbin
[root@ovm327 ~]#

This looks good to us, but let's make the '/root/bin' directory that we intentionally want to violate the compliance check to see what happens:

[root@ovm327 ~]# mkdir -m 0777 /root/bin
[root@ovm327 ~]# ls -ldL `echo $PATH | tr ':' '\n'`
drwxr-xr-x 2 root root  4096 Jan  2 12:45 /bin
drwxrwxrwx 2 root root  4096 Jan  2 13:55 /root/bin
drwxr-xr-x 2 root root  4096 Jan  2 12:45 /sbin
drwxr-xr-x 3 root root 16384 Jan  2 12:45 /usr/bin
drwxr-xr-x 2 root root  4096 Feb 16  2010 /usr/local/bin
drwxr-xr-x 2 root root  4096 Feb 16  2010 /usr/local/sbin
drwxr-xr-x 2 root root 12288 Jan  2 12:45 /usr/sbin
[root@ovm327 ~]# oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: false
Evaluation done.
[root@ovm327 ~]#

We have reasonably good confirmation that the OVAL compliance check works the way we expect. However, if we look at the entire set of permissions that enforce the discretionary access control policy, we then realize that there are also permissions on the '/root' directory that prevent the write operations by 'others' in the '/root/bin' directory from succeeding:

[root@ovm327 ~]# ls -ldL /root /root/bin
drwxr-x--- 4 root root 4096 Jan  2 13:55 /root
drwxrwxrwx 2 root root 4096 Jan  2 13:55 /root/bin
[root@ovm327 ~]#

We are not suggesting that the mode '0777' permissions on the '/root/bin' are acceptable because we have safer permissions on the '/root' directory, but the example shows that the OVAL check does not completely test the security controls exactly how the kernel enforces the permissions. We should justifiably state that the result of the OVAL security compliance check '0777' permissions on the '/root/bin' directory is a 'condition negative' with a 'test outcome negative' (i.e. a true negative), but also continue to note our other observations related to the access control enforcement.

Before proceeding, we will clean up the problem we just temporarily created on our test server:

[root@ovm327 ~]# chmod 0700 /root/bin
[root@ovm327 ~]# ls -ldL /root /root/bin
drwxr-x--- 4 root root 4096 Jan  2 13:55 /root
drwx------ 2 root root 4096 Jan  2 13:55 /root/bin
[root@ovm327 ~]# oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: true
Evaluation done.
[root@ovm327 ~]#

Hopefully you find this random security compliance check interesting and somewhat enlightening to illustrate what OpenSCAP can help you with. To continue, we decided to check a slightly different way to demonstrate the same security control:

[root@ovm327 ~]# wget

To simplify viewing the portion of the OVAL compliance entry we extracted it like we did with the DISA STIG item. If you also want to test this, here is the raw XML

Now we can show similar results using a slightly different implementation of the compliance check:

[root@ovm327 ~]# oscap oval eval fedora-accounts_root_path_dirs_no_write.xml
Definition true
Evaluation done.
[root@ovm327 ~]# chmod 0770 /root/bin
[root@ovm327 ~]# oscap oval eval fedora-accounts_root_path_dirs_no_write.xml
Definition false
Evaluation done.
[root@ovm327 ~]#

But we can also see that it is indeed a different check because it includes the test for group write permissions and the 'STIG ID: GEN000960' does not:

[root@ovm327 ~]# chmod 0770 /root/bin
[root@ovm327 ~]# oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: true
Evaluation done.
[root@ovm327 ~]#

Again, let's fix the problem we temporarily created on our test server:

[root@ovm327 ~]# chmod 0700 /root/bin
[root@ovm327 ~]#

You should also review the CIS Oracle Solaris 11.1 Benchmark v1.0.0 and the CIS Red Hat Enterprise Linux 6 Benchmark v1.2.0 to see that they both have the same entry to 'Ensure root PATH Integrity (Scored)' that has an audit section showing script commands that step through multiple potential security compliance issues to check. It is a common practice to combine similar checks in a group, but you may need to parse out the results to obtain a discrete value for a singular check.

As an additional consideration, let's shift our focus away from the differences within OVAL compliance definitions, to the different operating systems that the SCAP content was orignially written for. For this part of our testing we start up an Oracle Solaris 11.1 X86 instance running on a VM to demonstrate the OpenSCAP tests with the same OVAL compliance checks:

root@sol11:/root# pkg install security/compliance/openscap

root@sol11:/root# ls -ldL `echo $PATH | tr ':' '\n'`
drwxr-xr-x   4 root     bin         1126 Jan  2 14:05 /usr/bin
drwxr-xr-x   4 root     bin          445 Jan  2 13:54 /usr/sbin
root@sol11:/root# oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: true
Evaluation done.
root@sol11:/root# oscap oval eval fedora-accounts_root_path_dirs_no_write.xml
Definition true
Evaluation done.
root@sol11:/root# export PATH=$PATH:/tmp
root@sol11:/root# ls -ldL `echo $PATH | tr ':' '\n'`
drwxrwxrwt   5 root     sys          432 Jan  2 14:09 /tmp
drwxr-xr-x   4 root     bin         1126 Jan  2 14:05 /usr/bin
drwxr-xr-x   4 root     bin          445 Jan  2 13:54 /usr/sbin
root@sol11:/root# oscap oval eval GEN000960.xmlDefinition
oval:mil.disa.fso.rhel:def:77: false
Evaluation done.
root@sol11:/root# oscap oval eval fedora-accounts_root_path_dirs_no_write.xml
Definition false
Evaluation done.

Now let's repeat the same OpenSCAP checks with a non-root user account:

admin@sol11:~$ ls -ldL `echo $PATH | tr ':' '\n'`
drwxr-xr-x   4 root     bin         1126 Jan  2 14:05 /usr/bin
drwxr-xr-x   4 root     bin          445 Jan  2 13:54 /usr/sbin
admin@sol11:~$ oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: true
Evaluation done.
admin@sol11:~$ oscap oval eval fedora-accounts_root_path_dirs_no_write.xml
Definition true
Evaluation done.
admin@sol11:~$ export PATH=$PATH:/tmp
admin@sol11:~$ ls -ldL `echo $PATH | tr ':' '\n'`
drwxrwxrwt   5 root     sys          432 Jan  2 14:09 /tmp
drwxr-xr-x   4 root     bin         1126 Jan  2 14:05 /usr/bin
drwxr-xr-x   4 root     bin          445 Jan  2 13:54 /usr/sbin
admin@sol11:~$ oscap oval eval GEN000960.xml
Definition oval:mil.disa.fso.rhel:def:77: false
Evaluation done.
admin@sol11:~$ oscap oval eval fedora-accounts_root_path_dirs_no_write.xml
Definition false
Evaluation done.

We have discovered some additional interesting considerations when reviewing the OpenSCAP results executed on Oracle Solaris:

    The OVAL content appears to also work on Oracle Solaris 11.1
    The OVAL check is on the current PATH environment variable
    The OVAL check is for the current user shell or cron(1M) process running oscap(8)
    The OVAL check does not look for scripts that set the PATH for application run time environments
    The OVAL check does not account for more sophisticated access control technology

To further our understanding of the OVAL content, we decided to run the jOVAL tool which is not included with Oracle Solaris:

admin@sol11:~$ echo $PATH
admin@sol11:~$ /usr/share/jOVAL/jovaldi -l 1 -m -o GEN000960.xml

jOVAL Definition Interpreter
Build date: Thursday, January  2, 2014 04:46:39 PM PST
Copyright (c) 2011-2013 -

Plugin: Default Plugin
Copyright (C) 2011-2013 -

Start Time: Fri Jan 02 16:50:05 2014

 ** parsing /home/admin/GEN000960.xml
     - validating xml schema.
 ** checking schema version
     - Schema version - 5.4
 ** skipping Schematron validation
 ** creating a new OVAL System Characteristics file.
 ** gathering data for the OVAL definitions.
      Collecting object:  FINISHED                      
 ** saving data model to system-characteristics.xml.
 ** skipping Schematron validation
 ** running the OVAL Definition analysis.
      Analyzing definition:  FINISHED                    
 ** OVAL definition results.

    OVAL Id                                 Result
    oval:mil.disa.fso.rhel:def:77           true

 ** finished evaluating OVAL definitions.

 ** saving OVAL results to results.xml.
 ** skipping Schematron validation
 ** running OVAL Results xsl: /usr/share/jOVAL/xml/results_to_html.xsl.

admin@sol11:~$ echo $PATH
admin@sol11:~$ /usr/share/jOVAL/jovaldi -l 1 -m
  -o fedora-accounts_root_path_dirs_no_write.xml

jOVAL Definition Interpreter
Build date: Thursday, January  2, 2014 04:46:39 PM PST
Copyright (c) 2011-2013 -

Plugin: Default Plugin
Copyright (C) 2011-2013 -

Start Time: Fri Jan 02 16:50:30 2014

 ** parsing /home/admin/fedora-accounts_root_path_dirs_no_write.xml
     - validating xml schema.
 ** checking schema version
     - Schema version - 5.5
 ** skipping Schematron validation
 ** creating a new OVAL System Characteristics file.
 ** gathering data for the OVAL definitions.
      Collecting object:  FINISHED                         
 ** saving data model to system-characteristics.xml.
 ** skipping Schematron validation
 ** running the OVAL Definition analysis.
      Analyzing definition:  FINISHED                        
 ** OVAL definition results.

    OVAL Id                                 Result
    -------------------------------------------------------       false

 ** finished evaluating OVAL definitions.

 ** saving OVAL results to results.xml.
 ** skipping Schematron validation
 ** running OVAL Results xsl: /usr/share/jOVAL/xml/results_to_html.xsl.


For now, this concludes our initial investigation of OpenSCAP to show the potential effectiveness on Oracle VM Server for x86 with careful consideration of the results you may observe with your SCAP content. You will also want to understand the XCCDF security checklists that are most often used to perform more complete security compliance checks with OpenSCAP in the same way you can check for STIG compliance:

# oscap xccdf eval --profile stig-rhel6-server --report report.html 
   --results results.xml --cpe ssg-rhel6-cpe-dictionary.xml ssg-rhel6-xccdf.xml

We hope that the random security compliance example we chose will help to illustrate that the use of OpenSCAP is not a substitute for adequately proficient expertise for analyzing IT security controls, but it allows for the repetitive checks in your production Oracle VM Servers after you have completed sufficient testing. Please contact your Oracle representitives if you have any quetions or place service requests with Oracle Support when you encounter problems.

Finally, please remember that you should report any potential security vulnerabilities in Oracle products following the instructions found here.

Friday Dec 06, 2013

Oracle Linux containers continued

More on Linux containers... the use of btrfs in particular and being able to easily create clones/snapshots of container images. To get started : have an Oracle Linux 6.5 installation with UEKr3 and lxc installed and configured.

lxc by default uses /container as the directory to store container images and metadata. /container/[containername]/rootfs and /container/[containername]/config. You can specify an alternative pathname using -P. To make it easy I added an extra disk to my VM that I use to try out containers (xvdc) and then just mount that volume under /container.

- Create btrfs volume

If not yet installed, install btrfs-progs (yum install btrfs-progs)

# mkfs.btrfs /dev/xvdc1

# mount /dev/xvdc1 /container 
You can auto-mount this at startup by adding a line to /etc/fstab

/dev/xvdc1		/container		btrfs   defaults 0 0

- Create a container

# lxc-create -n OracleLinux59 -t oracle -- -R 5.9
This creates a btrfs subvolume /container/OracleLinux59/rootfs

Use the following command to verify :

# btrfs subvolume list /container/
ID 260 gen 33 top level 5 path OracleLinux59/rootfs

- Start/Stop container

# lxc-start -n OracleLinux59

This starts the container but without extra options your current shell becomes the console of the container.
Add -c [file] and -d for the container to log console output to a file and return control to the shell after starting the container.

# lxc-start -n OracleLinux59 -d -c /tmp/OL59console

# lxc-stop -n OracleLinux59

- Clone a container using btrfs's snapshot feature which is built into lxc

# lxc-clone -o OracleLinux59 -n OracleLinux59-dev1 -s
Tweaking configuration
Copying rootfs...
Create a snapshot of '/container/OracleLinux59/rootfs' in '/container/OracleLinux59-dev1/rootfs'
Updating rootfs...
'OracleLinux59-dev1' created

# btrfs subvolume list /container/
ID 260 gen 34 top level 5 path OracleLinux59/rootfs
ID 263 gen 34 top level 5 path OracleLinux59-dev1/rootfs

This snapshot clone is instantaneous and is a copy on write snapshot.
You can test space usage like this :

# btrfs filesystem df /container
Data: total=1.01GB, used=335.17MB
System: total=4.00MB, used=4.00KB
Metadata: total=264.00MB, used=25.25MB

# lxc-clone -o OracleLinux59 -n OracleLinux59-dev2 -s
Tweaking configuration
Copying rootfs...
Create a snapshot of '/container/OracleLinux59/rootfs' in '/container/OracleLinux59-dev2/rootfs'
Updating rootfs...
'OracleLinux59-dev2' created

# btrfs filesystem df /container
Data: total=1.01GB, used=335.17MB
System: total=4.00MB, used=4.00KB
Metadata: total=264.00MB, used=25.29MB

- Adding Oracle Linux 6.5

# lxc-create -n OracleLinux65 -t oracle -- -R 6.5

lxc-create: No config file specified, using the default config /etc/lxc/default.conf
Host is OracleServer 6.5
Create configuration file /container/OracleLinux65/config
Downloading release 6.5 for x86_64
Configuring container for Oracle Linux 6.5
Added container user:oracle password:oracle
Added container user:root password:root
Container : /container/OracleLinux65/rootfs
Config    : /container/OracleLinux65/config
Network   : eth0 (veth) on virbr0
'oracle' template installed
'OracleLinux65' created

- Install an RPM in a running container

# lxc-attach -n OracleLinux59-dev1 -- yum install mysql
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package mysql.i386 0:5.0.95-3.el5 set to be updated

This connects to the container and executes # yum install mysql inside the container.

- Modify container resource usage

# lxc-cgroup -n OracleLinux59-dev1 memory.limit_in_bytes 53687091

# lxc-cgroup -n OracleLinux59-dev1 cpuset.cpus

# lxc-cgroup -n OracleLinux59-dev1 cpuset.cpus 0,1

Assigns cores 0 and 1. You can also use a range 0-2,...

# lxc-cgroup -n OracleLinux59-dev1 cpu.shares

# lxc-cgroup -n OracleLinux59-dev1 cpu.shares 100

# lxc-cgroup -n OracleLinux59-dev1 cpu.shares

# lxc-cgroup -n OracleLinux59-dev1 blkio.weight

# lxc-cgroup -n OracleLinux59-dev1 blkio.weight 20

A list of resource control parameters :

Lenz has created a Hands-on lab which you can find here :

Wednesday Dec 04, 2013

Oracle Linux containers

So I played a bit with docker yesterday (really cool) and as I mentioned, it uses lxc (linux containers) underneath the covers. To create an image based on OL6, I used febootstrap, which works fine but Dwight Engen pointed out that I should just use lxc-create since it does all the work for you.

Dwight's one of the major contributors to lxc. One of the things he did a while back, was adding support in lxc-create to understand how to create Oracle Linux images. All you have to do is provide a version number and it will figure out which yum repos to connect to on and download the required rpms and install them in a local subdirectory. This is of course superconvenient and incredibly fast. So... I played with that briefly this morning and here's the very short summary.

Start out with a standard Oracle Linux 6.5 install and uek3. Make sure to add/install lxc if it's not yet there (yum install lxc) and you're good to go.

*note - you also have to create /container for lxc - so also do mkdir /container after you install lxc, thank Tony for pointing this out.

# lxc-create -n ol65 -t oracle -- -R 6.5.

That's it. lxc-create will know this is an Oracle Linux container, using OL6.5's repository to create the container named ol65.

lxc-create automatically connects to public-yum, figures out which repos to use for 6.5, downloads all required rpms and generates the container. At the end you will see :

Configuring container for Oracle Linux 6.5
Added container user:oracle password:oracle
Added container user:root password:root
Container : /container/ol65/rootfs
Config    : /container/ol65/config
Network   : eth0 (veth) on virbr0
'oracle' template installed
'ol65' created

Now all you need to do is :

lxc-start --name ol65

And you are up and running with a new container. Very fast, very easy.

If you want an OL5.9 container (or so) just do lxc-create -n ol59 -t oracle -- -R 5.9. Done. lxc has tons of very cool features, which I will get into more later. You can use this model to import images into docker as well, instead of using febootstrap.

#  lxc-create -n ol65 -t oracle -- -R 6.5
#  tar --numeric-owner -jcp -C /container/ol65/rootfs . | \
    docker import - ol6.5
#  lxc-destroy -n ol65

Tuesday Dec 03, 2013

Oracle Linux 6.5 and Docker

I have been following the Docker project with great interest for a little while now but never got to actually try it out at all. I found a little bit of time tonight to at least try hello world.

Since docker relies on cgroups and lxc, it should be easy with uek3. We provide official support for lxc, we are in fact a big contributor to the lxc project (shout out to Dwight Engen) and the docker website says that you need to be on 3.8 for it to just work. So, OL6.5 + UEK3 seems like the perfect combination to start out with.

Here are the steps to do few very simple things:

- Install Oracle Linux 6.5 (with the default UEK3 kernel (3.8.13))

- To quickly play with docker you can just use their example

(*) if you are behind a firewall, set your HTTP_PROXY

-> If you start from a Basic Oracle Linux 6.5 installation, install lxc first. Your out-of-the-box OL should be configured to access the public-yum repositories.

# yum install lxc

-> ensure you mount the cgroups fs

# mkdir -p /cgroup ; mount none -t cgroup /cgroup

-> grab the docker binary

# wget -O docker
# chmod 755 docker

-> start the daemon

(*) again, if you are behind a firewall, set your HTTP_PROXY setting (http_proxy won't work with docker)

# ./docker -d &
-> you can verify if it works

# ./docker version
Client version: 0.7.0
Go version (client): go1.2rc5
Git commit (client): 0d078b6
Server version: 0.7.0
Git commit (server): 0d078b6
Go version (server): go1.2rc5

-> now you can try to download an example using ubuntu (we will have to get OL up there :))

# ./docker run -i -t ubuntu /bin/bash

this will go and pull in the ubuntu template and run bash inside

# ./docker run -i -t ubuntu /bin/bash
WARNING: IPv4 forwarding is disabled.

and now I have a shell inside ubuntu!

-> ok so now on to playing with OL6. Let's create and import a small OL6 image.

-> first install febootstrap so that we can create an image

# yum install febootstrap

-> now you have to point to a place where you have the repoxml file and the packages on an http server. I copied my ISO content over to a place

I will install some basic packages in the subdirectory ol6 (it will create an OL installed image - this is based on what folks did for centos so it works the same (

# febootstrap -i bash -i coreutils -i tar -i bzip2 -i gzip \
-i vim-minimal -i wget -i patch -i diffutils -i iproute -i yum ol6 ol6 http://wcoekaer-srv/ol/

# touch ol6/etc/resolv.conf
# touch ol6/sbin/init

-> tar it up and import it

# tar --numeric-owner -jcpf ol6.tar.gz -C ol6 .
# cat ol6.tar.gz | ./docker import - ol6


List the image

# ./docker images

# ./docker images
REPOSITORY          TAG                 IMAGE ID      
      CREATED             SIZE
ol6                 latest              d389ed8db59d    
      8 minutes ago       322.7 MB (virtual 322.7 MB)
ubuntu              12.04               8dbd9e392a96     
      7 months ago        128 MB (virtual 128 MB)

And now I have a docker image with ol6 that I can play with!

# ./docker run -i -t ol6 ps aux
WARNING: IPv4 forwarding is disabled.
root         1  1.0  0.0  11264   656 ?        R+   23:58   0:00 ps aux

Way more to do but this all just worked out of the box!

# ./docker run ol6 /bin/echo hello world
WARNING: IPv4 forwarding is disabled.
hello world

That's it for now. Next time, I will try to create a mysql/ol6 image and various other things.

This really shows the power of containers on Linux and Linux itself. We have all these various Linux distributions but inside lxc (or docker) you can run ubuntu, debian, gentoo, yourowncustomcrazything and it will just run, old versions of OL, newer versions of OL, all on the same host kernel.

I can run OL6.5 and create OL4, OL5, OL6 containers or docker images but I can also run any old debian or slackware images at the same time.

Wednesday Nov 27, 2013

Oracle Linux 6.5

Oracle Linux 6.5 has been uploaded to ULN and public-yum. OL6.5 includes UEKr3 based on Linux 3.8 alongside the base 2.6.32-based kernel.

The ISOs are available for download from My Oracle Support and will be available shortly(early December) from edelivery.

The ISOs are also being mirrored to public external mirror sites, one of them is my own mirror site.

Release notes are here.

Monday Nov 11, 2013

Spacewalk 2.0 provided to manage Oracle Linux systems

Oracle Linux customers have a few options to manage and provision their servers. We provide a license to use Oracle Enterprise Manager's Linux OS management, monitoring and provisioning features without additional cost for every server that has an Oracle Linux support subscription. So there is no additional pack to license and no additional per server cost, it's all included in our Basic, Premier and Systems support subscriptions. The nice thing with Oracle Enterprise Manager is that you end up with a single management product that can manage all aspects of your software stack. You have complete insight into the applications running, you have roles and responsibilities, you have third party connectors for storage or other products and it makes it very easy and convenient to correlate data and events when something happens. If you use Oracle VM as well, you end up with a complete cloud portal with selfservice, chargeback, etc...

Another, much simpler option, is just using yum. It is very easy to take a server and create directories and expose these through apache as repositories. You can have a simple yum config on each server pointing to a few specific repositories. It requires some manual effort in terms of creating directories, downloading packages and creating local repo files but it's easy to do and for many people a preferred solution.

There are also a good number of customers that just connect their servers directly to ULN or to our free update server public-yum. Just to re-iterate, our public-yum servers have all the errata and updates available for free.

Now we added another option. Many of our customers have switched from a competing Linux vendor and they had familiarity with their management tools. Switching to Oracle for support is very easy since we don't require changes to the installed servers but we also want to make sure there is a very easy and almost transparent switch for the management tools as well. While Oracle Enterprise Manager is our preferred way of managing systems, we now are offering Spacewalk 2.0 to our customers. The community project can be found here. We have made a few changes to ensure easy and complete support for Oracle Linux, tested it with public-yum, etc.. You can find the rpms in our public-yum repos at There are repositories for spacewalk server and then for each version (OL5,OL6) and architecture (x86 and x86-64) we have the client repositories as well. Spacewalk itself is only made available for OL6 x86-64.

Documentation can be found here.

I set it up myself and here are some quick steps on how you can get going in just a matter of minutes:

Spacewalk Server Installation :

1) Installing an Oracle Database

Use an existing Oracle Database or install a new Oracle Database (Standard or Enterprise Edition) [at this time use 11g, we will add support for 12c in the near future]. This database can be installed on the spacewalk server or on a separate remote server.

While Oracle XE might work to create a small sample POC, we do not support the use of Oracle XE, spacewalk repositories can become large and create a significant database workload.

Customers can use their existing database licenses, they can download the database with a trial licence from or Oracle Linux subscribers (customers) will be allowed to use the Oracle Database as a spacewalk repository as part of their Oracle Linux subscription at no additional cost.

|NOTE : spacewalk requires the database to be configured with the UTF8 characterset.
|Installation will fail if your database does not use UTF8.
|To verify if your database is configured correctly, run the following command in sqlplus:
|select value from nls_database_parameters where parameter='NLS_CHARACTERSET';
|This should return 'AL32UTF8'

2) Configure the database schema for spacewalk

Ideally, create a tablespace in the database to hold the spacewalk schema tables/data;

create tablespace spacewalk datafile '/u01/app/oracle/oradata/orcl/spacewalk.dbf'
 size 10G autoextend on;

Create the database user spacewalk (or use some other schema name) in sqlplus.

example :

 create user spacewalk identified by spacewalk;
 grant connect, resource to spacewalk;
 grant create table, create trigger, create synonym, create view, 
 alter session to spacewalk;
 grant unlimited tablespace to spacewalk;
 alter user spacewalk default tablespace spacewalk;

4) Spacewalk installation and configuration

Spacewalk server requires an Oracle Linux 6 x86-64 system. Clients can be Oracle Linux 5 or 6, both 32- and 64bit. The server is only supported on OL6/64bit.

The easiest way to get started is to do a 'Minimal' install of Oracle Linux on a server and configure the yum repository to include the spacewalk repo from public-yum.

Once you have a system with a minimal install, modify your yum repo to include the spacewalk repo.

Example :

edit /etc/yum.repos.d/public-yum-ol.repo and add the following lines at the end of the file :


Install the following pre-requisite packages on your spacewalk server :


rpm -ivh oracle-instantclient11.2-basic- 
rpm -ivh oracle-instantclient11.2-sqlplus-
The above RPMs can be found on the Oracle Technology Network website :

As the root user, configure the library path to include the Oracle Instant Client libraries :

cd /etc/
echo /usr/lib/oracle/11.2/client64/lib > oracle-instantclient11.2.conf

Install spacewalk :

# yum install spacewalk-oracle
The above yum command should download and install all required packages to run spacewalk on your local server.

| NOTE : if you did a full, desktop or workstation installation, 
| you have to remove the JTA package
| BEFORE installing spacewalk-oracle (rpm -e --nodeps jta)

Once the installation completes, simply run the spacewalk configuration tool and you are all set. (make sure to run the command with the 2 arguments)

spacewalk-setup --disconnected --external-db

Answer the questions during the setup, ensure you provide the current database user (example : spacewalk) and password (example : spacewalk) and database server hostname (the standard hostname of the server on which you have deployed the Oracle database)

At the end of the setup script, your spacewalk server should be fully configured and you can log into the web portal. Use your favorite browser to connect to the website : http://[spacewalkserverhostname]

The very first action will be to create the main admin account.

Wednesday Oct 30, 2013

Oracle Linux and Oracle VM pricing guide

A few days ago someone showed me a pricing guide from a Linux vendor and I was a bit surprised at the complexity of it. Especially when you look at larger servers (4 or 8 sockets) and when adding virtual machine use into the mix.
I think we have a very compelling and simple pricing model for both Oracle Linux and Oracle VM. Let me see if I can explain it in 1 page, not 10 pages.

This pricing information is publicly available on the Oracle store, I am using the current public list prices. Also keep in mind that this is for customers using non-oracle x86 servers. When a customer purchases an Oracle x86 server, the annual systems support includes full use (all you can eat) of Oracle Linux, Oracle VM and Oracle Solaris (no matter how many VMs you run on that server, in case you deploy guests on a hypervisor). This support level is the equivalent of premier support in the list below.

Let's start with Oracle VM (x86) :
Oracle VM support subscriptions are per physical server on which you deploy the Oracle VM Server product.

  • (1) Oracle VM Premier Limited -> 1- or 2 socket server : $599 per server per year
  • (2) Oracle VM Premier -> more than 2 socket server (4, or 8 or whatever more) : $1199 per server per year

  • The above includes the use of Oracle VM Manager and Oracle Enterprise Manager Cloud Control's Virtualization management pack (including self service cloud portal, etc..)

    24x7 support, access to bugfixes, updates and new releases. It also includes all options, live migrate, dynamic resource scheduling, high availability, dynamic power management, etc

    If you want to play with the product, or even use the product without access to support services, the product is freely downloadable from edelivery.

    Next, Oracle Linux :
    Oracle Linux support subscriptions are per physical server.
    If you plan to run Oracle Linux as a guest on Oracle VM, VMWare or Hyper-v, you only have to pay for a single subscription per system, we do not charge per guest or per number of guests. In other words, you can run any number of Oracle Linux guests per physical server and count it as just a single subscription.

  • (1) Oracle Linux Network Support -> any number of sockets per server : $119 per server per year
  • Network support does not offer support services. It provides access to the Unbreakable Linux Network and also offers full indemnification for Oracle Linux.

  • (2) Oracle Linux Basic Limited Support -> 1- or 2 socket servers : $499 per server per year
  • This subscription provides 24x7 support services, access to the Unbreakable Linux Network and the Oracle Support portal, indemnification, use of Oracle Clusterware for Linux HA and use of Oracle Enterprise Manager Cloud control for Linux OS management. It includes ocfs2 as a clustered filesystem.

  • (3) Oracle Linux Basic Support -> more than 2 socket server (4, or 8 or more) : $1199 per server per year
  • This subscription provides 24x7 support services, access to the Unbreakable Linux Network and the Oracle Support portal, indemnification, use of Oracle Clusterware for Linux HA and use of Oracle Enterprise Manager Cloud control for Linux OS management. It includes ocfs2 as a clustered filesystem

  • (4) Oracle Linux Premier Limited Support -> 1- or 2 socket servers : $1399 per server per year
  • This subscription provides 24x7 support services, access to the Unbreakable Linux Network and the Oracle Support portal, indemnification, use of Oracle Clusterware for Linux HA and use of Oracle Enterprise Manager Cloud control for Linux OS management, XFS filesystem support. It also offers Oracle Lifetime support, backporting of patches for critical customers in previous versions of package and ksplice zero-downtime updates.

  • (5) Oracle Linux Premier Support -> more than 2 socket servers : $2299 per server per year
  • This subscription provides 24x7 support services, access to the Unbreakable Linux Network and the Oracle Support portal, indemnification, use of Oracle Clusterware for Linux HA and use of Oracle Enterprise Manager Cloud control for Linux OS management, XFS filesystem support. It also offers Oracle Lifetime support, backporting of patches for critical customers in previous versions of package and ksplice zero-downtime updates.

  • (6) Freely available Oracle Linux -> any number of sockets
  • You can freely download Oracle Linux, install it on any number of servers and use it for any reason, without support, without right to use of these extra features like Oracle Clusterware or ksplice, without indemnification. However, you do have full access to all errata as well. Need support? then use options (1)..(5)

    So that's it. Count number of 2 socket boxes, more than 2 socket boxes, decide on basic or premier support level and you are done. You don't have to worry about different levels based on how many virtual instances you deploy or want to deploy. A very simple menu of choices. We offer, inclusive, Linux OS clusterware, Linux OS Management, provisioning and monitoring, cluster filesystem (ocfs), high performance filesystem (xfs), dtrace, ksplice, ofed (infiniband stack for high performance networking). No separate add-on menus.

    NOTE : socket/cpu can have any number of cores. So whether you have a 4,6,8,10 or 12 core CPU doesn't matter, we count the number of physical CPUs.

    Monday Oct 07, 2013

    Oracle Linux 5.10 channels are now published

    We just released Oracle Linux 5.10 channels on both and on the Unbreakable Linux Network. ISO's are going to be updated on edelivery in a few days. The channels are available immediately.

    As many of you know, we are now using a CDN to distribute the RPMS for public-yum globally so you should have good bandwidth everywhere to freely access the RPMs.

    Wednesday Oct 02, 2013

    Oracle Linux 6 on Microsoft Azure

    One of the great keynotes at Oracle OpenWorld last week, was from Microsoft. You can watch the replay here. I think Brad did an awesome job, very engaging and a very positive partner message. There was a lot of Oracle Linux talk in the Microsoft session, just awesome.

    We have worked closely with Microsoft to ensure that we can deploy Oracle Linux inside their Azure platform (and also just in general on Hyper-v). Part of the work is to provide templates that include Oracle products such as Oracle RDBMS and Oracle WebLogic on Oracle Linux in Azure. This is a similar concept as Oracle VM templates. You can go through the catalog on Azure, select a template and a few minutes later you end up with a complete running Virtual Machine. These templates with Oracle products are available for both Windows and Oracle Linux environments.

    Microsoft has a free trial offering which I tried out last night (with my personal account) and within a few minutes and no prior knowledge of how their environment works, I had an Oracle Linux 6 update 4 instance up and running. Logged in using ssh. They have a very easy to navigate portal. We have configured Oracle Linux out of the box with public-yum for updates. So if you need an enterprise grade Linux distribution on Azure that comes with free updates/errata and fast connectivity to the update servers, go use Oracle Linux. And the nice thing is, if you need support for some of those VM's deployed, you just pay for those VM's you want support for.

    This is also nice for ISVs that want to provide their own application solutions in Azure, they can use Oracle Linux and embed it in their VM with their app and, again, an enterprise grade solution that can be freely used without signing contracts with us, and be current with updates and errata. If the ISV then wants support, they can resell Oracle Linux subscriptions. This is a very simple, open, hassle-free solution.

    Monday Sep 02, 2013

    Oracle Linux 6 UEK3 beta

    Last week we published UEK3 beta on

    It is very easy to get started with this and play around with the new features. Just takes a few steps :

  • Install Oracle Linux 6 (preferrably the latest update) on a system or in a VM
  • Add the beta repository file in /etc/yum.repos.d
  • Enable the beta channel
  • Reboot into the new kernel
  • Add updated packages like lxc tools and dtrace
  • Oracle Linux is freely downloadable from Oracle Linux is free to use on as many systems as you want, is freely re-distributable without changing the CD/ISO content (so including our cute penguin), provides free security errata and bugfix errata updates. You only need to pay for a support subscription for those systems that you want/need support for, not for other systems. This allows our customers/users to run the exact same software on test and dev systems as well as production systems without having to maintain potentially two kinds of repositories. All systems can run the exact same software all the time.

    The free yum repository for security and bugfix errata is at This site also contains a few other repositories :

  • Playground channel (a yum repository where we publish the latest kernels as released on We take the mainline tree and build it into RPMs that can easily be installed on Oracle Linux (Oracle Linux 6 and x86_64 specifically).
  • Beta channel (a yum repository where we publish new early versions of UEK along with corresponding packages that need to be updated along with it.
  • Now, back to UEK3 beta. Just a few steps are needed to get started.

    I will assume you have already installed Oracle Linux 6 (update 4) on a system and it is configured to use public-yum as the repository.

    First download and enable the beta repository.

    # cd /etc/yum.repos.d/
    # wget
    # sed -i s/enabled=0/enabled=1/g public-yum-ol6-beta.repo 

    You don't have to do sed you can just edit (vi/emacs) the repo file and manually set it to 1 (enable). Now you can just run yum update

    # yum update

    This will install UEK3 (3.8.13-13) and it will update any relevant packages that are required to be on a later version as well. At this point you should reboot into UEK3.

    New features introduced in UEK3 are listed in our release notes. There are tons of detailed improvements in the kernel since UEK2 (3.0 based). Kernelnewbies is an awesome site that keeps a nice list of changes for each version. We will add more detail to our release notes over time but for those that want to browse through all the changes, check it out.

  • To try out dtrace, you need to install the dtrace packages. We introduced USDT in UEK3's version of dtrace, there is some information in the release notes about the changes.

    # yum install dtrace-utils

    To try out lxc, you need to install the lxc packages. lxc is capable of using Oracle VM Oracle Linux templates as a base image to create a container.

    # yum install lxc


    Tuesday Aug 27, 2013

    Single Instance/RAC Oracle VM templates update

    Superstar Saar just released a new set of Oracle VM templates. We (Oracle) just released 2 patch sets for the Oracle RDBMS - and (x86 and x86_64)

    Simultaneously, Saar updated his Oracle VM templates to include these latest patchsets as well for both architectures (x86 and x86_64).

  • with OL5
  • with OL6
  • with OL5
  • with OL6
  • These templates can be deployed on Oracle VM using the DeployCluster tool, all you need to do is create a very simple textfile with the parameters.

    All templates default to UEK2 2.6.39-400. The templates can be used to create Single Instance, Single Instance with HA (Oracle Restart) and Oracle RAC databases.

    The options vary from ASM, NFS, OCFS2 for db files, local filesystem, no DB, Clusterware only etc.

    Full stack, download, deploy. Production RDBMS code, Production Oracle Linux.

    Simple Sample script:

    # cat netconfig.ini 
    PUBGW=  # May be blank
    DNSIP=  # Starting from 2013 Templates allows multi value
    CLONE_SINGLEINSTANCE=yes  # Setup Single Instance

    and then # deploycluster -u admin -p mypassword -H localhost -M mydbvm1 -> done

    Thursday Aug 22, 2013

    A little sample snmp module for Oracle VM Server 3.2

    I was looking at snmp for a few days and decided to put together a little snmp module (extension) that would work on Oracle VM Server (3.2 and up). In 3.2 we started to include the net-snmp rpms to allow customers to monitor any given Oracle VM server with standard SNMP tools. Whether that be cacti, snmpwalk, even Oracle Enterprise Manager (snmp fetchlets) or whatever tool. The standard net-snmp installation will expose MIBs and return data pretty much exactly the same as what you would get when installing net-snmp on Oracle Linux and monitoring an Oracle Linux server.

    The little snmp module I added exposes a few extra Oracle VM specific objects. To start with I basically looked at the data you can see on the local console of the server (version, cluster state, management uuid,...). I created a custom MIB (falls in the oracle enterprise oid range ( – )) and packaged it all up in a little RPM (ovs-snmp.rpm) that can be installed in dom0.

    ovs-snmp is an extension to net-snmp. It is a dynamically loadable module that allows extra bits to be monitored in dom0 that are specific to Oracle VM. Once the RPM is installed, snmpd.conf must be updated to load the module at start of snmpd. When you restart the snmpd service, you then have access to an extra MIB.

    This extra MIB is documented in /usr/share/snmp/mibs/OVS-MIB.txt The raw oid range for the OVS extension is from – The module also contains a trap at The trap is defined around ovsAgentState (Running/Stopped) and will allow an admin to monitor the state of the Oracle VM Server agent which is a critical component of every server installed and get a notification from the snmpd.

    If you copy the OVS-MIB.txt file over to another regular server and put the file in the same directory (/usr/share/snmp/mibs) then you can use the text version instead of the raw oid numbers. For instance : is the same as : ORACLE-OVS-MIB::ovsType. This is more humanly readable.

    The following set of attributes are defined in the MIB :

    ovsType           : Oracle VM Server
    ovsVersion        : Version of Oracle VM Server installed
    ovsMaster         : Master node in serverpool?
    ovsClusterState   : Cluster configured / online?
    ovsClusterType    : NFS or Lun based
    ovsClusterStorage : the nfs mount or lun used for the server pool filesystem
    ovsManagerUUID    : UUID of the Oracle VM Manager instance
    ovsServerpoolName : serverpool name this server is a member of (or None)
    ovsServerpoolIP   : Virtual IP address of the serverpool master
    ovsAgentState     : Agent running or stopped
    ovsFreeMemory     : free memory available for Virtual Machines on this server
    ovsHostname       : hostname as known by the Oracle VM Manager instance
    vmTable           : table with an index listing all the currently running VMs
                        columns -> vmIndex, vmType

    example snmpd.conf file:

    # more /etc/snmp/snmpd.conf
    rocommunity public
    syslocation "hq"
    dlmod ovs /usr/lib64/ovs-snmp/

    Some examples :

    # snmpwalk -v 1 -c public -O e localhost ORACLE-OVS-MIB::ovsAgentState
    ORACLE-OVS-MIB::ovsAgentState.0 = STRING: Running
    # snmpwalk -v 1 -c public -O e localhost
    SNMPv2-SMI::enterprises. = STRING: "Oracle VM Server
    You can download the rpm from MOS, bug number is 17344092. At this point it's provided as-is, tech preview. Once I get some feedback on it we will consider integrating this.

    have fun

    Tuesday Aug 13, 2013

    The life of a Linux RPM (package)

    Another frequently asked question related to Oracle Linux is how versions of specific packages (RPMs) are picked.

    A Linux distribution is basically a collection of a ton of open source projects that make up the Operating System environment, with at its core the Linux kernel. Linux as a development project is about the Linux kernel specifically. There are then many (1000's) of random open source projects out there and a Linux distribution basically is an OS made up of, at its core, the kernel, and tons of those other projects packaged up. Now some packages are more critical than others, there's a small true core of packages that you will find in any Linux distribution, glibc (c library), gcc (compiler), filesystem utils, core utils, binutils, bash,... etc. A good guess would be about 150-200 packages that make up pretty much any more or less usable environment (yes you can do with far fewer but I 'm talking standard OS installs here...)

    All these projects have their own development cycle, their own maintainers/developers, their own project plans, and their own dependencies. They just kind of all move along on their own pace, they're worked on by (usually) different people and so on. So how do these 1000's of packages get into Oracle Linux?

    Oracle Linux is an exact replica of Red Hat Enterprise Linux (same packages, same source code, same versions,...)... so that's what we base our distribution on (similar to CentOS). Now Red Hat Enterprise Linux, in turn, uses the Fedora Linux distribution as its upstream baseline. Fedora is a community distribution ( that typically lives far ahead of what any one would install for a stable environment. It's a community driven, cutting edge Linux distribution. Fedora as a project has very frequent releases (like every 6 or so months). The fedora maintainers distribute maintainership of these 1000's of RPMs across a group of people and they gather newer versions of all these projects and build them for a given version of Fedora. They then stabilize this and release it. By stabilizing, I mean, they create packages, test out if all the dependencies work, if there's a build environment that's consistent and if there are bugs, and fix them of course.

    So Fedora evolves rapidly (like every 6 months a new release) and as you go from Fedora 12 to 13 to 14 etc, you see the packages of gcc and glibc and all other stuff evolve version by version, gcc 3.2 to 3.3 to 4.0 to 4.4 etc. Depending on when the fedora project starts "freezing" the package list of the next version, that's what the various versions of those 1000's of packages will be based off of. The maintainers usually will, at some point of the release cycle, take the latest "stable" version of a given project (say gcc) and check that version into the Fedora tree. What happens here is that you typically see Fedora pick up the stable versions of projects pretty regularly. It helps shake out bugs, it reaches a large (end)-userbase and it helps the Linux community that wants to be on the cutting edge by doing a lot of the packaging for them and it helps the downstream use of Fedora because many of the base/generic obvious bugs and build issues have been resolved during the Fedora test, dev and release process.

    Now, because Fedora moves so rapidly, newer versions of RHEL, and as such OL, obviously skip quite a few versions of Fedora. For instance, RHEL6 is based on Fedora 12/13, RHEL5 was based on Fedora Core6, etc... A new version of RHEL is released every few years. So Red Hat decides at a certain point in time to take a snapshot of a given Fedora release they deem stable enough at that point in time and then fork that internally into a separate repository, change the trademarks,logos, add some packages that might not be in Fedora, tests the components for a more commercial use, server use, (most of Fedora is desktop use) and then releases that as the next version of RHEL. And we then similarly follow with OL.

    An important point to make is that within a given release cycle of RHEL or OL, the version of the packages typically doesn't change, at least not the major version (usually not 2 digit versions). For the lifetime of, say, OL5, the version of, say, glibc, will remain pretty static. It will include bugfixes over the lifetime of the distribution version, security fixes and sometimes minor important things that might get backported of a newer version (albeit rare) but that's it. So you have a relatively static vesion of an OS, it improves in stability, quality, security but it doens't improve much in terms of functionality. (Most of the enhancements would probably be in the kernel.).

    This also means that a Linux distribution (RHEL, OL, CentOS,...) can skip package versions, if some external project goes from version 2.1 to 2.2 to 2.3 to 3.0 to 3.1 etc... over the lifecycle of OL5 and OL6, then OL5 might contain 2.1 and OL6 might contain 3.0 or 3.1. You won't see versions in between get picked up. Or, again, in some rare cases, if there's something really important that went into 2.3 that would be really relevant, it might have gotten backported to 2.1 as part of RHEL/OL. You cannot expect that OL5 would go from 2.1 to 2.2 to 3.0 for that given package. That's just not how things work. So if you expect major enhancements or features of some package that's newer than the version that's in the current distribution, you might (likely) have to wait until the next major release. Example : OL5 contains glibc 2.5, OL6 contains glibc 2.12. If there was something really, really important in glibc 2.8, that might have gotten backported into 2.5 and gotten into OL5, but it's unlikely. And OL5 will not start adding 2.6, or 2.7 or so into the distribution. And then the same cycle starts again with OL6, it contains glibc 2.12... but the current version of glibc upstream is 2.18, and Fedora 19 contains glibc 2.17. So the future version of RHEL7/OL7 might end up with 2.16 or 2.17, and it would have skipped over 2.13-2.15. One cannot expect that the commercial distributions backport features of future package versions into prior versions. That doesn't happen for OL, or for RHEL or CentOS or SLES etc.

    What does happen, and it's important to point this out, is the fixing of CVE's/Security vulnerabilities or critical bugfixes. Example, let's say there is a security issue found in glibc 2.17 (upstream), and this is also relevant to glibc 2.5 found in RHEL5/OL5. We obviously will end up fixing that in 2.5 (backport the security fix) and in 2.12 (OL6)... So in terms of critical fixes and security vulnerabilities found in any version of a supported Linux distribution's package, those will be found in various versions, where they matter.

    You can always track this by looking at the changelog of an RPM or look for a CVE number and you will see hits on different versions of an RPM in different versions of OL where it is relevant.

    This is pretty normal stuff, new features go into new versions of a product, like new features go into new versions of the oracle database and we will fix problems and backport changes into older versions but you will not see new features for a new versions of the database pop up in an old version of the database. It's not rocket science. A Linux Distributions is a product based on tons of small subcomponents but in the end the major release is the overarching "feature" release.

    The few exceptions (obviously there are exceptions :-) are : (1) it's possible that new packages for new products or components are introduced during the lifetime of the OS release, a new RPM can be introduced in 5.8 or 5.9 or so... (2) some of the backports of features I talked about earlier can introduce some enhancements, although rare (3) the Linux kernel is probably the most lively component in the OS where the rate of enhancements is the largest compared to anything else.

    I hope this helps.

    Of updates and errata.

    A frequently discussed topic inside Oracle and also outside with customers and partners is Oracle Linux versions and how to treat updates and support and certifications and minimum levels. Here's our take on it, from the Oracle Linux side.

    When talking about Oracle Linux and versions, there really are 3 major components :

    -1- A major new release, such as Oracle Linux 5, Oracle Linux 6,...

    A major new release is an update of the entire OS, kernel, userspace, all the 1000's of packages that make up Oracle Linux. A major release is significant in change compared to the previous version. You will see pretty much every package(RPM) updated to a whole new version, like Apache, MySQL, GCC, glibc, X-windows, gnome, etc etc... In a number of cases, the owners of the packages are not so careful about maintaining backward compatibility and introduce different style config files that could make upgrades difficult or sometimes impossible. This is one of the reasons why it's not easy to 'upgrade' from, say, Oracle Linux 5 to Oracle Linux 6. While it would be ok for a good number of RPMs, it's not guaranteed to work for everything. A config file might get overwritten or the older edited config file is not compatible with the new version...etc. So upgrading here, very often is a new/fresh install of the OS on a server. We see most customers go to new versions at time of a hardware refresh and use that as a good opportunity, and that makes total sense.

    Because a major version has signifcant changes (new kernel, with new features, new glibc,... so core components), unfortunately sometimes changes that affect or really need change not just testing (upstart from OL5 to OL6,... potentially interesting changes due to systemd in future versions), there's typically a good reason to do certification testing of userspace applications on top of the new version. The way we work here is that we build on the lowest common version of the OS we want to support and run certification against newer versions. So we build an Oracle product on Oracle Linux 5, we won't support anything on a version that's older than Oracle Linux 5, and we do extras testing and certification for any new major version, like Oracle Linux 6. This can require us to do some changes to our application (like the database) in order for us to be able to complete that certifcation (or OS bugfixes), this was a big reason as to why it took a long time to be able to consider Oracle Linux 6 certified for the database. (due to, for instance, things like upstart)

    ISV's typically will do the same thing, it takes some time for development teams to add a new major version of the OS, again, because sometimes application changes might be required or OS changes might be required, like bugfixes due to testing.

    A major OS release happens every few years, not more frequently. OL4 -> (for us 2006 since we started with update 4), OL5 -> mid 2006 OL6 -> early 2011

    -2- A minor update, really just a point in time current snapshot of a given major release with bugfixes and security updates applied (Oracle Linux 5 update 9, Oracle Linux 6 update 4...)

    A minor update, released on a regular basis (several months), is really just a snapshot in time of the major OS release. As a version is out, on a regular basis, there are bugfixes, security updates introduced into that release, so out of the 1000's of packages, a number of those will have a minor bugfix update every now and then. These updates really are focused on fixing bugs and fixing security vulnerabilities. New features are normally not introduced into packages as part of a certain major OS release. Sometimes there are some new things added but they are only introduced if it doesn't break compatibility or doesn't change the understood use of that package. Because a major OS release is only every few years and to make it easy to provide a good snapshot of fixes within the release cycle, updates are done on a regular basis. The update is literally a snapshot at a given date and then the latest version of all packages within the release are bundled together and put onto an updated media (iso).

    This makes it convenient for users in a number of ways : 1) As mentioned in the first point, sometimes we need to create OS bugfixes for an application to work on a given version of the OS, these fixes go into the OS at some point or another. It is very convenient to use the update releases to point customers to the minimum version as a starting point. 2) a very important component that gets updated regularly (bugfixes, updated device drivers/hardware support) is the kernel. So if new hardware is released, you typically need a new boot kernel to recognize newer hardware, an update release pretty much always have a newer version of the linux kernel with updated device drivers on the installation media and that's required to be able to install a given OS version on newer hardware. So it might be that you need a minimum level of Oracle Linux 5.9 to install on the latest version of a given server... but it's still "Oracle Linux 5" as a product.

    So, look at update releases as minimum patch levels of an OS release, not as a product version. Oracle Linux 5.9 is not a product version, Oracle Linux 5 is the product, update 9 just implies a recent point in time of the fixes made in the product. Too often a customer (or product team) certifies on very specific update versions, and make the mistake of implying really just that version and not that version and newer. We all stand by the fact that if a minimum version required is, let's say Oracle Linux 5 update 6, then that always implies Oracle Linux 5, starting from the snapshot point update 6 and anything newer since then, as part of Oracle Linux 5.

    There is no point in sticking to a given update version and consider that a release, there are -always- important fixes, whether it be potential crashes or security vulnerabilities released after that update and it really is not a best practice to stick at a certain point in time. We always do a lot of testing on any bugfixes update or security vulnerability and there's no breakage or introduction of incompatibilities within a given OS version. Look at Oracle Linux X update Y as running Oracle Linux X, update Y is just a point in time. ISVs should point out a starting update of what is considered supported or certified but with the understanding that it will imply anything changed from that point on for the same major version X. So if 5.6 is the base certification level, then anything post 5.6 as part of Oracle Linux 5 should be OK and it makes sense to try to remain as current as possible.

    -3- An errata update, an update of a given package, either because of a bugfix or a security vulnerability fix.

    At any point in time during the lifetime of a major OS version (OL5, OL6,...) we obviously fix bugs or address a security issue. These fixes are introduced in a continous stream of updates for each major OS. They always update the minor digits of the package version, for instance the kernel, 2.6.39-300.0.1 to 2.6.39-300.0.2 etc... They do not introduce behavior change or impact how an application runs, they make things more stable and secure. We try to not do this more often than necessary. It would be highly recommended to apply these updates soon after they are released. Most critically the kernel and glibc ones as they are under every application. Of course, with ksplice updates we make kernel updates a breeze since there's no reboot involved. And as mentioned in -2-, these errata are on a regular basis bundled into a newer snapshot of the OS.

    What's the take-away here?

    We recommend you look at an update release as a starting point, not as a product version, and we highly recommend customers and partners to be as current as possible in applying errata packages on their OS. It makes things more stable... it contains fixes... it contains patched security vulnerabilities... those all seem rather important to keep in mind.

    So often, a customer service request comes into the support organization and it's a problem that's known and fixed in an errata, a downtime that could've been prevented by being more current...

    Wednesday Aug 07, 2013

    Oracle VM templates for Database 12c both single instance and rac

    Today we made available a few new Oracle VM templates on edelivery. A set of VM templates for database 12c and another set for database 11g

    You can find more information on the otn pages here.

    A very important new feature added is the ability to deploy single instance database. In the past the database templates were focused on RAC deployments (Real Application Cluster) but because of popular demand, we also added support for Single Instance. With Single Instance you can really create a new VM with the database up and running in a matter of a few (very few) minutes, and with a very simple config file.

    Example config file for single instance :

    $ cat netconfig.ini 
    PUBGW=  # May be blank
    DNSIP=  # Starting from 2013 Templates allows multi value
    CLONE_SINGLEINSTANCE=yes  # Setup Single Instance

    That's literally it. You don't need to do anything other than run a few Oracle VM CLI or UI commands and run deploycluster and you're all set. After a few minutes, the VM will be pingable and you can run sqlplus against the database running inside the VM.

    If you use the CLI, here is a sample workflow :

  • import the template
  • - importtemplate repository name=[reponame] url=[http://myurl/template.tbz] server=[servername]
  • create vm from template
  • - clone vm name=[templatename] destType=Vm destName=[vmname] serverpool=[serverpoolname]
  • Create new vnic
  • - create vnic name=[macaddress] network=[network] (list network, will show you the various networks)
  • remove old vnics (you could rename one or alter one but to simplify I just remove the old vnics of the cloned vm and add the newly created
  • - remove vnic name=[macaddr] from vm name=[vmname]
    - show vm name=[vmname] to see the attached vnics

    And that's it, now you can use that netconfig.ini example, edit it for your environment and run deploycluster:

    On top of single instance, the templates also expose or give you the ability to easily configure and enable many of the new rdbms 12c functionality :

    - Oracle Flex Cluster and/or Flex ASM, Hub/Leaf nodes
    - Container Database with x number of pluggable databases
    - Database Express
    - ACFS filesystem
    - Oracle Restart (single instance database with HA)
    - local or shared filesystem installs, including OCFS2 and ACFS
    - Admin Managed or Policy managed database creation with serverpools
    - OS kernel updated to the latest uek 2 version 2.6.39-400

    And all of the above are simple parameters in the config files. This can be 100% automated, 100% reproducible and you don't need to know how to configure them all yourself. As always, high quality work by Saar Maoz.

    Production ready, not trial, not using a random OS, all ready to go. Production-ready virtual appliances.


    Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

    You can follow him on Twitter at @wimcoekaerts


    « March 2015