Wednesday Jun 26, 2013

Linux-Containers — Part 1: Overview

Containers by Phil Parker, on Flickr
"Containers" by Phil Parker (CC BY 2.0).

Linux Containers (LXC) provide a means to isolate individual services or applications as well as of a complete Linux operating system from other services running on the same host. To accomplish this, each container gets its own directory structure, network devices, IP addresses and process table. The processes running in other containers or the host system are not visible from inside a container. Additionally, Linux Containers allow for fine granular control of resources like RAM, CPU or disk I/O.

Generally speaking, Linux Containers use a completely different approach than "classicial" virtualization technologies like KVM or Xen (on which Oracle VM Server for x86 is based on). An application running inside a container will be executed directly on the operating system kernel of the host system, shielded from all other running processes in a sandbox-like environment. This allows a very direct and fair distribution of CPU and I/O-resources. Linux containers can offer the best possible performance and several possibilities for managing and sharing the resources available.

Similar to Containers (or Zones) on Oracle Solaris or FreeBSD jails, the same kernel version runs on the host as well as in the containers; it is not possible to run different Linux kernel versions or other operating systems like Microsoft Windows or Oracle Solaris for x86 inside a container. However, it is possible to run different Linux distribution versions (e.g. Fedora Linux in a container on top of an Oracle Linux host), provided it supports the version of the Linux kernel that runs on the host. This approach has one caveat, though - if any of the containers causes a kernel crash, it will bring down all other containers (and the host system) as well.

For example, Oracle's Unbreakable Enterprise Kernel Release 2 (2.6.39) is supported for both Oracle Linux 5 and 6. This makes it possible to run Oracle Linux 5 and 6 container instances on top of an Oracle Linux 6 system. Since Linux Containers are fully implemented on the OS level (the Linux kernel), they can be easily combined with other virtualization technologies. It's certainly possible to set up Linux containers within a virtualized Linux instance that runs inside Oracle VM Server for Oracle VM Virtualbox.

Some use cases for Linux Containers include:

  • Consolidation of multiple separate Linux systems on one server: instances of Linux systems that are not performance-critical or only see sporadic use (e.g. a fax or print server or intranet services) do not necessarily need a dedicated server for their operations. These can easily be consolidated to run inside containers on a single server, to preserve energy and rack space.
  • Running multiple instances of an application in parallel, e.g. for different users or customers. Each user receives his "own" application instance, with a defined level of service/performance. This prevents that one user's application could hog the entire system and ensures, that each user only has access to his own data set. It also helps to save main memory — if multiple instances of a same process are running, the Linux kernel can share memory pages that are identical and unchanged across all application instances. This also applies to shared libraries that applications may use, they are generally held in memory once and mapped to multiple processes.
  • Quickly creating sandbox environments for development and testing purposes: containers that have been created and configured once can be archived as templates and can be duplicated (cloned) instantly on demand. After finishing the activity, the clone can safely be discarded. This allows to provide repeatable software builds and test environments, because the system will always be reset to its initial state for each run. Linux Containers also boot significantly faster than "classic" virtual machines, which can save a lot of time when running frequent build or test runs on applications.
  • Safe execution of an individual application: if an application running inside a container has been compromised because of a security vulnerability, the host system and other containers remain unaffected. The potential damage can be minimized, analyzed and resolved directly from the host system.

Note: Linux Containers on Oracle Linux 6 with the Unbreakable Enterprise Kernel Release 2 (2.6.39) are still marked as Technology Preview - their use is only recommended for testing and evaluation purposes.

The Open-Source project "Linux Containers" (LXC) is driving the development of the technology behind this, which is based on the "Control Groups" (CGroups) and "Name Spaces" functionality of the Linux kernel. Oracle is actively involved in the Linux Containers development and contributes patches to the upstream LXC code base.

Control Groups provide means to manage and monitor the allocation of resources for individual processes or process groups. Among other things, you can restrict the maximum amount of memory, CPU cycles as well as the disk and network throughput (in MB/s or IOP/s) that are available for an application.

Name Spaces help to isolate process groups from each other, e.g. the visibility of other running processes or the exclusive access to a network device. It's also possible to restrict a process group's access and visibility of the entire file system hierarchy (similar to a classic "chroot" environment).

CGroups and Name Spaces provide the foundation on which Linux containers are based on, but they can actually be used independently as well.

A more detailed description of how Linux Containers can be created and managed on Oracle Linux will be explained in the second part of this article.

Additional links related to Linux Containers:

- Lenz Grimmer

Follow me on:
Personal Blog | Facebook | Twitter | Linux Blog |

Friday Jun 21, 2013

How Oracle Solaris Makes the Database Scream

Few things are as satisfying as a screaming burnout (image removed from blog). When Oracle Database engineers team up with Oracle Solaris engineers, they do a lot of them. Here are a few of the reasons why.

Article: How the OS Makes the Database Fast - Oracle Solaris

For applications that rely on Oracle Database, a high-performance operating system translates into faster transactions, better scalability to support more users, and the ability to support larger capacity databases. When deployed in virtualized environments, multiple Oracle Database servers can be consolidated on the same physical server. Ginny Henningsen describes what Oracle Solaris does to make the Oracle database run faster.

Video Interview: Why Is The OS Still Relevant?

In a world of increasing virtualization and growing interest in cloud services, why is the OS still relevant? Michael Palmeter, senior director of Oracle Solaris, explains why it's not only relevant, but essential for data centers that care about performance.

Interview: An Engineer's Perspective: Why the OS Is Still Relevant

Sysadmins are handling hundreds or perhaps thousands of VM's. What is it about Solaris that makes it such a good platform for managing those VM's? Liane Praza, senior engineer in the Solaris core engineering group provides an engineer's perspective.

Interview in the Lab: How to Get the Performance Promised by Oracle's T5 SPARC Chips

If you want your applications to run on the new SPARC T5/M5 chips, how do you make sure they use all that new performance? Don Kretsch, Senior Director of Engineering, explains.

Interview: Why Oracle Database Engineering Uses Oracle Solaris Studio

The design priorities for Oracle Solaris Studio are performance, observability, and productivity. Why this is good for ISV's and developers, and why it's so important to the Oracle database engineering team. Taped in Oct 2012.

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Thursday Jun 20, 2013

Hands-On Labs + Proctors = Genius

If Albert Einstein (image removed from blog) had attended OTN's virtual sysadmin days, he wouldn't have gotten so old figuring out his Theory of Relativity. Thanks to the relentless advance of technology, you can outsmart Einstein from the comfort of your own office. See below.

OTN Virtual Sysadmin Day - July 2013

It's free - register here

We held our first ever virtual sysadmin day for North America on January 15 of this year. Almost 600 sysadmins attended and over 80% of them remained online for the duration of the event. Which means they found it a good use of their time. If you missed that one, we're doing another one in July. Oddly enough, we chose the same date and time: the 15th at 9:00 am PT. Which is at exact same spot of the Earth's rotation, but on the other side of the sun and closer to our upcoming collision with Adromeda.

That galactic fender-bender aside, we have updated some of the hands-on labs about Oracle Solaris and Oracle Linux that we presented at our in-person sysadmin days, and we added three new labs about Oracle VM:

  • Deploying Infrastructure as a Service
  • How to Virtualize and Deploy Oracle Applications Using Oracle VM Templates
  • Creating an x86 Enterprise Cloud Infrastructure

Details here.

The event is free, but you do need to register. And there's a little homework involved. Nothing too complicated. We just expect you to have VirtualBox installed and the proper images already imported before we begin class. You'll see the instructions after you register.

When was that again?

Monday, July 15 at 9:00 am Pacific Time. (Time converter here.)

Register here

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Friday Jun 07, 2013

How to Get Started Using DTrace on Oracle Linux

I. hate. slow. code. (Image removed from blog.)

We all hate slow code. Bunch of princesses is what we've become. During the American Civil War, they had to deliver their text messages by horseback! It took weeks! And half the time, they got blown off their horse by a cannonball to the neck!

Today? Today we have to have our stuff back in milliseconds, or we start tweeting about it. So, if you're developing or deploying applications, how do you keep them performing at the speed to which we have become accustomed? DTrace, of course.

"But I'm a Linux guy," you say. "I don't DO Oracle Solaris."

That's fine. The folks at Oracle Solaris are not only wicked smart, they are generous. Now you can use DTrace on Oracle Linux. Let me point out, by the way, that DTrace is just as useful for sysadmins as it is for developers. In this video, taken a couple of years ago, Brendan Gregg explains how sysadmins can make their deployed applications run faster even after the developers who wrote them pushed back the last bits of their code:

Video Interview: How to Improve the Performance of Deployed Applications Using DTrace

Brendan Gregg describes the best ways for sysadmins to tune deployed applications to get more performance out of them in their particular computing environment.

Bonus: More info about Brendan Gregg plus links to his personal and professional blogs.

If you'd like to try DTrace on Oracle Linux, here are some resources to get you started.

What DTrace Probes Are Available on Oracle Linux?

If you are running Oracle Linux 6 with the DTrace-enabled Unbreakable Enterprise Kernel Release 2 (2.6.39), you can run this command to list all the DTrace probes available on your system:

dtrace -l

If you are not running that version of Oracle Linux, you can download it from the ol6_x86_64_Dtrace_latest channel on the Unbreakable Linux Network (ULN). For more info about installing and configuring DTrace, see the DTrace chapter in the Oracle Linux Administrator's Solutions Guide for Release 6.

For each probe listed by dtrace -l, the output includes a name, the portion of the program where it resides, and the Oracle Linux kernel module that does the probing. Once you have that, go to Chapter 11 of the DTrace Guide to find out what each probe does.

Article: How to Get Started Using DTrace on Oracle Linux

DTrace is a powerful tool, and it can do some amazing things. But it's not that difficult to get started doing simple things. You can build up from there. In this article, Richard Friedman gives you a high-level overview of DTrace and its major components:providers, modules, functions, and probes. He explains how you can use either one-liner commands on the command line, or write more complex instructions in scripts, using the D language. He provides simple examples for each. It's a great way to get your feet wet.

Article: How to Get Started Using DTrace on Oracle Linux
Bonus: Brendan Gregg's one liners for DTrace (some of the existing DTrace one-liners will require modification to work on Oracle Linux).

The DTrace Book

You can get all the info you need about DTrace through the Dynamic Tracing in Oracle Solaris, Mac OS X, and FreeBSD, by Brendan Gregg and Jim Mauro. Of course, you can also buy your own paper or electronic copy through any of the major retailers. (We're working on getting a good discount for the book, but you'll have to subscribe to the OTN Systems Community Newsletter to find out about it.)

Bonus:How the DTrace book got done, by Deirdre Straughan

DTrace Forums

Lots of developers and sysadmins are using Dtrace and posting their questions and tips on the DTrace Forum. Here's an example of one conversation:

Q: Unexpected output of dtrace script
m1436 wrote a dtrace script to monitor the bytes returned by the read() system call to the user programme, but was getting strange results. He includes the dtrace script and the strange output.

A: kvh responds, explaining that the problem m1436 encountered is the result of a common misconception about copyin(). "It is intended to be used to copy content of userspace memory into a scratch buffer so that it can be accessed directly from within kernel space (where the DTrace core executes). That said, it is often interpreted as somehow being equivalent to malloc() whereas in reality it actually works like alloca() instead. So, what you are seeing is basically the artifact of the scratch buffer being overwritten with other data. ... in order for this to work, you should do things a bit differently.

The DTrace forum always has great discussions. Let me know if you find any that are worthy of highlighting. And good luck!

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel


Rick Ramsey
Kemer Thomson
and members of the OTN community


« June 2013 »
Blogs We Like