Combining Virtualization Technologies
By tpm on May 17, 2008
At the risk of sounding like a late-night TV commercial for toothpaste ... I'm occasionally asked the question "which virtualization technology should I use?" As if there's a one-size-fits-all answer. Well, admittedly it's more often a desire for a simplicity in the face of a complex world, and probably mostly because I've been confusing them with too many possibilities! Often, one technology does makes more sense than another, depending on what the customer is trying to do. However, there are many virtualization opportunities up and down the SW stack, and sometimes combining technologies can lead to more effective solutions.
Since Solaris Containers (Zones) and hardware virtualization components like hypervisors operate using different mechanisms at different levels in the SW stack, they can be used simultaneously to provide enhanced capabilities and efficiencies. One combined usage pattern we've seen treats the hardware virtualization facility almost as a static partitioning technology, doing coarse-grain resource partitioning, while exploiting the unique capabilities that hardware virtualization brings e.g. being able to run completely different operating systems, or operating system versions side by side. Obvious examples are x86/x64 machines using VMware products, or the xVM hypervisor built into OpenSolaris, or other x86 hypervisor solutions. Or customers with our CMT SPARC systems using the Logical Domains hypervisor built into the firmware.
Now, inside the Solaris or OpenSolaris domains running in these environments, customers use Solaris containers to encapsulate a set of applications as their unit of deployment, and manage fine-grain resource allocation for those containers from inside the global zone using e.g. processor pools, the fair share scheduler, and all the other resource management and resource accounting capabilities Solaris can bring to bear. Making the hypervisor configuration export a relatively static set of virtual hardware resources is a workaround for some of the more problematic aspects of two (or more) resource schedulers fighting over the same resource e.g. a multiprocessor CPU scheduler in the hypervisor at odds with an multiprocessor CPU scheduler in the guest, with the latter unaware of the presence of the former.
Of course in this picture, all the operating systems run as hypervisor guests, and thus all of them lose a bit of performance to the overheads associated with hardware virtualization technologies. And these overheads can be non-trivial for I/O intensive workloads - at least on todays x86 systems. And yet I think it's abundantly clear that the market is prepared to accept this overhead in exchange for the new management capabilities and business agility that a hypervisor-based virtualization appliance coupled with a sophisticated systems management solution brings; this observation is also at the core of our xVM Server and OpsCenter projects, as well as related efforts around both open source and proprietary hypervisors by several other companies. And as you'd expect, we're all working to reduce the overhead with hardware and software solutions, though some of the problems are thorny.
But is that the only useful virtualization technology combination?
Well, there's an interesting "flipped-over" version of this stack that the combination of Containers and a type 2 hypervisor like VirtualBox provides. Today you can run Solaris 10 or OpenSolaris as a host operating system directly "on the metal", running several Containers at the level efficiency and throughput that the container-style solutions bring, i.e. with no significant overhead involved. But what's new in the 1.6 release of VirtualBox is that you can now run other operating systems inside VirtualBox, with the VirtualBox hypervisor also running under the resource and namespace constraints of a Solaris Container i.e. hosted in a Solaris container.
As I noted in my "live" blog from Community One, VirtualBox 1.6 enables a new kind of Solaris container that can run any x86 OS that VirtualBox can support as a guest - and as people who've used VirtualBox know, that's a whole lot of different guest OSes. I was talking to a large customer two weeks ago about a workload dominated by a set of application servers. The application servers were already configured in containers; they'd done that because containers provided them with a convenient way to manage the app servers while consolidating the workload onto fewer machines. But they still have to have a number of Windows boxes to host some proprietary Windows applications that were a part of their stack that they hadn't yet found open source alternatives for. They'd tried running their entire workload on a commercial hypervisor platform, but the performance impact on their app server workload was just too high.
So what got both of us excited was the idea of putting those legacy Windows applications inside a VirtualBox container running Windows, because it meant the best of both worlds - their high performance applications will run as fast as native Solaris lets them - "on the metal" - while their low utilization legacy apps will run side-by-side in VirtualBox containers. The key point being that they would be paying for the overhead of hardware virtualization only where they needed to pay for it, not penalizing all the workloads running on the hardware. Which is also a reminder of an important consideration for assessing virtualization solutions - efficiency.
However, before anyone gets too excited, I have to point out that we're still working on integrating VirtualBox with the networking capabilities that arrive with the Crossbow project, so some of the server applications of this idea don't quite work yet. But if the purpose of the zone-hosted VirtualBox guest OS is to run as a client-side desktop, then the idea already works today. And that has an interesting impact on the Trusted Desktop.
So, let's combine VirtualBox, Containers and Trusted Extensions.
Recall that the Solaris Trusted Extensions technology that Glenn Faden has been blogging about here is built on zones. And what that means is that you can associate a label with a container, and because of VirtualBox hosted in a container, a label around an entire OS and the system resources it is consuming. All of which leads to the following screen shot that Christoph Schuba sent around internally 10 days after we announced the acquisition, and it was what Christoph was showing in one of the demos at the VirtualBox talk at Community One.
That's Vista, running in VirtualBox, hosted inside an OpenSolaris container with Trusted Extensions.
Spelling it out -- it's Vista, contained :)Technorati Tag: OpenSolaris
Technorati Tag: VirtualBox