Server Virtualization - Techniques
By pallab on Oct 13, 2008
In my previous blog I discussed share-management of the resources. Over the years, the system resources that get managed evolved, requiring the share-management to get complex. Not long ago, VT100 type dumb terminals connected to serial line concentrators was a popular technique to share a server system among end users. With demand for Desktop Graphics, Sun invented the SunRay technology allowing thousands of Graphics Desktops to be concentrated on few servers without having Graphics Cards. The basic share-management software in these technologies are similar to what a Multi-User Operating System provides.
With increasing demand for Name-Space and Configuration isolation, Sun created Zones (a.k.a Solaris Containers), an Operating System level lightweight Virtualization technology. Each Zone represents a whole system with its own Name-Space and Configuration that can be different from another Zone. Zones share the same kernel on given system. But a special type of Zone, called Branded Zones allows running Solaris 8 and Solaris 9 Operating Systems instances to be run on Solaris 10. Branded Zones created on Solaris 10 x86 Operating Environment can also run 32bit Linux OS. CPU and memory resources can be shared or dedicated. A new type of scheduler called Fair Share Scheduler helps maintain balance of CPU usage among the Zones.
From the above, it is evident that at least some system resource must get shared with an active share management in place for a setup to be termed as Virtualized. The resources are
- CPU - can be dedicated or shared among the Domains.
- Memory - is normally not shared, but in case it gets shared among Domains, it can lead to performance penalties
- IO - can also be shared or dedicated at a leaf level or an entire IO subsystem can be dedicated to a Domain.
These new sharing requirements, introduces a concept of an arbitrator, which owns all the resources of the system and allows access to these resources. This arbitrator is called the Hypervisor. Traditionally a CPU executes instructions either in user-mode or in super-user-mode. But with multiple Domains accessing the the same CPU a new mode need to be introduced - Hyper-Privileged mode. This mode is assigned to the Hypervisor. The location and exact role played by the Hypervisor in allocating, dedicating or sharing resources among Domains differentiates one Virtualization technology from another. Some Hypervisors are extensions to existing kernels while other Hypervisors are part of the System Firmware.
When a IO device is shared by multiple domains, a Proxy mechanism is normally used. The Proxy performs the task of actual IO on behalf of the Guest Domain. The Guest communicates with the Proxy over channels. The channels are allocated and maintained by the Hypervisor. The actual functionality provided by the channel is dependent on the Virtualization technology used. The Hypervisor is often also responsible for managing the IO space between the Guest Domains and the Proxy. It sometimes perfom the task of copying the data from one IO space to another, or grant access to a piece of memory belonging to a Domain or Proxy to another Domain or Proxy so that it can relive itself from doing the actual copy. This copy can sometime pose as extra overhead and often is the source of reduced Virtualized IO performance when compared to Native IO performance. New features in the PCI-Express subsystem allow a Guest Domain to directly do IO with the physical device. This advancement in PCI-Express subsystem has led the Virtualization Technology providers to come up with two new solutions viz. Direct-IO and IOV. I will go into the details of these later.
It is apparent from above, that the Guest Operating System needs to be modified to some extent to allow it to communicate with the Proxy. When the Guest Operating System needs modification or is made Virtualization-aware, it is called Para-Virtualization. But it also possible to emulate an entire computer system and present it to the Guest Operating System. At minimum, if the IO susbsytem is emulated, then it is possible to run a Guest Domain with un-modified Native Operating System. This is often termed as Full Virtualization. Because this technique involves lot of emulation, its performance often lags that of Para-Virtualized domains. Performance acceleration requires help from the hardware and is termed as Hardware Assisted Virtualization.
In this new Virtualization space, Sun offers two solutions - xVM Server for x86 Platform and LDOMs for the SPARC Platform.
In the next section, I will write about LDOMs.