Tuesday Apr 22, 2014

OTN Garage: Which Type of Virtualization Should I Use?

Rick posted a nice article on his OTN Garage that trys to give some guidelines to the question "Which Type of Virtualization Should I Use?".

[Read More]

Wednesday Oct 23, 2013

How to do more with Oracle VM Templates

On Oracle's Virtualization Blog you can find an interesting post regarding working and using Oracle VM Templates, title is "Opening The Oracle VM Templates Blackbox".

[Read More]

Friday Aug 09, 2013

Launch Webcast: Latest Engineered System for Enterprise Virtualization

Install and Deploy Your Virtual Infrastructure in Just Hours

Join us and learn about our latest engineered system. It provides a converged infrastructure powered by Oracle VM server virtualization and Oracle Virtual Networking, dramatically enhancing data center operations.

Time and Date: Tuesday, August 13, 2013 10:00 a.m. PDT ( 19:00 CEST)

For more details and registration click here.

Tuesday Jul 30, 2013

IDC White Paper: Oracle Virtual Networking Delivering Fabric Virtualization and Software Defined Networks

In July IDC has published a White Paper that takes a close look at the fabric virtualization technology and profiles the Oracle Virtual Networking line of products.

The paper is titled "Oracle Virtual Networking Delivering Fabric Virtualization and Software Defined Networks"

[Read More]

Friday Mar 08, 2013

ODA: Correlation table on HW type, Virtualization, OAK- and DB-Version

Find below a table which shows the correlation between HW Type, OAK Version, Virtualization and Clusterware, ASM&DB Version.

HW Type

OAK Version


Clusterware, ASM & DB



Yes (3.11 OVM)  & *



Yes (3.11 OVM) & *



No  &



No &



No &




Tuesday Aug 21, 2012

Virtualization @ Oracle (Part 9: Ops Center as a Management Tool for Virtualization )

After overviews of all the different Oracle virtualization technologies available to customers, in our closing article we come to an interesting guide for simplifying life. One GUI to manage them all ...

Not quite really all of them though, because Oracle VM Virtual Box and VDI - described in the previous two episodes - have their own management tools. So it’s about centralized management for Oracle server virtualization technologies.

Ops Center as a Management Tool for Virtualization

For datacenter management Oracle offers Oracle Enterprise Manager, aka 'OEM'. OEM is actually a product suite: OEM 12c Cloud Control and OEM 12c Ops Center are the two products within the suite which are relevant to management of physical and virtual environments.

To many people, the term 'Oracle Enterprise Manager' is synonymous with the OEM Cloud Control product, or its predecessor OEM Grid Control. However, since 2010 there has also existed OEM Ops Center. Cloud Control and Ops Center are still two different products with separate installations. If preferred, they can be coupled together for data exchange. Looking forward, you will likely have just the one product to install; Oracle Enterprise Manger which should offer the combined feature set of both.

So what is Ops Center?

Ops Center offers complete infrastructure management with a focus on Oracle hardware (servers, switches, storage appliances) and Oracle operating systems, plus non-Oracle Linux variants that are supported on Oracle servers.

“Focus on” means not “limited to”, but this is a different story.

Using Ops Center, it's possible to manage the complete life cycle of a system:

  • Install / check the firmware revisions
  • Configure built-in RAID controllers and BIOS settings for x86 systems
  • Install an operating system
  • Install software applications
  • Keep the installation updated via patch or update management.

Virtualization Management

Instead of installing Solaris or Linux as the base operating system on the server, you can (for example) install Oracle VM Server for x86 on the bare metal system using Ops Center. From there, you can then deploy virtual guests. Ops Center is used to deploy the hypervisor, then allows integration with Oracle VM Manager so that both hypervisor and virtual guests can be managed from the same console.

Another example. If the server is running Solaris, use Ops Center to deploy and manage Solaris Zones on top of it. With the appropriate underlying hardware you can also deploy and manage Oracle VM Server for SPARC (aka LDOMs).

Ops Center has an effective and easy to use interface. It provides an intuitive graphical overview of the physical and virtual environments you are running, and makes them easy to manage and maintain.

Sample screenshot:

On the left hand side of this first screenshot, I have selected the object ('test-paas-zone-03') that I’m interested in doing something with:

You see a nice grouping:

't6320-paas-045-sc' on the top is the handle I select for managing and monitoring the hardware on which my software stack is running.

The underlying hardware is, as the name suggests, a T-Series server. T-Series servers do have a built-in hypervisor. This is represented by the next icon: 't6320-paas-045' . This was the object I selected during setup of the environment.

Relating to this system, the initial Ops Center job I ran from the GUI was effectively telling Ops Center to 'Build me a Control Domain on this hypervisor'.

Afterwards other jobs for defining and installing guest LDOMs were run. Logical domains 'Test-paas-ldom-03' and 'test-paas-ldom-04' were created. Both have an anchor point for resource management and possible live migration (the icons 'test-paas-03' and 'test-paas-04').

And one of my guest LDOMs, test-paas-ldom-04, is running a non global zone: test-paas-zone-03. Yes – this was also built with the Ops Center GUI.

Selecting the test-paas-zone-03 zone object , I can inspect the zone's configuration details in the next screenshot. On the right hand side are the possible actions I may perform on this zone .

“Edit Configuration” gives me the chance to change resource definitions for this zone:

Server Pools

One may argue that in general, when using Solaris command-line you have more flexibility, for example while setting up your Control domains.

However, using Ops Center offers an additional benefit. You can group servers of compatible architecture together, assign them shared storage where virtualized guests may run, define the type of virtualization technology to be used (Oracle VM server for SPARC, Oracle VM Server for x86, Oracle Solaris Zones – SPARC, Oracle Solaris Zones – x86) and then define a server pool.

For the server pool itself you define characteristics such as:

  • Placement Policy: how to determine on which physical server (for example) a new virtual guest is to be deployed
  • Auto Balancing Policy: do you like Ops Center to watch if the server pool is for example equally utilized or is there an optimization via migration of a guest possible? Do you want Ops Center to perform this migration job without further invention or do you want to be alerted?
  • Automatic Recovery: do you like Ops Center to power off a failing physical server and migrate the virtual guests to other physical servers which are part of the pool?

Hopefully this gives you some valuable insight into the virtualization management features available to you from within Ops Center. And there is much more to discover in this tool.

Cloud Support

As the full product name 'Oracle Enterprise Manager 12c Ops Center' implies, cloud features are offered at the infrastructure level.

Ops Center allows users to build the managed assets into Virtual Data Centers, or vDCs.

A vDC provides a full stack of coordinated resources for cloud computing, including systems, network infrastructure, storage, virtualization hosts and a self-contained user and permission model. Ops Center users can leverage this functionality in two ways: by logging directly into the Ops Center UI, or by using the programmatic API and command line tools for Infrastructure as a Service (IaaS). These features enable Ops Center to greatly speed the deployment and management process for cloud-based infrastructure.

Additional benefits

Besides the various benefits a graphical management tool offers, please be aware of the another important advantage Oracle provides to its customers;

Customers who maintain an Oracle Premier Support for Systems or Oracle Premier Support for Operating Systems agreement can download and use Oracle Enterprise Manager Ops Center to manage their covered Oracle systems at no additional cost with support included. This Ops Center Everywhere program has been in place since November 2011.

Further Reading

This series of 9 is now completed with the above article and had the following entries:

  • December 2011: Introduction to Virtualization (Matthias Pfützner)
  • January 2012: Oracle VM Server for SPARC (Matthias Pfützner)
  • February 2012: Oracle VM Server for x86 (Matthias Pfützner)
  • March 2012: Oracle Solaris Zones and Linux Containers (Detlef Drewanz)
  • April 2012: Resource Management (Detlef Drewanz)
  • May 2012: Network Virtualization and Network Resource Management (Detlef Drewanz)
  • June 2012: Oracle VM VirtualBox (Detlef Drewanz)
  • July 2012: Oracle Virtual Desktop Infrastructure (VDI) (Matthias Pfützner)
  • August 2012: Ops Center as Management Tool for Virtualization (Elke Freymann)

If you have questions, feel free to contact me at Elke Freyman, or contact the maintainer of this series, Uwe Strahlendorf.

Read more:

<<< Part 8: Oracle Virtual Desktop Infrastructure - OVDI

Tuesday Jul 31, 2012

Virtualization @ Oracle (Part 8: Oracle Virtual Desktop Infrastructure - OVDI)

Having finished the overview of individual virtualization technologies from Oracle using Hypervisors and Operating System features, now it's time to look at the desktop product, also known as

Oracle Virtual Desktop Infrastructure (OVDI)

Before diving deeper into Oracle's offering, lets define, what VDI is. Therefore lets again quote Wikipedia (http://en.wikipedia.org/wiki/Virtual_Desktop_Infrastructure):

“Desktop virtualization involves encapsulating and delivering either access to an entire information system environment or the environment itself to a remote client device. The client device may use an entirely different hardware architecture from that used by the projected desktop environment, and may also be based upon an entirely different operating system. The desktop virtualization model allows the use of virtual machines to let multiple network subscribers maintain individualized desktops on a single, centrally located computer or server. The central machine may operate at a residence, business, or data center. Users may be geographically scattered, but all must be connected to the central machine by a local area network, a wide area network, or the public Internet.”

Or short:

“Virtual desktop infrastructure (VDI) is the practice of hosting a desktop operating system within a virtual machine (VM) running on a hosted, centralized or remote server.”

To simplify the understanding, lets put this into a picture:

In order to achieve such an environment or such an infrastructure, multiple pieces are needed. End user devices to access the centralized hosted desktops, tools to transport (encode) the desktop via the network to the end user devices, and tools to virtualize the desktop operating systems so that they can be encoded and transported via the network. Oracle has all the pieces, some of them have already been touched on in former articles. But users want choice, so even the use of non-Oracle products is possible in setting up an OVDI.

Before we describe these parts in more detail, here's an overview picture taken from the documentation (http://docs.oracle.com/cd/E26217_01/E35769/html/architecture-intro.html):

Lets start with the end user devices:

Some fifteen years ago Sun created the Sun Ray (http://en.wikipedia.org/wiki/Sun_Ray), which now is in its third generation, and is also available as a software only product (the OVDC, the Oracle Virtual Desktop Client), which can be run on the iPad, MacOS, Windows and Linux. With this, a broad variety of end user device solutions are possible, ranging from Zero Admin Devices (the physical Sun Ray itself) over classical desktop systems running software to access the VDI desktop, up to mobile devices like the iPad, allowing instant access to a user's desktop anywhere on this planet.

Moving on from the end user device closer into the datacenter, lets look at the network part:

As the above mentioned Desktop to Network Virtualization needs an encoding tool on the server side, the Sun Ray Server Software is the corresponding counterpart for these client devices. The protocol used is called Appliance Link Protocal (ALP, which is a Sun Ray specific protocol), and is particularly well suited for wide area networks, so massively centralized infrastructures can be build in large global enterprises.

Moving again closer to the “desktop run in a datacenter”, lets look at the virtualization components:

Somehow the desktop operating systems per se need to be virtualized. This can be done via a variety of so-called desktop providers/connectors, one of which is the aforementioned VirtualBox product. Providers/connectors for Citrix XenDesktop, Vmware vSphere, Vmware View, or Microsoft Hyper-V Server 2008 or Microsoft Remote Desktop Services 2008 also exist.

Moving away from the underlying enabling technologies, lets look at the management part:

As such VDI setups are accessed by thousands of users and host thousands of individual desktops, access management to all these possible mappings between users and their sessions/desktops needs to be handled. This is done by the desktop broker, which is an integral part of Oracle VDI and allows connection to an enterprise database containing such information, like Active Directory or LDAP, and stores its internal information in a MySQL database. This allows for easy management and migration of already existing corporate desktop infrastructures into an Oracle VDI environment.

Then there is the storage space:

Here the setup and provisioning of new user desktops needs to be managed as all of these are stored in the desktop operating system disk images. This also uses other Oracle technologies to speed up the process, like cloning of existing desktop “golden images” via storage subsystem methods. The cheapest VDI solution then would be a single x86 server with a bunch of internal disks. But also setups of many x86 servers with external storage like the Oracle S7000 series is possible.


With Oracle Virtual Desktop Infrastructure, there is a complete VDI solution from Oracle, making intelligent reuse of already existing technologies.

Further Reading






This series already had the following articles:

  • December 2011: Introduction to Virtualization (Matthias Pfützner)
  • January 2012: Oracle VM Server for SPARC (Matthias Pfützner)
  • February 2012: Oracle VM Server for x86 (Matthias Pfützner)
  • March 2012: Oracle Solaris Zones and Linux Containers (Detlef Drewanz)
  • April 2012: Resource Management (Detlef Drewanz)
  • May 2012: Network Virtualization and Network Resource Management (Detlef Drewanz)
  • June 2012: Oracle VM VirtualBox (Detlef Drewanz)

The series will continue as follows (tentative):

  • August 2012: OpsCenter as Management Tool for Virtualization (Matthias Pfützner)

If you have questions, feel free to contact: Uwe Strahlendorf

Read more:

<<< Part 7: Oracle VM VirtualBox >>> Part 9: Ops Center as a Management Tool for Virtualization

Friday Jun 22, 2012

Virtualization @ Oracle (Part 7: Oracle VM VirtualBox)

After discussing several aspects of resource management and network virtualization in the previous articles, let us go back now to another Oracle Virtualization Product – a Type 2 Hypervisor,

Oracle VM VirtualBox

Type 2 Hypervisor, what does that mean again… ? Remember the picture we used in the first articles of this series to position the different ways of virtualization:

To run a type 2 Hypervisor, you need an Operating System running underneath. For Oracle VM VirtualBox it has to be an already running Host Operating System on your x86-based desktop, laptop or server. Install your Hypervisor Software Oracle VM VirtualBox on top of that. Then you can simultaneously run multiple operating systems inside Oracle VM VirtualBox (multiple virtual machines). The Host OS can be Windows, MacOS, Linux or Solaris. Many OS are supported as guests (see https://www.virtualbox.org/manual/ch03.html#idp18337920 for the complete list). For example, run Windows and Linux as guests on your Mac, Windows Server 2008 and Solaris as a guest on your Linux server, run Linux as a guest on your Windows PC, and so on, all alongside your existing applications. You can install and run as many virtual machines as you like – the only practical limits are disk space and memory.

The screenshot shows VirtualBox, installed on a Microsoft Windows 7 System, running Windows XP and Ubuntu as a guest.

Oracle VM VirtualBox is separated into 3 parts.

  • The first part is the base software package that is available for each supported host operating system.
  • The guest additions can be installed into the installed guest and add shared folder support, seamless window integration and 3D support.
  • Additionally VirtualBox extension packs can be installed to extend VirtualBox functionality. For the Oracle provided extension pack this is the support for USB 2.0, the VirtualBox Remote Desktop Protocol (VRDP) and the PXE boot ROM.

Data can be transferred to and from the guests through external iSCSI-storage, with shared folders from the Host or via network services. There are different options to connect VirtualBox guests with the Host or the outside world:

  • NAT Networking: with NAT address translation for clients on a VirtualBox-private LAN
  • Bridged Networking: to bridge guests to the Host network and makes them full network citizens
  • Internal Networking: to bind guests to an isolated network, which is independent and separated from the Host
  • Host-only Networking: as Hybrid between Bridged and Internal Networking to connect the isolated private network with the Host

There are different interfaces to use VirtualBox like a Command-line interface with “VBoxManage”, a Python interface “vboxshell.py” and a Web-services API “phpvirtualbox”. This enables a very flexible use and administration of Oracle VM VirtualBox and if you need a configuration option, that you can not find in the GUI, check the VirtualBox Manual and the various additional “VBoxManage” options.

With its graphical user interface, the way to install and manage guests or the capabilities to take and manage snapshots, VirtualBox is seen as very easy to use and gained rapidly a high popularity.
As an example Oracle uses VirtualBox to pre-build Developer-VMs. These pre-build VMs can be used by developers to learn Oracle Technologies without having the hassle of installing an OS or Application Software. Just download the VM, import it into your VirtualBox installation and you are done. Ready to start your development work. (see http://www.oracle.com/technetwork/community/developer-vm/index.html for several pre-built Developer VMs). Oracle VM VirtualBox is also used in Oracle’s Virtual Desktop Infrastructure (see our next planned article).


Having all this in mind, it is fair to say Oracle VM VirtualBox is the coolest, easiest to use and most feature-rich personal desktop virtualization product. It can be used for various purposes like desktop virtualization, training, testing, developing and personal lab.

We now close this article on Oracle VM VirtualBox and hope we've kept you eager to read the ones coming up in the following newsletters.

Further Reading


This series already had the following articles:

  • December 2011: Introduction to Virtualization (Matthias Pfützner)
  • January 2012: Oracle VM Server for SPARC (Matthias Pfützner)
  • February 2012: Oracle VM Server for x86 (Matthias Pfützner)
  • March 2012: Oracle Solaris Zones and Linux Containers (Detlef Drewanz)
  • April 2012: Resource Management (Detlef Drewanz)
  • May 2012: Network Virtualization and Network Resource Management (Detlef Drewanz)

The series will continue as follows (tentative):

  • July 2012: Oracle Virtual Desktop Infrastructure (VDI) (Matthias Pfützner)
  • August 2012: OpsCenter as Management Tool for Virtualization (Matthias Pfützner)

If you have questions, feel free to contact me at: Detlef Drewanz

Read more:

<<< Part 6: Network Virtualization and Network Resource Management >>>> Part  8: Oracle Virtual Desktop Infrastructure - OVDI

Monday May 21, 2012

Virtualization @ Oracle (Part 6: Network Virtualization and Network Resource Management)

After discussing Oracle VM, OS-Virtualization and aspects of resource management in the previous articles, we cover in this article a special area of resource management and virtualization of resources,

Network Virtualization and Network Resource Management

The network is a special shared resource that glues all the VMs, zones and systems together. The network is their communication channel with the world. Thus the network is a very important layer of the virtualization stack.

Network virtualization is categorized as external or internal.

  • External network virtualization combines many networks, switches, network ports, virtual ports or virtual interfaces into virtual units or networks. Those virtual units are called virtual LANs or just VLANs. VLANs are created by using VLAN tags to group networks from different ports, switches and physical networks together into one common virtual network. A VLAN tag is an identifier that is sent together with network packets to identify which packets belong to a virtual network. A virtual network can also be called a broadcast domain. That is a group of network participants that all receive a network broadcast.

  • Internal network virtualization is the virtualization of a network stack, network interfaces or other networking functionality within one system. This virtualization functionality is provided by the Host-OS or the hypervisor. Internal network virtualization enables the shared usage of a limited number of network ports by many VMs, zones or containers. All of the virtualized environments need their “own” network interfaces and with network virtualization some physical network interfaces (PNIC) can be “divided” into many virtual network interfaces (VNIC). This is one of the basic functionalities of internal network virtualization.

Because of the high usage of the shared resource network by many consumers like processes, VMs, zones or containers, network resource management is very important in conjunction with network virtualization. This resource management helps to deliver powerful and stable network connections to the virtualized environments. The available network bandwidth can now be better spread between multiple virtualized environments to meet their service level agreements. Extensive usage of network virtualization should only be considered together with well-implemented network resource management.

Using hypervisor-based
virtualization and Solaris Zones together with network
virtualization and resource
management enables a whole new range of new capabilities to create network-based
architectures. The picture on the right shows one example, where physical systems and network components have been replaced by Solaris Zones and virtual switches.

In this article we concentrate on the functionalities and side effects of network virtualization and resource management in conjunction with hypervisors, containers and zones in one system. Here we concentrate on internal network virtualization.

Features of Internal Network Virtualization

The following base features are common across various type of hypervisors or zone technologies, however specific implementations differ.

  • Virtual network interfaces are needed to share a small number of physical network ports (PNIC) by a larger number of VMs or zones - let’s call them consumers. Every consumer requires its own network interface that it can use as if it would be a physical port. It is the task of the hypervisor, the host operating system or the Global Zone to provide this network interface. The administrator can decide if this network interface is mapped to a dedicated physical port or if it is a virtual interface (VNIC) and then assigned to a shared physical port. In the latter case the physical port is shared by many virtual interfaces and resource management features are useful to limit the bandwidth each virtual interface can make use of. The picture on the right shows an example of how VNICs are built in Oracle Solaris on top of physical interfaces and then are used by Solaris Zones. In this example we also use bandwidth limitations assigned to VNICs.

  • Virtual network switches connect multiple virtual network interfaces that are created on one physical interface. This makes it possible for VNICs on one physical port to communicate with each other, but also to share the physical interface. The feature names for this - used by various products - differ, but the functionality is similar. In Oracle VM for x86 this is called a 'bridge', which is automatically created if a virtual interface is created on a physical port. For Oracle VM for SPARC a virtual switch has to be created by the admin in the service domain, where the network interfaces of the guest domains connect to. The pictures on the right show the examples for Oracle VM (x86 top, SPARC bottom).

    Oracle Solaris creates a switch above the physical interface, if the first VNIC is created. Oracle VM VirtualBox creates virtual PCI Ethernet cards and assigns them to VMs as network interfaces. There are different ways as to how these interfaces communicate with the host operating system or the outside world (NAT, Bridged Networking, Internal Networking, Host-only networking)

  • A special implementation of a virtual network switch that is only available in Oracle Solaris 11 is an 'etherstub'. This is a special type of data link that can be used instead of a physical NIC to create VNICs and the virtual switches that connect them. With etherstubs, complex network architectures or just network-in-a-box setups can be created and tested without needing any physical network switches.

  • If Solaris Zones are used, IP-Interfaces, VNICs or physical interfaces are provided by the Global Zone. An Oracle Solaris Zone can then use a shared-IP instance or an exclusive-IP instance to communicate with the Global Zone or the outside world. With shared-IP instance, the zones share one IP-stack infrastructure in the kernel with i.g. its arp-cache, routing table and IP-
    configuration flags (not the IP-address).
    A zone with an exclusive-IP instance has its own IP-stack. To use the latter one, a dedicated physical interface or virtual network interface is needed. Using a shared-IP instance does not require a dedicated network interface. The picture on the right shows the general difference.

Features of network resource management

The network is always a shared resource, either outside the server chassis by using central cables, switches or routers - or inside the chassis, by sharing physical ports, network stacks or just the CPUs that are handling the traffic, doing check-summing or handling the network adapter interrupts. To meet different service level agreements of network consumers in one chassis, network resource management is needed. The requirements can be based on available network bandwidth, network latency or network data loss rate. While network latency and data loss rate is typically based on the used network technology and the OS- or hypervisor-specific implementation, the available bandwidth can be controlled by resource management. Related to internal network virtualization, various product-specific implementations exist:

  • Dedication of a network port enables the host or the hypervisor to assign a separate physical port to a consumer. With this the consumer gets the whole bandwidth of this port, but may need many network ports, many network adapters and may be limited by the number of available PCI slots.

  • A specific CPU can be assigned to network interfaces or VNICs to handle their device interrupts, doing the data buffer handling or computing network checksums. In relation to the resource management features of the previous article of this series, we can compare these two functionalities with resource partitioning.

  • During the creation of VNICs, an interface-based network bandwidth cap can be assigned. With that the useable bandwidth is capped on a configured boundary. This enables the sharing of a physical network port by many network consumers by limiting the useable bandwidth for each consumer. This setup is very flexible and can be often changed dynamically. In the previous article we discussed this functionality as resource constraints.

  • While the previous network bandwidth capping is interface based, there is also a need to control the bandwidth on a network connection base. Such a network connection can be described by a source-IP, a destination-IP address and by a protocol. In Oracle Solaris this is called a 'flow'. The configured flows can be used to control network bandwidth independently of network interfaces, only on a connection base. The picture on the right side shows an example. A configured flow for the network data type “network backup” can be used, to give the “green” and the “blue” traffic more available bandwith in critical load situations. Compared to the basic resource management functionalities of the previous article we can compare this with resource scheduling, because if “green” and “blue” do not have bandwidth needs, “network backup” can get the maximum available bandwidth.


Virtual network interfaces, virtual bridges, virtual switches or virtual PCI ethernet cards are basic internal network virtualization features that are part of virtualization products. The networking 'glues' all the VMs, zones or containers together and let them communicate together or with the outside world. To enable stable communication for all of them on the shared resource network, the use of network resource management features is recommended. We have also seen that for networks, various types of resource managements like constraints, scheduling or partitioning are used.

With that we'd like to close this article on Network Virtualization and hope we've kept you eager to read the ones coming up in the following newsletters.

Further Reading

This series already had the following articles:

  • December 2011: Introduction to Virtualization (Matthias Pfützner)
  • January 2012: Oracle VM Server for SPARC (Matthias Pfützner)
  • February 2012: Oracle VM Server for x86 (Matthias Pfützner)
  • March 2012: Oracle Solaris Zones and Linux Containers (Detlef Drewanz)
  • April 2012: Resource Management (Detlef Drewanz)

The series will continue as follows (tentative):

  • June 2012: Oracle VM VirtualBox (Detlef Drewanz)
  • July 2012: Oracle Virtual Desktop Infrastructure (VDI) (Matthias Pfützner)
  • August 2012: OpsCenter as Management Tool for Virtualization (Matthias Pfützner)

If you have questions, feel free to contact me at: Detlef Drewanz

Read more:

<<< Part 5: Resource Management as Enabling Technology for Virtualization >>> Part  7: Oracle VM VirtualBox

Tuesday Apr 24, 2012

Virtualization @ Oracle (Part 5: Resource Management as Enabling Technology for Virtualization)

After discussing Oracle VM and OS-Virtualization with Zones and Containers in the previous articles, we will cover some enabling technologies for virtualization in the next two articles and start with

Resource Management as Enabling Technology for Virtualization

Of course here we are talking about IT Resource Management as technology, but why is this important? In the first article of this series we have used a definition of virtualization:

"Virtualization, in computing, is the creation of a virtual (rather than actual) version of something, such as a hardware platform, operating system, a storage device or network resources."

Resources are the foundation that get virtualized by the different virtualization technologies. These are:

  • Hardware like the CPU, memory or devices
  • The network
  • The Operating System
  • The desktop
  • A general software layer.

Resource management limits access to shared resources, but also monitors resource consumption and collects accounting information.

The management of these resources is important, because many consumers like VMs, zones, containers or virtual desktops are requesting resources. Consolidating different workloads on one system also means to combine workloads with different needs for throughput, response time, availability and service level agreements on one system.

But resources are always limited and they are shared among many virtualization environments on one IT system, therefore it is important to restrict access to specific shared resources, isolate resources from being used by certain workloads or at least limit shared resource consumption of workloads. With that, we can guarantee a service level for each virtualized environment or can influence their performance. Without resource management all workloads would be handled equally, based on their resource requests. The result could be that e.g. one VM consumes much memory during runtime and other VMs on the same system get blocked because an important memory request can no longer be served, because of no more available memory. Another example is how many resources (e.g. CPU) should be shown to or seen by a virtualized environment. This could be important for license or software behavior reasons.

So the goals of resource management related to zones, container or VM are:

  • Prevent them from consuming resources unlimited
  • Change a priority, based on external events
  • Balance resource guarantees against the goal of maximizing system utilization
The following basic mechanisms are used to achieve that goals:


By using constraints we set bounds on the consumption of specific resources. With that we can control ill-behaved environments, that would otherwise compromise the performance of other environments, the whole system or might also have an effect to the availability through unregulated resource requests. Typically constraints are enforced through resource controls, which are set by the system administrator. Examples for resource controls can be e.g.:

  • Used semaphore
  • Number of open files
  • Used virtual memory
  • Number of processes
  • Used network bandwidth

There are different ways to act by the system, if a specific bound has been reached.

  • Allowing the request, but let the requester know that the bound has been reached
  • Cap the resource delivery on the defined bound
  • Reject the whole resource request with an error message to the application
  • Generate an action on the system to free-up resources and provide the requester with the needed resources

Depending on the implementation, applications or virtualized environments must be modified to know about resource controls and constraints. But the use of constraints is very flexible and enables the change of boundaries during runtime. And the use of constraints enables a workload to use free resources that have been assigned, but not needed by a different workload.

Example 1: Constraints are important and useful for all kind of shared parallel access to resources. Good examples are processes, project or Oracle Solaris Zones. They all use and share one Kernel. That’s why many resource controls have been introduced in Oracle Solaris, to be able to limit their resource consumption. There is e.g. the resource control of zone.max-processes in Oracle Solaris 11, which limits the number of processes a zone can run. This is important to limit, because the process table of each OS Kernel is large, but limited in size. With this resource control we limit the portion of a zone on this table and prevent the system from ill-behaved administrative work in a zone like infinite shell-scripts creating processes. With the resource control enabled, the kernel will at some point no longer allow the zone to create a new process.

Example 2: Another common shared resource in systems is typically the network, so the connection of the workloads to the outside world. If all VM share one network cable, the bandwidth-consumption needs to be limited by VM. We will cover this example more in detail within the next article.


With scheduling we divide a resource into specific intervals and allocate them, based on a predictable algorithm. If an allocation is not needed, the resource interval can be used by others.

An example for a scheduled resource is CPU-time. With this mechanism the available time of a CPU is divided into allocation units, which are used by applications. Scheduling-based resource management enables full utilization of a configuration. But in a critically committed or over-committed situation the scheduling algorithm guarantees controlled access of all applications to the resource. Depending on the scheduling algorithm it is to be defined what “controlled access” means and under what reasons and measurements the allocation units are changed or assigned to applications. This can be based e.g. on predefined importance of an application.

Example 1: Scheduling is achieved by the use of the fair share scheduler (FSS) in Oracle Solaris together with Oracle Solaris Zones. The FSS allows the allocation of CPU resources. Each zone can have a share assigned to it. The shares are used to manage the CPU resources in the event that the zones compete for CPU time.

  • If the workload is less than 100%, no management is done since free CPU capacity is still available.

  • If the workload is at 100%, the fair share scheduler is activated and modifies the priority of the participating processes such that the assigned CPU capacity of a zone corresponds to the defined share.

  • The defined share is calculated from the share value of an active zone divided by the sum of the shares of all active zones.

With the FSS we guarantee the response time of workloads, based on CPU shares if the system is fully utilized.

Example 2: Another example is the creation and handling of virtual CPUs in Oracle VM for x86 if we do not pin directly vCPU to physical CPU. In that case a virtual CPU (vCPUs) is managed (scheduled) by a local run queue that “divides” a physical CPU into multiple vCPUs. This work is done by the hypervisor. The queue is sorted by vCPU priority. In the queue, every vCPU gets its fair share of CPU resources. The priority that a CPU would get can be managed by manipulating the weight and a cap value. The relative weight parameter is used to assign the amount of CPU cycles that a domain receives. A vCPU with a weight of 64 would receive twice as much CPU cycles as a vCPU with a weight of 32. A second parameter to tune is the cap parameter. This parameter defines in a percentage the maximum amount of CPU cycles that a domain will receive. This is an absolute value; If it is set to 100, it means that the vCPU may consume 100% of available cycles on a physical CPU, if you set it to 50, then that would mean that the VCPU can consume never more than half of the available cycles. In this example we see a combination of scheduling and constraints(capping).


Partitioning is used to assign a subset of resources to a workload. This assignment guarantees that this subset of resources is always available to the workload. But these resources can also not be used by other workloads, because they are assigned and guaranteed to one specific workload. Thus, configurations that use partitioning can avoid overcommitment of resources. However, in avoiding this overcommitment, the ability to achieve high utilizations can be reduced. A reserved resource is not available for use by another workload when the assigned workload is idle. Typical examples for partitioning are the assignment of physical CPU, parts of physical memory or parts of the I/O-system to workloads or virtualized environments.

Example 1: Let’s discuss again the way how Oracle VM for x86 is handling CPU. If we use the feature to pin vCPU to physical CPU and assign them to domains, we have a partitioning of CPU. Certain CPUs are then fixed assigned to domains. With that we guarantee always a fixed performance, but also the vCPU can not be used by other domains, even if they are idle.

Example 2: Partitioning with Oracle VM for SPARC is used for several resource types. CPU and memory are always assign directly to Logical Domains. There are also options to assign PCI slots and complete PCI infrastructure to certain domains. The advantage for this are high performance domains with close to zero overhead and guaranteed performance, if direct I/O is used.


Constraints, Scheduling or Partitioning are basic mechanisms of resource management to enable and guarantee access of various virtualization technologies to limited and shared resources. They are used for different resources based on requirements of different workloads and virtualization technologies.

Partitioning is the most used way to control resources in hypervisor based virtualization. In that case the hypervisor controls resources like CPU, Memory, Privilege-checks or hardware interrupts.

To avoid overcommitment of the CPU resources, they are typically partitioned and the physical CPUs are assigned as virtual CPU to virtual environments. In some cases a physical CPU is divided with a scheduler into multiple virtual CPU, but this generates virtualization overhead and can lead to an overcommitment on CPU resources.

The memory is typically controlled by the memory management system of the hypervisor which allocates and protects memory to guests based on rules. In some cases there is no memory management in the hypervisor, but a direct physical assignment (partitioning) of memory to guests. Overcommitment of memory resources should by avoided or is mostly not possible to configure with hypervisors.

With that we'd like to close this article on Resource Management and hope we've kept you eager to read the ones coming in the following newsletters.

Further Reading

This series already had the following articles:

  • December 2011: Introduction to Virtualization (Matthias Pfützner)
  • January 2012: Oracle VM Server for SPARC (Matthias Pfützner)
  • February 2012: Oracle VM Server for x86 (Matthias Pfützner)
  • March 2012: Oracle Solaris Zones and Linux Containers (Detlef Drewanz)

The series will continue as follows (tentative):

  • May 2012: Network Virtualization (Detlef Drewanz)
  • June 2012: Oracle VM VirtualBox (Detlef Drewanz)
  • July 2012: Oracle Virtual Desktop Infrastructure (VDI) (Matthias Pfützner)
  • August 2012: OpsCenter as Management Tool for Virtualization (Matthias Pfützner)

If you have questions, feel free to contact me at: Detlef Drewanz

Read more:

<<< Part 4: Oracle Solaris Zones and Linux Containers >>> Part 6: Network Virtualization and Network Resource Management


eSTEP LogoeSTEP is an integrated program for our partner, focusing at the technical community to provide them with relevant technical information for their day-to-day business with us


« July 2016