X

Everything you want and need to know about Oracle SPARC systems performance

Logical Domains and Micro Partitions

Martin Mueller
Senior Principal Software Engineer

Consolidation of different workloads often involves the use of virtualization technologies and/or partitioning of a physical server platform. Scenarios that involve partitioning sometimes result in under utilized hardware because of a too coarse granularity in compute resources. A single core might already be much more than the workload running on it needs. The same issue only seldom arises in memory assignments, the majority of partitioning technologies allow memory assignments in very small chunks. There are different approaches to mitigate a possible underutilization, we will discuss two approaches here: micro-partitioning on POWER architectures, and Logical Domains on Oracle SPARC CPUs.

Oracle SPARC CPUs, and before Sun Microsystems SPARC CPUs of the sun4v architecture type all offered a partition technology based on a hypervisor running in its own address space: the sun4v architecture offers a third address mode alongside the well known modes userland and kernel or privileged mode. You will find plenty of blog articles on this technology and its administration.

The article you are looking at focuses on one particular aspect of this technology, and compare it to a competing technology from IBM called logical partitions or LPARs for short. IBM LPARs also employ a third address space which implements the separation of the virtual machines.

We are often confronted with an addition to IBM's LPARs called micro-partitions and I would like to clarify how Oracle's Logical Domain technology LDOMs compares to IBM's logical partition technology LPARs.

Partitioning with Logical Domains or Micro-Partitions

Both technologies assign compute resources to partitions in a way that these resources are then owned by the respective partition. "Owning" does not mean that it cannot be changed dynamically, both technologies assign these resources exclusively at a given point in time. This is fundamentally different from partitioning technologies like VMware, where resources can be oversubscribed. "Oversubscription" here usually targets compute resources, you can assign more cores in total to the virtual machines than the the underlying hardware provides. This is implemented through a transparent software scheduling technology that usually time-slices the actual physical resources.

Oracle's Logical Domain technology (or Oracle VM for SPARC) does not offer any transparent software that time-slices hardware resources because this always happens at a cost, this cost is commonly known as the "virtualization tax" which can be in the range of double digit percentages.

Micro-Partitioning and Hardware Threading

Oracle's SPARC CPUs offer a very simple yet powerful threading model that allows sharing a physical resource between virtual machines in the context of Logical Domains: beginning with the SPARC T1000 and T2000 servers then Sun Microsystems and now Oracle's SPARC CPUs shared a physical core between eight hardware threads which the operating system could treat as separate "CPUs". The early releases of these heavily multi-threaded CPUs (back then known as "Cool Threads Servers") had many fairly weak cores and thus hardly fit arbitrary workloads. But with SPARC T4 onwards these CPUs offer a very competitive performance, and the heavily threaded cores.

The hardware threading technology allows switching without any overhead, and switching is the default for these CPUs. While other hardware threading technologies need a specific event (a hypervisor running a time-slicing scheduler, an interrupt, ...) for a thread switch to occur, these SPARC CPUs need a reason not to switch. The CPU itself does not have a notion of switching at all it just treats all ops alike, and keeps a tag of which hardware thread they belong to. So one can freely choose if one wants all wood behind one arrow or many, and that can be changed with every clock cycle because it is only depending on how the CPU is used, there is no configuration necessary

So how does that relate to micro-partitioning? The POWER CPUs and their partition technology LPARs offer a feature called micro-partitioning which allows an administrator to assign less than a whole core to a virtual machine, down to very tiny fractions of a core (called "processor" in this context). This feature of course should be used with caution since one starts sharing a lot of hardware resources between virtual machines, and workloads on these virtual machines may suffer from starvation when competing for a scarce resource. There might also be an overhead to account for to "create" the "micro" cores.

The obvious benefit of these micro-partitions is that one can share a powerful core among many workloads if these don't require a lot of resources.

A SPARC CPU by means of the heavily threaded core offers exactly the same functionality down to a granularity of 1/8 of a core: just assign less than 8 hardware threads to a virtual machine, of course you have exercise caution when it comes to performance considerations because you effectively share a core among different workloads

Conclusion

Both architectures offer a way of assigning less than a whole physical compute resource to a virtual machine. At first sight the Micro-Partitions may seem superior to LDOMs because you can assign compute resources at a much finer granularity. But this admittedly very fine grained assignment capability comes at a cost: the core is subdivided by means of a hypervisor. LDOMs have a conceptual advantage here: They are extremely easy to use, just assign compute or "CPU" resources one by one and not in increments of eight. So the finest increment is 1/8 of a core, but this subdivision comes at no additional cost, no "virtualization tax" to pay. For an Oracle SPARC CPU it makes no difference to handle one, three, or eight hardware threads per core. It is up to the administrator and user how they use the core.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.