Virtual SMP in VirtualBox 3.0
By jsavit on Jul 01, 2009
This is a really big step. Now you can host guests with up to 32 virtual CPUs on machines with VT-X or AMD-V. Think of all the work that had to be done to provide architecturally consistent CPU and memory in the virtual machine while dispatching virtual CPUs. This affects locking and atomic memory semantics, CPU scheduling to prevent starvation - a lot of things to think about and implement. I may wind up eating my words, but 32 virtual CPUs ought to be enough to handle most people for a very long time.
Use cases for multiprocessor guests
We also have things to think about when using it. For starters, "why would you do this?" The most obvious answer is "when you have work in a virtual machine that requires more than one CPU's power". Well, that should be pretty clear, and I'm going to suggest how to make use of this while avoiding some potential pitfalls. Also, multi-CPU guests can be used to test applications to make sure the operate correctly in SMP environments, handling mutual exclusion and locking correctly, and avoiding race and deadlock conditions that might not show up in a uniprocessor environment - and that the application scales well in such an environment, distributing load across the (virtual) CPUs visible to the application running in the guest. Yes, the timing and performance characteristics will be different from "real" iron, but this can be used to filter out bugs and gain insight. Finally, this is an excellent testbed for trying out procedures that would otherwise require dedicated, large, multi-CPU machines: for example, testing out how to configure zones with dedicated CPUs, or manually creating dynamic resource pools of CPUs. You can't do that on a 1-CPU virtual machine!
Consolidation and multiple virtual CPUsThe obvious use case is for guests whose workload is big enough to require more than one CPU. VirtualBox makes it possible to do this now, making it more suitable for production workloads. Consider a modern server with multiple quad-core CPUs - this lets a single guest drive more work at the same time. Or, you might have multiple guests with different peak periods, each occasionally using substantial CPU, but with non-overlapping peak periods that permit significant server consolidation. With Solaris as the host OS, you can even run VirtualBox within zones or projects, permitting fine grain control on the amount of CPU power to give to each guest. That makes a very powerful consolidation platform.
Oversubscribing has to be watched carefully!
As a general rule, you do not want to schedule more active virtual CPUs than you have physical CPUs to run them on, as that just adds overhead without adding capacity. Yes, there are exceptions, such as when you want to have a single-threaded application only be able to suck up a single (virtual) CPU's capacity while leaving other CPUs for other guest applications. But, that's a rather blunt instrument, and you should really use a resource manager to control the CPU allocated to the guest's applications. Not always possible if the guest has only primitive resource management! Certainly, use Solaris' RM when it is the guest, as the better mechanism.
So, some suggestions: don't define virtual machines with so many CPUs that the number of actively CPU-busy virtual CPUs exceeds the physical CPUs on the box. In fact, you may wish to keep one or more physical CPUs unused by virtual machines if the computer is multiple-use (like, it runs some work "first level". Maybe it's your desktop computer...). Again, if your host is Solaris, you can also constrain VirtualBox resource consumption via the Solaris Resource Manager. As a specific rule of thumb: if a guest's workload fits in a single CPU, then give it only one CPU. Best performance will probably be achieved by giving a guest the smallest number of CPUs that do the job, since that reduces internal overhead within the guest OS, and overhead in the hypervisor as well. But, if you need more CPU power, VirtualBox now makes that possible as well.