Oracle VM Server for SPARC 3.1
By jsavit on Aug 20, 2013
- Dramatically improved virtual network performance - My favorite enhancement: it is now possible to drive 10GbE networks at essentially wire speed
for guest network traffic to and from external hosts. You can have better than 10Gb/s bandwidth for domain to domain communication on the same server.
This provides virtual networking flexibility (add and remove when you want, have arbitrary number of network devices) without having to compromise on performance, providing dramatic improvements for applications that are dependent on network throughput Major kudos to Raghuram Kothakota, the author of this valuable improvement. For further details, please see his blog article.
- Cross CPU live migration enhancements (domain migration between different SPARC CPU types) - it is now possible to live migrate domains between T4, T5, M5 and future systems,
while maintaining full function, such as hardware crypto acceleration and large pages.
A new migration class has been defined that supports these servers. Cross-CPU live migration required "generic" CPU architecture definitions that disabled crypto acceleration; this is still used with the older T2, T2+ and T3 servers, which cannot live migrate onto T5 and later. For further information, please see Eric Sharakan's blog article.
- Dynamic Single Root I/O Virtualization (SR-IOV) - SR-IOV lets qualified devices act like multiple devices, with multiple "virtual functions" (VF) defined and assigned directly to a domain. This provides native performance, but up till now could only be added and removed with a service interruption. Now they can be dynamically configured to running domains. For more information, please see this blog by Raghuram.
- Infiniband SR-IOV - SR-IOV benefits are now available on Infiniband devices as well as Ethernet, providing more options for high performance I/O.
- Non-primary root domains - Previously, SR-IOV and Direct I/O (DIO) devices had to be PCIe cards in a PCIe bus owned by the primary (control) domain. Now, such devices can be in buses own by any PCIe root complex owning domain.
The SR-IOV and non-primary root domain features have firmware dependencies, which are documented in the Release Notes document.
Packaging and Upgrading
Upgrading is really easy, especially for Solaris 11 users. Just upgrade your control domain to the new SRU (Solaris 11.1 SRU 10.5), and it will automatically upgrade the logical domains daemon. A patch will be available shortly to upgrade Solaris 10 systems. This also replaces the virtual network device drivers with the version with improved performance. The guest domains should also be upgraded to the new SRU and patch to gain the performance benefits described above, since they are implemented in the Solaris drivers, and the command "ldm set-domain extended-mapin-space=on" needs to be set for service and guest domains. That becomes a default for new domains, but may need to be set for domains you have already created.
CompatibilityI found the upgrade to be transparent to existing domains without side effect. In my lab tests, I upgraded several different servers by moving domains to Solaris 11.1 SRU 10.5 and everything Just Worked(tm).
That includes older T2 and T2+ lab servers that I manage with Oracle VM Manager. To upgrade a server, simply ran 'pkg update', and when ready to reboot into the new Solaris level. Before rebooting the server, I placed it in "maintenance mode" in the Oracle VM Manager user interface. That caused guests to be migrated to another server in the same pool. After reboot, I took the server out of maintenance mode so the Manager could freely use it for deploying guests. With very little effort I repeated this process for each lab server I work with. Even with such old equipment, I measured a close to 3x network performance improvement after the upgrade. Full performance will require a more recent T4 or later system.