Oracle VM Server for SPARC 3.1.1 Now Available
By Jsavit-Oracle on Mar 20, 2014
"Dot releases" and "dot-dot-releases" don't always have new functionality, but this release has two very useful enhancements. Both are very significant for production workloads run with Oracle VM Server for SPARC.
- Support for Fibre-Channel Single Root I/O Virtualization (SR-IOV) - extending the support already available for Ethernet and InfiniBand devices.
- Network Bandwidth Controls - let administrators set bandwidth limits for guest domain virtual network devices.
Fibre Channel SR-IOV
The Fibre-Channel SR-IOV support makes it possible to flexibly provide native, bare-metal disk I/O performance to logical domains. Domains have always had virtual disk I/O provided by a service domain, which is extremely flexible and provides good performance for most applications. There also is the ability to assign a PCIe card or root complex to a domain for native performance, but the resource granularity was limited by the number of assignable PCIe devices. FC SR-IOV makes it easier to flexibly provide disk I/O without any virtualization overhead to multiple domains. A single FC card can export many SR-IOV "virtual functions" that can be individually assigned as FC devices to separate domains. This is a big deal, because it lifts constraints on logical domain performance.
I/O operations on these devices are controlled directly the the logical domain without going through a service domain or adding any overhead. This extends the value of Oracle VM Server for SPARC for the most I/O intensive applications, and with more flexible assignment and scalability than was previously available.
Please see the excellent blog entry by Raghuram Kothakota for an in-depth explanation of this new feature and how to enable it. Also see the Release Notes section PCIe SR-IOV Hardware and Software Requirements and MOS note Oracle VM Server for SPARC PCIe Direct I/O and SR-IOV Features (Doc ID 1325454.1) for detailed requirements.
Network Bandwidth Controls
Another enhancement is the ability to control the network bandwidth consumed by a virtual network device. This is very handy for any server consolidation situation, because it makes it possible to ensure that no guest domain can "hog" the network bandwidth it shares with other domains. The feature requires Solaris 11.1 service domains, and is documented at Setting the Network Bandwidth Limit.
I tried this out on my old T5220 and T5240 lab systems. They have built-in 1GbE network devices, and I can saturate them by running iperf between guest domains on different servers, getting about 930 MBit/sec. If I'm doing a server consolidation, I can make sure that no guest consumes more than a maximum bandwidth by setting a maxbw limit on the virtual network device.
First, here is now a domain's network definition looks when no limit is set (some fields snipped out to make it fit on this page)
primary# ldm list -o network ldg2 NAME ldg2 MAC 00:14:4f:f8:30:f7 NETWORK NAME SERVICE ID DEVICE MTU MAXBW LINKPROP net0 primary-vsw0@primary 0 network@0 1500 phys-state
Domain ldg2 has no limit on its virtual network device (the field below MAXBW is blank). Running parallel iperf streams between this domain and a guest domain on a different server (so I was going over the physical network) transfers about 930 Mbits/second over the 1GbE link. Now I'll set a limit of 200 MBit/second
primary# ldm set-vnet maxbw=200M net0 ldg2 primary# ldm list -o network ldg2 NAME ldg2 MAC 00:14:4f:f8:30:f7 NETWORK NAME SERVICE ID DEVICE MTU MAXBW LINKPROP net0 primary-vsw0@primary 0 network@0 1500 200 phys-state
At this point, I ran iperf again, and got 201 Mbits/second. That's probably rounding error ;-) but illustrated that the limit was in place.
Finally, I turned off the bandwidth controls altogether, just to show how it's done:
primary# ldm set-vnet maxbw= net0 ldg2
Getting to 3.1.1
Updating to Oracle VM Server for SPARC 3.1.1 was trivial. I already had the correct Solaris publisher settings, so I just updated Solaris and rebooted:
primary# pkg update --accept Packages to install: 5 Packages to update: 125 Create boot environment: Yes Create backup boot environment: No DOWNLOAD PKGS FILES XFER (MB) SPEED Completed 130/130 6506/6506 250.6/250.6 2.9M/s PHASE ITEMS Removing old actions 491/491 Installing new actions 3294/3294 Updating modified actions 5800/5800 Updating package state database Done Updating package cache 125/125 Updating image state Done Creating fast lookup database Done A clone of solaris-2 exists and has been updated and activated. On the next boot the Boot Environment solaris-3 will be mounted on '/'. Reboot when ready to switch to this updated BE. --------------------------------------------------------------------------- NOTE: Please review release notes posted at: https://support.oracle.com/epmos/faces/DocContentDisplay?id=1501435.1 --------------------------------------------------------------------------- primary# init 6
That's all there was to it - when the control domain came up it was running an updated Solaris kernel and the new Oracle VM Server for SPARC
Oracle VM Server for SPARC 3.1.1 is a new update that includes two useful new functions: SR-IOV is now extended to Fibre Channel devices, providing a new way to deliver high disk I/O performance, and adding the ability to control guest network bandwidth. These enhance the performance and manageability of production systems.