vNICs on vNets now available
By jsavit on Apr 09, 2013
Oracle VM Server for SPARC (formerly known as Logical Domains) version 220.127.116.11 now supports Solaris 11 network virtualization, removing a previous restriction that prevented "vNICs on VNets". Previous versions did not support this because a domain's virtual network device only presented a single MAC address. Read on for further details on this capability and how to use it.
Solaris network virtualization
One of the powerful features of Solaris 11 is support for virtual networks, described in
Using Virtual Networks in Oracle Solaris 11.1,
and also known by the project name "Crossbow".
Solaris 11 lets administrators define virtual networks consisting of virtual network interface cards (VNICs), virtual switches,
and "etherstubs", which are simulated NICs used for private (within the Solaris instance) virtual networks.
Instead of just consolidating servers, you can consolidate a data center's servers and network topologies into a single Solaris instance. You can set up multiple isolated virtual networks, associate zones with the different virtual networks, and establish rules for isolation, connectivity and quality of service (QoS).
This also permits more independent control of Solaris zones by the zone administrator, extending Solaris 10 "exclusive IP" zones that assigned each zone a physical NIC. That model also provides isolation and lets the non-global zone administrator specify IP address and other properties, but is limited by the number of physical NICs and doesn't provide QoS. Both are in contrast to the shared-IP model, in which the global zone administrator sets the zone's IP address, routing, packet size, and so forth, and all the zones are essentially on the same network without separation or flow control. The shared-IP model also has restrictions: for example, the zone cannot be a DHCP client.
With Solaris virtual networking, each zone administrator has control over their settings - more like administrators of physical Solaris instances, while the global zone administrator controls isolation and resource allocation. (Note: the original "shared-IP" network model for Solaris zones still exists but is no longer the default.)
Solaris network virtualization and logical domains - the restriction
Unfortunately, this useful feature did not work in a logical domain, essentially because the virtual network devices given to a guest domain only presented a single MAC address. Here is an example showing what happens if you try to use this with a simple Solaris 11 zone. Note that the only required parameter is the zonepath.
# zonecfg -z vm10 Use 'create' to begin configuring a new zone. zonecfg:vm10> create create: Using system default template 'SYSdefault' zonecfg:vm10> set zonepath=/zones/vm10 zonecfg:vm10> exit # zoneadm -z vm10 install ... usual messages ...If you do this in a physical Solaris instance it works, but in a guest domain with a virtual network device, you get the following message when you start the domain:
# zoneadm -z vm10 boot zone 'vm10': failed to create vnic for net0: operation failed zoneadm: zone 'vm10': call to zoneadmd failed
You could certainly do a Solaris-10 style shared-IP zone or a non-VNIC exclusive-IP. For completeness sake, here's the shared-IP case. Note that in the shared-IP model the global zone administrator specifies IP address and routing:
# zonecfg -z vm11 Use 'create' to begin configuring a new zone. zonecfg:vm11> create -b zonecfg:vm11> set zonepath=/zones/vm11 zonecfg:vm11> set ip-type=shared zonecfg:vm11> add net zonecfg:vm11:net> set physical=net0 zonecfg:vm11:net> set address=10.139.246.39/23 zonecfg:vm11:net> set defrouter=10.139.246.1 zonecfg:vm11:net> end zonecfg:vm11> verify zonecfg:vm11> commit zonecfg:vm11> exit
Solaris network virtualization and logical domains - the resolution
This restriction is eliminated in the latest version of Oracle VM Server for SPARC, 18.104.22.168. In order to make use of this feature, you must run Solaris 11.1 in both the control domain and guest. The control domain must be running SRU4 or later, since the 22.214.171.124 ldomsmanager package is delivered with that SRU.
To enable the feature, use the
ldm add-vnet or
ldm set-vnet option
indicate that the virtual network device hosts multiple MAC addresses.
alt-mac-addrs values are either a MAC address in octets, or the word "auto"
indicating that the system should generate the address. For example, the following line permits two alternate MAC addresses.
# ldm add-vnet alt-mac-addrs=auto,auto vnet1 primary-vsw0 mydomain1Alternatively, you can do this in separate commands, and use the second line to add more MAC addresses to an existing virtual network device:
# ldm add-vnet vnet1 primary-vsw0 mydomain1 # ldm set-vnet alt-mac-addrs=auto,auto,auto,auto vnet1 mydomain1
In my test, I re-booted zone vm10 and it came up without difficulty. After doing that I issued the following within the guest domain's global zone:
# dladm show-phys -m LINK SLOT ADDRESS INUSE CLIENT net0 primary 0:14:4f:fa:29:1d yes vnet0 1 0:14:4f:fa:d7:9c yes vnicZBI52346836 2 0:14:4f:f9:a8:be yes net0This shows that I have two alternate MAC addresses. Note that one address is used for the virtual switch implicitly created when the virtual network device is created for the zone (as all virtual network devices require a virtual switch). If I look from the control domain at the guest's configuration, I see additional MAC addresses from the "outside":
# ldm list -l mydomain1 ... NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet01 primary-vsw0@primary 0 network@0 00:14:4f:fa:29:1d 1 1500 00:14:4f:fa:d7:9c 00:14:4f:f9:a8:beIf I want to add more zones I could re-issue the
ldm set-vnetwith more operands while the domain is down. This property is not yet dynamically reconfigurable.
Note: at the moment this is being written, the zone "anet" property "mac-address" was initialized to
random by default, rather than
It has to be explicitly changed to "auto" so the zone can boot up. Until the default is changed,
instead of using the very short form to define a zone as shown with vm10 above, add an
anet stanza as follows. Also note that the "vanity name" is used to specify the
# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 0 unknown vnet0 # zonecfg -z vm12 Use 'create' to begin configuring a new zone. zonecfg:vm12> create create: Using system default template 'SYSdefault' zonecfg:vm12> set zonepath=/zones/vm12 zonecfg:vm12> add anet zonecfg:vm12:anet> set lower-link=net0 zonecfg:vm12:anet> set mac-address=auto zonecfg:vm12:anet> end zonecfg:vm12> verify zonecfg:vm12> commit zonecfg:vm12> exit
Using this method I can add more zones, each with an independent, virtual IP stack and network.
This blog entry isn't a tutorial on Solaris network virtualization, so I've only scratched the surface of what it can do. For example, you can create multiple virtual switches, assign virtual network devices to different switches, and then set rules on which network segments can communicate with one another. You can also impose QoS controls to manage bandwidth. For further information, see Overview of Network Virtualization and How to Configure a Zone for the Virtual Network