vNICs on vNets now available

Overview

Oracle VM Server for SPARC (formerly known as Logical Domains) version 3.0.0.2 now supports Solaris 11 network virtualization, removing a previous restriction that prevented "vNICs on VNets". Previous versions did not support this because a domain's virtual network device only presented a single MAC address. Read on for further details on this capability and how to use it.

Solaris network virtualization

One of the powerful features of Solaris 11 is support for virtual networks, described in Using Virtual Networks in Oracle Solaris 11.1, and also known by the project name "Crossbow". Solaris 11 lets administrators define virtual networks consisting of virtual network interface cards (VNICs), virtual switches, and "etherstubs", which are simulated NICs used for private (within the Solaris instance) virtual networks.

Instead of just consolidating servers, you can consolidate a data center's servers and network topologies into a single Solaris instance. You can set up multiple isolated virtual networks, associate zones with the different virtual networks, and establish rules for isolation, connectivity and quality of service (QoS).

This also permits more independent control of Solaris zones by the zone administrator, extending Solaris 10 "exclusive IP" zones that assigned each zone a physical NIC. That model also provides isolation and lets the non-global zone administrator specify IP address and other properties, but is limited by the number of physical NICs and doesn't provide QoS. Both are in contrast to the shared-IP model, in which the global zone administrator sets the zone's IP address, routing, packet size, and so forth, and all the zones are essentially on the same network without separation or flow control. The shared-IP model also has restrictions: for example, the zone cannot be a DHCP client.

With Solaris virtual networking, each zone administrator has control over their settings - more like administrators of physical Solaris instances, while the global zone administrator controls isolation and resource allocation. (Note: the original "shared-IP" network model for Solaris zones still exists but is no longer the default.)

Solaris network virtualization and logical domains - the restriction

Unfortunately, this useful feature did not work in a logical domain, essentially because the virtual network devices given to a guest domain only presented a single MAC address. Here is an example showing what happens if you try to use this with a simple Solaris 11 zone. Note that the only required parameter is the zonepath.

# zonecfg -z vm10 
Use 'create' to begin configuring a new zone.
zonecfg:vm10> create
create: Using system default template 'SYSdefault'
zonecfg:vm10> set zonepath=/zones/vm10
zonecfg:vm10> exit
# zoneadm -z vm10 install
... usual messages ...
If you do this in a physical Solaris instance it works, but in a guest domain with a virtual network device, you get the following message when you start the domain:
# zoneadm -z vm10 boot
zone 'vm10': failed to create vnic for net0: operation failed
zoneadm: zone 'vm10': call to zoneadmd failed

You could certainly do a Solaris-10 style shared-IP zone or a non-VNIC exclusive-IP. For completeness sake, here's the shared-IP case. Note that in the shared-IP model the global zone administrator specifies IP address and routing:

# zonecfg -z vm11
Use 'create' to begin configuring a new zone.
zonecfg:vm11> create -b
zonecfg:vm11> set zonepath=/zones/vm11
zonecfg:vm11> set ip-type=shared
zonecfg:vm11> add net
zonecfg:vm11:net> set physical=net0
zonecfg:vm11:net> set address=10.139.246.39/23
zonecfg:vm11:net> set defrouter=10.139.246.1
zonecfg:vm11:net> end
zonecfg:vm11> verify
zonecfg:vm11> commit
zonecfg:vm11> exit

Solaris network virtualization and logical domains - the resolution

This restriction is eliminated in the latest version of Oracle VM Server for SPARC, 3.0.0.2. In order to make use of this feature, you must run Solaris 11.1 in both the control domain and guest. The control domain must be running SRU4 or later, since the 3.0.0.2 ldomsmanager package is delivered with that SRU.

To enable the feature, use the ldm add-vnet or ldm set-vnet option alt-mac-addrs=value,value,value,... to indicate that the virtual network device hosts multiple MAC addresses. The alt-mac-addrs values are either a MAC address in octets, or the word "auto" indicating that the system should generate the address. For example, the following line permits two alternate MAC addresses.

# ldm add-vnet alt-mac-addrs=auto,auto vnet1 primary-vsw0 mydomain1 
Alternatively, you can do this in separate commands, and use the second line to add more MAC addresses to an existing virtual network device:
# ldm add-vnet vnet1 primary-vsw0 mydomain1 
# ldm set-vnet alt-mac-addrs=auto,auto,auto,auto vnet1 mydomain1

In my test, I re-booted zone vm10 and it came up without difficulty. After doing that I issued the following within the guest domain's global zone:

# dladm show-phys -m
LINK                SLOT     ADDRESS            INUSE CLIENT
net0                primary  0:14:4f:fa:29:1d   yes  vnet0
                    1        0:14:4f:fa:d7:9c   yes  vnicZBI52346836
                    2        0:14:4f:f9:a8:be   yes  net0
This shows that I have two alternate MAC addresses. Note that one address is used for the virtual switch implicitly created when the virtual network device is created for the zone (as all virtual network devices require a virtual switch). If I look from the control domain at the guest's configuration, I see additional MAC addresses from the "outside":
# ldm list -l mydomain1
...
NETWORK
    NAME             SERVICE                     ID   DEVICE     MAC               MODE   PVID VID                  MTU   LINKPROP  
    vnet01           primary-vsw0@primary        0    network@0  00:14:4f:fa:29:1d        1                         1500            
                                                                 00:14:4f:fa:d7:9c
                                                                 00:14:4f:f9:a8:be

If I want to add more zones I could re-issue the ldm set-vnet with more operands while the domain is down. This property is not yet dynamically reconfigurable.

Note: at the moment this is being written, the zone "anet" property "mac-address" was initialized to random by default, rather than auto. It has to be explicitly changed to "auto" so the zone can boot up. Until the default is changed, instead of using the very short form to define a zone as shown with vm10 above, add an anet stanza as follows. Also note that the "vanity name" is used to specify the lower-link:

# dladm show-phys
LINK              MEDIA                STATE      SPEED  DUPLEX    DEVICE
net0              Ethernet             up         0      unknown   vnet0
# zonecfg -z vm12 
Use 'create' to begin configuring a new zone.
zonecfg:vm12> create
create: Using system default template 'SYSdefault'
zonecfg:vm12> set zonepath=/zones/vm12
zonecfg:vm12> add anet
zonecfg:vm12:anet> set lower-link=net0
zonecfg:vm12:anet> set mac-address=auto
zonecfg:vm12:anet> end
zonecfg:vm12> verify
zonecfg:vm12> commit
zonecfg:vm12> exit

Using this method I can add more zones, each with an independent, virtual IP stack and network.

Summary

This blog entry isn't a tutorial on Solaris network virtualization, so I've only scratched the surface of what it can do. For example, you can create multiple virtual switches, assign virtual network devices to different switches, and then set rules on which network segments can communicate with one another. You can also impose QoS controls to manage bandwidth. For further information, see Overview of Network Virtualization and How to Configure a Zone for the Virtual Network

Comments:

Post a Comment:
Comments are closed for this entry.
About

jsavit

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today