This is the second in a series of blog entries that discuss the network virtualization features in Solaris 11 Express.
entry discussed the basic concepts and the virtual network elements,
including virtual NICs, VLANs, virtual switches, and InfiniBand datalinks.
This entry adds to that list the resource controls and security features that are necessary for a well-managed virtual network.
Virtual Networks, Real Resource Controls
In Oracle Solaris 11 Express, there are four main datalink resource controls:
- a bandwidth cap, which limits the amount of traffic passing through
a datalink in a small amount of elapsed time
- assignment of packet processing tasks to a subset of the system's CPUs
- flows, which were introduced in the previous blog post
- rings, which are hardware or software resources that can be dedicated
to a single purpose.
Let's take them one at a time. By default, datalinks such as VNICs can consume as much of the physical NIC's bandwidth as they want. That might
be the desired behavior, but if it isn't you can apply the property "maxbw" to a datalink. The maximum permitted bandwidth can be specified in
Kbps, Mbps or Gbps. This value can be changed dynamically, so if you set this value too low, you can change without affecting the traffic flowing
over that link. Solaris will not allow traffic to flow over that datalink at a rate faster than you specify.
You can "over-subscribe" this bandwidth cap: the sum of the bandwidth caps on the VNICs assigned to a NIC can exceed the rated bandwidth of
the NIC. If that happens, the bandwidth caps become less effective.
In addition the bandwidth cap, packet processing computation can be constrained to the CPUs associated with a workload.
First some background. When Solaris boots, it assigns interrupt handler
threads to the CPUs in the system.
CPUs for an explanation of the meaning of "CPU".)
Solaris attempts to spread the interrupt handlers out
evenly so that one CPU does not become a bottleneck for interrupt handling.
If you create non-default CPU pools, the interrupt handlers will retain their CPU assignments. One unintended side effect of this is a situation
where the CPUs intended for one workload will be handling interrupts caused by another workload. This can occur even with simple configurations
of Solaris Zones. In extreme cases, network packet processing for one zone can severely impact the performance of another zone.
To prevent this behavior, Solaris 11 Express offers the ability to assign a datalink's interrupt handler to a set of CPUs or a pool of CPUs.
To simplify this further, the obvious choice is made for you, by default, for a zone which is assigned its own resource pool. When
such a zone boots, a resource pool is created for the zone, a sufficient quantity of CPUs is moved from the default pool to the zone's pool, and
interrupt handlers for that zone's datalink(s) are automatically reassigned to that resource pool.
Network flows enable you to create multiple lanes of traffic. This
allows the parallelization of network traffic. You can assign a bandwidth
cap to a flow. Flows were introduced in
post and will be discussed further in future posts.
Finally, the newest high speed NICs support hardware rings: memory resources
that can be dedicated to a particular set of network traffic. For inbound
packets, this is the first resource control that separates network traffic
based on packet information such as destination MAC address. By assigning
one or more rings to a stream of traffic, you can commit sufficient
hardware resources to it and ensure a greater relative priority for those
packets, even if another stream of traffic on the same NIC would otherwise
cause congestion and impact packet latency of all streams.
If you are using a NIC that does not support hardware rings, Solaris 11
Express support software rings which cause a similar effect.
Virtual Networks, Real Security
In addition to rescource controls, Solaris 11 Express offers datalink
protection controls. These controls are intended to prevent a user
from creating improper packets that would cause mischief on the network.
The mac-nospoof property requires that outgoing packets have a MAC address
which matches the link's MAC address. The ip-nospoof property implements
a similar restriction, but for IP addresses. The dhcp-nospoof property
prevents improper DHCP assignment.
Summary (so far)
The network virtualization features in Solaris 11 Express enable the
creation of virtual network devices, leading to the implementation of
an entire network inside one Solaris system. Associated resource control
features give you the ability to manage network bandwidth as a resource
and reduce the potential for one workload to cause network performance
problems for another workload. Finally, security features help you minimize
the impact of an intruder.
With all of the introduction out of the way, next time I'll show some
of these concepts.