Network Virtualization High Availability

How to add high availability to the network infrastructure of a multitenant cloud environment using the DLMP aggregation technology introduced in Oracle Solaris 11.1.
This article is Part 1 of a two-part series. In Part 1, we will cover how to implement network HA using datalink multipathing (DLMP) aggregation technology, which was introduced in Oracle Solaris 11.1.

In Part 2 of this series, we will explore how to secure the network and perform typical network management operations for an environment that uses DLMP aggregations.

Once we virtualize a network cloud infrastructure using Oracle Solaris 11 network virtualization technologies—such as virtual network interface cards (VNICs), virtual switches, load balancers, firewalls, and routers—the network itself becomes an increasingly critical component of the cloud infrastructure.

In order to add resiliency to the network infrastructure layer, we need to implement an HA solution at this layer, such as we would do for any other mission-critical component of the data center.

A DLMP aggregation allows us to deliver resiliency to the network infrastructure by providing transparent failover and increasing throughput.
The objects that are involved in the process are VNICs, irrespective of whether they are configured inside Oracle Solaris Zones or in logical domains under Oracle VM Server for SPARC.

Using this technology, you can add HA to your current network infrastructure without the cross-organizational complexity that might often be associated with this kind of solution.

The benefits of this technology are clear and they take into account the limitations of existing technologies:

Since the IEEE 802.3ad trunking standard does not cover the case for building a trunk across multiple network switches, the network switch becomes a single point of failure (SPOF). Some vendors have added this capability to their product, but these implementations are vendor-specific and, therefore, prevent combining switches from multiple vendors when building a multi-switch trunk. Because Oracle Solaris provides resilience, DLMP aggregation can be implemented across two different network switches, thus eliminating the network switch as a SPOF. As an additional benefit, because the aggregation is implemented at the operating system layer, there is no need to set anything up on the switch.

Building a network HA solution that is based on previously available IP network multipathing (IPMP) can be a complex task. With IPMP, HA is implemented at Layer 3 (the IP layer), which needs to be configured in the global zones and within each zone, and requires multiple VNICs to be assigned to each zone or virtual machine (VM). This involves more configuration steps, requires spare IP addresses out of the address pool, and generally can be an error-prone process. In contrast, the DLMP aggregation setup is much simpler since all the configuration takes place at Layer 2 in the global zone; therefore, every non-global zone can directly benefit from the underlying technology without the need for additional configuration. In addition, every new Oracle Solaris Zone that is provisioned automatically benefits from this capability. Moreover, we can create an aggregation over four 10 Gb/sec network interfaces; combining all the interfaces together, we can achieve up to 40 Gb/sec of network bandwidth.

DLMP can provide additional benefits when employed together with other network virtualization technologies that are implemented in the Oracle Solaris 11 operating system, such as link protection and the ability to configure a bandwidth limit on a VNIC or a traffic flow to meet service-level agreements (SLAs). Combining these technologies provides for a uniquely compelling network solution in terms of HA, security, and performance in a cloud environment.


Question about your article: You state that we need to do IPMP for every zone, but its not needed necessarily? You could also simply provide 2 VNETs to the LDOM or do IPMP on the Globalzone, and have the Non-globalzones use that interface.

For eg:

set address=
set physical=vnet0
set defrouter=

and the IPMP is done on the globalzone:
cat /etc/hostname.vnet0
host1 netmask + broadcast + group ipmp1 up

cat /etc/hostname.vnet1
group ipmp1 standby up

Now, when the link fails over, it is transparent to the zone.

However, you are right, we do need to provide 2 vnets to each guest domain.

Posted by Murali on July 09, 2014 at 11:58 PM IDT #

Hi Murali

Both IPMP and Link Aggregation are based on the grouping of network interfaces, and some of their features overlap, such as higher availability. These technologies are however implemented at different layers of the stack, and have different strengths and weaknesses.
The following table presents a general comparison between link aggregation and IPMP:

Posted by guest on July 10, 2014 at 09:51 AM IDT #

The question that raised is a good one and so is the answer but the real question is about performance true a network or its interface or multi nicks combined and standby and dlmp layer. Vnice you need lots off for ipmp to get performance and since we are still only have copper not fiber on Ethernet where do you think the delay is not at the interfaces its old cables and transceivers and switched that are still attached to networks. The interfaces can run faster now than before. even conflicting ip address slow a network down and there lot of that happening. a 10mb network I see on large scale clusters configurations which is really a joke.. networking is an art and not many understand it

Posted by guest on July 12, 2014 at 05:38 AM IDT #

Does IPMP work when the zone containers are based on solaris 8, 9, or 10?

Posted by guest on July 14, 2014 at 05:46 PM IDT #

Hi guest,

You can find more information about the new solaris 11 network monitoring tools, You can use those tools in order to find where the bottleneck is

Posted by guest on July 14, 2014 at 07:27 PM IDT #

Hi guest,

Although the blog covers DLMP, You can setup IPMP in Solaris 10 branded zone, for more information see:

Posted by Orgad Kimchi on July 14, 2014 at 07:30 PM IDT #

This other blog ( shows IPMP + IPoIB.

Can we take advantage of DLMP + IPoIB as well?

Thanks in advance.

Posted by Carlos Azevedo on July 17, 2014 at 09:05 PM IDT #


Kindly note that Solaris doesn't support DLMP + IPoIB


Posted by Orgad Kimchi on July 19, 2014 at 08:46 PM IDT #

Post a Comment:
Comments are closed for this entry.

This blog covers cloud computing, big data and virtualization technologies


« June 2016