LDoms networking in Solaris 11
By user12611315 on Dec 10, 2011
Since Oracle Solaris11 is officially released now, I thought I will try explain how LDoms networking is integrated with all the networking enhancements in S11, mainly with project Crossbow. The network stack for Oracle Solaris 11 has been substantially re-architected in an effort known as the project Crossbow. One of the main goals of Crossbow is to virtualize the hard NICs into Virtual NICs (VNICs) to provide more effective sharing of networking resources. The VNIC feature allows dividing a physical NIC into multiple virtual interfaces to provide independent network stacks for applications.
LDoms networking in Oracle Solaris11 has been re-designed along with Crossbow to utilize the underlying infrastructure enhancements provided by Crossbow. The following is a high-level view of how LDoms virtual switch in an S11 service domain and LDoms virtual network device in an S11 Guest domain. The diagram also shows an example of an S10 domain to be fully compatible with S11 Service domain.
LDoms virtual switch in Solaris11 service domain is now re-designed to be layered on top of Crossbow MAC layer. It is designed to be at the same level of a VNIC. The actual virtual switching is now done at the Crossbow MAC layer, as a result the LDoms virtual switch is fully compatible with VNICs on the same physical NIC. Now, there is specific requirement to plumb LDoms vsw in the service domain to communicate with Guest Domains. LDoms virtual network device driver has been cre-designed to exploit various features such as rings and polling e.t.c.
- All existing LDoms networking features are fully supported with Solaris11 in both Service domain and Guest domains, this includes:
- VLANs feature
- Link based failure detection support
- NIU Hybrid I/O
- Jumbo Frames
- Link Aggregation device as an LDoms virtual switch backend devices.
- The Guest domains running both Solaris10 and Solaris11 are fully compatible with Solar11 Service domain.
- A Guest domain running Solaris11 is fully compatible with a service domain running Solaris10 or Solaris11.
- The existing LDoms configuration continues to work even if the existing Service domain or Guest domain is re-installed with Solaris11. That is, no need to re-create LDoms configuration.
- The Crossbow VNIC features such as b/w limit and link priorities are not available for LDoms virtual network(vnet) devices.
- Creation of VNICs on top of LDoms vsw or vnet devices is not supported.
- Solaris11 introduces a new IPMP failure detection mechanism known as the transitive probing which helps avoid the requirement of test IP addresses. That is, now the virtualization customers can use transitive probing to detect network failures but not worry about the requirement of large test IP addresses.
- Solaris11 has a feature known as Auto Vanity Naming that generates simple names such as net0 to all network devices. When you are creating an LDoms virtual switch, you can either use the vanity name or the actual physical NIC interface name for the net-dev option. The vanity name is preferred, but make sure you are using the right network device.
- CR 7087781: First Boot After Adding a Virtual Switch to a service domain may hang. See the Solaris11 Release notes at the following URL for more details and the workaround.
- Creation of VNICs on LDoms vnet and vsw may succeed without failing the command but the VNICs on vnet or vsw won't communicate. Note, VNICs on top of LDoms vsw and vnet are not supported.
- Zones creation Solaris11 may auto create a VNIC on LDoms vnet device which won't function. As a workaround, create a vnet device for each zone in the guest domain and explicitly assign the vnet device to a Zone. If the deployment requires a large number of vnets, then you may choose to disable inter-vnet-link feature in LDoms to save the LDC resources, there by having the ability to create a lot more vnets or other virtual devices. NOTE: the ability to disable inter-vnet links is introduced in LDoms2.1.