An UltraSPARC-T1 processor ("Niagara") based system like the T1000
has limited number of onboard network/disk devices and PCI-E explansion slots. Though the PCI-E bus ports on these systems can be split and assigned to two different domains, this is far less than the max number (32) of allowed logical domains. This lack of direct I/O device access is addressed by providing I/O access to domains via virtualized devices that communicate with a 'service' domain that completely owns a device along with its driver, and functions as a proxy to the device.
LDoms virtual device support comprises of devices and services through which IO functionality is provided to all logical domains by proxying the capabilities of a physical device. The current LDoms virtual IO functionality includes support for virtual networking, disk, and console along with their corresponding proxy servers. It also includes a general purpose infrastructure for inter-domain communication called Logical Domain Channels (LDC).
The virtual network support is implemented using two components, the virtual network and switch device. The virtual network (vnet) device is an simple Ethernet device, that communicates with other vnet devices in the system via the virtual switch and/or directly using a point-to-point connection. The virtual switch (vsw) device functions as both a layer-2 network switch and a (de)mux for packets sent/received through the physical network adapter. The virtual switch classifies incoming packets on the basis of target vNet MAC addr and switches the packets to the appropriate vnet device. Similarly it acts as the forwarding agent for all packets originating from the vNets and destined to clients outside the box.
The virtual disk infrastructure provides all logical domains access to block-level storage, though a virtual disk that is backed by a real disk, disk volume or file that is a physical resource of another domain i.e. the service domain. Though the virtual disks (vdc) appear as regular disks in the client domain, the corresponding physical disk is owned and exported by the virtual disk service (vds) running on the service domain. Using a simple request/response mechanism, the client virtual disk forwards all its disk requests to the virtual disk server, and passes back to its filesystem the response it receives from the server.
In addition to the disk and network infrastructure, the console IO from all domains except primary domain is redirected to the service domain instead of the system's service processor. The service domain acts as a concentrator for all console traffic and exports console access to domain users via Unix sockets. A service domain providing console services consists of two components: a virtual console concentrator (vcc) driver and the virtual NTS daemon (vntsd).
Finally, the core infrastructure for communication between devices, services and clients in all domains is provided via Logical Domain Channels (LDC). The LDC functionality consists of support both in the Hypervisor and Solaris, and is used to send small messages or share regions of memory between the logical domains.
Please refer to the LDoms Administration and Beginners guides for more information and examples on using virtual IO. In future blogs I will discuss common virtual IO tips, tricks and gotchas.