Virtual network performance greatly improved!
By user12611315 on Aug 20, 2013
With latest OVM Server for SPARC, the virtual network performance has been improved greatly. We are now able to drive the line rate(9.xGbps) on a 10Gbps NIC and up to 16Gbps for Guest-to-Guest communication. These numbers are achieved with standard MTU(1500), that is no need to use Jumbo Frames to get this performance. This is made possible by introducing support for LSO in our networking stack. The following graphs are from a SPARC T5-2 platform, with 2 cores to Control domain and Guest domains.
Note: In general for any network device, the performance numbers depends on the type of workload, the above numbers are obtained with iperf workload and a message size of 8k. Note, the interface is configured with standard MTU.for throughput.
These improvements are available in S11.1 SRU9, the latest SRU is always recommended. The S10 patches with the same improvement will be available very soon. We recommend highly to use S11.1 in the Service domains.
What you need to get this performance?
- Install S11.1 SRU9 or later in both Service domain(the domain that hosts LDoms vsw) and the Guest domains. It is important both Service domain and the Guest domains to be updated to get this performance.
- S10 patches with equivalent performance are also available. The S10 patch 150031-07 is required to be installed in the S10 domain(s). Please contact Oracle support teams for any additional information.
- Update the latest system Firmware that is available for your platform platform.
- These performance numbers can only expected on SPARC T4 and newer platforms only.
- Ensure that the extended-mapin-space is set to on for both Service domain and Guest domains.
- Note OVM Server for SPARC 3.1 Software and associated FW sets extended-mapin-space to on by default so that this performance comes out of the box, in any case, confirm if it is set to on all domains.
- You can check this with the following command:
# ldm ls -l <domain-name> |grep extended-mapin extended-mapin-space=on
- If the extended-mapin-space is not set to on, you can set it to on with the following commands. Note, the changes to extended-mapin-space will trigger delayed reconfig for primary domain and require a reboot and the Guest domains required to be stopped.
# ldm set-domain extended-mapin-space=on <domain-name>
- Ensure there are sufficient CPU and memory resources assigned to both the Service domain and Guest domains. Note to drive 10Gbps or beyond performance a Guest domain need to be configured to be able to drive such performance, we recommend 2 CPU Cores or more and 4GB or more memory to be assigned to each Guest domain. As the service domain is also involved in proxying the data for the Guest domains, it is very important to assign sufficient CPU and memory resources, we recommend 2 CPU Cores or more and 4GB or more memory resources to the Service domain.
- No jumbo frames configuration is required, that is, this performance improvement will be available for standard MTU(1500) as well. We introduced support for LSO to be able to optimize the performance for standard MTU. In fact, we recommend avoid configuring Jumbo Frames unless you have specific need.