Server Virtualization - Using LDOMs on T5440
By pallab on Oct 13, 2008
The Sun Fire T5440 can have at most 4 UltraSPARC T2 processor. Each UltraSPARC-T2 Procesor is directly connected to ¼th of the entire system memory with 1Gigabyte memory interleaving and owns a PCIe Root-Complex. When fully populated with Processor and memory, Solaris can see 256 CPUs and 512GB of memory. That is a lot for many applications except for some large databases. With this class of system, it is not usually possible to consume the entire system with a singe instance of most applications. But that is in fact a very good opportunity to consolidate a bunch of such applications in this system using LDOMs, there-by reducing Power consumption and rack space. An example is the SugarCRM application. It is a web based application written using PHP and has a MySQL database backend. Yun Chew has written a nice blog demonstrating how to consolidate SugarCRM application on this system using LDOMs. I can think of many such applications that can be consolidated on this and T5140 and T5240 based systems.
The good thing about LDOMs based virtualization is that, even if the Primary Domain goes down, other domains continue to be functional. Many other virtualization technology does not have this advantage, which is why Live Migration is very critical for such virtualization technology.
To get the best performance out of a LDOMs based application deployment, it is important to understand the system topology a bit so that it becomes easier to determine what to place where. I have tried to create a sketch of the system topology below for reference.
- Is the application CPU intensive?
- Does it scale up with additional CPUs?
- Is the application Disk or Network IO intensive?
- Moderately IO intensive applications would consume less than 50% of maximum IO capacity of the device
- Is the application both CPU and IO intensive?
- How many interrupt sources the domain would need to manage?
- PCIe based Fiber Channel HBAs normally have 2 interrupt source.
- PCIe based 1G network devices have either 1 or 2 interrupt sources, while 10G network devices have 8 interrupt sources
- Each virtualized IO device created out of vsw or vds have 1 interrupt sources
number of VCPUs that need to be allocated to a Domain depends largely
on the ability of the application to make good use of the VCPUs. In
addition to the VCPUs needed by the application, extra VCPUs should be
allocated to handle interrupts. For optimal performance, when VCPUs
are allocated to a domain, then they should be allocated in multiples
of 4 at least, preferably in multiples of 8 where possible.
In the next section I will describe how to create IO domain with Inter-IO Domain Dependency