Cross-CPU Live Migration in Oracle VM Server for SPARC 2.2
By jsavit on May 24, 2012
Oracle VM Server for SPARC (originally, and still often called Logical Domains) provides secure live migration, which non-disruptively moves a running guest domain from one T-series server to another. As with most things, this requires prior planning and has several technical requirements:
- T-series servers running compatible versions of Oracle VM Server for SPARC and firmware.
- Common network accessibility.
- Networked storage (that is, no use of internal direct-attach disks within the server).
- Guests only use virtual devices - that is, virtual network and disk devices provided by a service domain, as opposed to physical assignment of devices.
- Identical CPU chip and clock frequencies.
The last requirement meant that a guest domain could only be live migrated between servers of the same chip type.
For example, you can migrate domains between T2-based systems (between T5120 and T5220 servers) or between T4 systems,
but you couldn't live migrate a domain between a T2 and a T4 system.
Additionally, the clock frequency had to be identical, shown
by the value displayed by the
prtconf command for
Even within a chip type, not all products have the same clock speed.
This restricted customer ability to freely move domains between systems - and quite a few customers have a mixture of T2, T2+, T3 and T4 systems.
Enabling Cross-CPU Migration
Version 2.2 removes this restriction by adding new, cooperative functionality in firmware, the logical domains manager, and Solaris. For example, Solaris has to understand when it being moved from one processor chip set and speed and adjust accordingly.
In order to use cross-CPU migration, the servers have to be at current firmware levels as described
Oracle VM Server for SPARC 2.2 Release Notes, and the guest domains have to (at this writing) run Oracle Solaris 11.
Finally, the guest domain must be created with the constraint
using the command
ldm set-domain cpu-arch=generic mydomain, rather than the default of
In this example, I will migrate a domain between a T5240 and a T5120. First, I'll show the different procesor types. The T5240 has the following chip type and clock frequency:
t5240:~# prtconf -pv|grep stick-frequency stick-frequency: 457656f0 t5240:~# psrinfo -vp The physical processor has 8 virtual processors (0-7) UltraSPARC-T2+ (chipid 0, clock 1165 MHz)
and the T5120 has these:
t5120:~# prtconf -pv|grep stick-frequency stick-frequency: 457646c0 t5120:~# psrinfo -vp The physical processor has 8 virtual processors (0-7) UltraSPARC-T2 (chipid 0, clock 1165 MHz)
On the T5240, I've created a domain
thing1 and migrated it by entering:
ldm migrate thing1 t5120.
Logged into the guest, I was able to continue executing commands and running processes during the migration.
virtinfo command is useful when you want to obtain information about the domain and the system hosting it.
I ran that command during the migration and saw the control domain name change when the migration completed.
thing1:~# virtinfo -ct Domain role: LDoms guest Control domain: t5240 ...a little while later, from the same session... thing1:~# virtinfo -ct Domain role: LDoms guest Control domain: t5120 # psrinfo -vp thing1:~The physical processor has 8 virtual processors (0-7) sun4v-cpu (chipid 0, clock 1165 MHz)
In summary, the domain moved from a T2 processor to a T2+ while it continued running.
A few moments later, I moved it back again.
Note the processor type is set to the generic type
Video demonstration on YouTube
Brand new: a video demonstration showing live migration between a T3 and a T4, and also showing the use of the Oracle VM Manager GUI instead of the command line.
Oracle VM Server for SPARC 2.2 can now live migrate running guest domains between T-series servers even if they don't have the same chip type or frequency. This enhances operational flexibility, and lets domain migration be used in a wider set of use-cases.