An administrator can configure a Kernel Zone to use CPUs from an existing resource pool to run its virtual CPUs on. If such a Kernel Zone is booting up, it binds its virtual CPUs to physical CPUs belonging to the processor set of the resource pool. Because the relation between physical and virtual CPUs is established during the boot of the Kernel Zone, subsequent changes to the resource pool may cause that not all physical CPUs are utilized efficiently by the Kernel Zone. To fix this, it was usually necessary to reboot the Kernel Zone. In Oracle Solaris 11.4 SRU57 we improved the Live Zone Reconfiguration to resolve this
inefficiency without rebooting the Kernel Zone.
If the number of virtual CPUs is not specified by the virtual-cpu resource, the Kernel Zone boots with the same number of virtual CPUs as the number of CPUs in the pool. This configuration is recommended for most workloads because it provides optimal performance as each virtual CPU has a physical CPU counterpart to schedule its workload on.
It is also possible to configure more virtual CPUs than the number of CPUs available in the pool. In this case multiple virtual CPUs compete for a CPU which may lead to suboptimal performance because we spend resources in unneeded overhead of switching virtual CPUs on less physical CPUs. On the other hand, this configuration may be useful for example for testing or debugging purposes when an application running within the Kernel Zone needs to be presented with more CPUs than is really available.
Although a Kernel Zone can be configured with a smaller number of virtual CPUs than the number of CPUs in the pool, this configuration is not recommended because CPU(s) in the pool can be wasted unless the pool is used by additional consumers (e.g. other Kernel Zones).
If the configuration of the pool has changed while the Kernel Zone is running, the virtual CPU binding may become suboptimal:
- if a CPU is removed from the pool, the Kernel Zone will immediately stop using it but the number of virtual CPUs stays the same. It means that the virtual CPU(s) originally bound to that CPU will start using remaining CPUs in the pool, sharing them with other virtual CPU(s).
- on the other hand, if a CPU is added to the pool, the Kernel Zone will not start using them for scheduling its virtual CPUs because the CPU(s) to virtual CPU(s) mapping is determined during the boot of the Kernel Zone. Thus until Oracle Solaris 11.4 SRU57 it was necessary to reboot the Kernel Zone in order to get newly added CPU(s) utilized.
Example
# poolcfg -c 'info pset my_pset' pset my_pset int pset.sys_id -2 boolean pset.default false uint pset.min 0 uint pset.max 0 string pset.policy assigned string pset.restype cpu string pset.reslist 10-12 string pset.units population uint pset.load 0 uint pset.size 0 string pset.comment boolean pset.temporary false boolean pset.psrset false # pbind -q 1575 pbind(8): LWP 1575/5 strongly bound to processor(s) 10 11 12. pbind(8): LWP 1575/6 strongly bound to processor(s) 10 11 12. pbind(8): LWP 1575/7 strongly bound to processor(s) 10 11 12.
Note: pbind(8) was changed in Oracle Solaris 11.4 SRU57 to display binding of all threads by default. Use pid/0- notation if you are on an older SRU. Now we replace CPU 10 with CPU 13 in the pset:
# poolcfg -c 'unassign from pset my_pset ( cpu 10 )' # poolcfg -c 'assign to pset my_pset ( cpu 13 )' # pooladm -c # poolcfg -c 'info pset my_pset' pset my_pset int pset.sys_id -2 boolean pset.default false uint pset.min 0 uint pset.max 0 string pset.policy assigned string pset.restype cpu string pset.reslist 11-13 string pset.units population uint pset.load 0 uint pset.size 0 string pset.comment boolean pset.temporary false boolean pset.psrset false
All 3 virtual CPUs now share only 2 CPUs 11 and 12; CPU 13 is not being utilized at all:
# pbind -q 1575 pbind(8): LWP 1575/5 strongly bound to processor(s) 11 12. pbind(8): LWP 1575/6 strongly bound to processor(s) 11 12. pbind(8): LWP 1575/7 strongly bound to processor(s) 11 12.
Example with the Live Zone Reconfiguration
Starting with Oracle Solaris 11.4 SRU57 it is possible to use Live Zone Reconfiguration to update the virtual CPU binding without rebooting the Kernel Zone:
# zoneadm -z my_kz apply Checking: Modifying virtual-cpu-binding Applying the changes
Virtual CPUs of the Kernel Zones are now using all 3 CPUs in the pset:
# pbind -q 1575 pbind(8): LWP 1575/5 strongly bound to processor(s) 11 12 13. pbind(8): LWP 1575/6 strongly bound to processor(s) 11 12 13. pbind(8): LWP 1575/7 strongly bound to processor(s) 11 12 13.
If the number of CPUs in the pset has changed and Kernel Zone configuration does not specify the number of virtual CPUs using the virtual-cpu resource, the LZR will also change the number of virtual CPUs in order to match the number of CPUs in the pset:
# poolcfg -c 'assign to pset my_pset ( cpu 10 )' # pooladm -c # zoneadm -z my_kz apply Checking: Modifying virtual-cpu-binding Checking: Modifying virtual-cpu ncpus=4 Applying the changes
A 4th virtual CPU was added to the Kernel Zone. All CPUs in the pset are utilized:
# pbind -q 1575 pbind(8): LWP 1575/5 strongly bound to processor(s) 10 11 12 13. pbind(8): LWP 1575/6 strongly bound to processor(s) 10 11 12 13. pbind(8): LWP 1575/7 strongly bound to processor(s) 10 11 12 13. pbind(8): LWP 1575/16 strongly bound to processor(s) 10 11 12 13. # zlogin my_kz psrinfo 0 on-line since 03/02/2023 13:45:08 1 on-line since 03/02/2023 13:45:09 2 on-line since 03/02/2023 13:45:09 3 on-line since 03/02/2023 14:00:13
Note that if the virtual-cpu resource is used, that configured number of virtual CPUs is honored but the CPU rebinding still happens.