Wednesday May 02, 2012

figuring out cpu topology in oracle vm

I have had a few people in the past ask about CPU topology on Oracle VM and how to figure out which CPUs, cores or threads are used by a VM... Here's a quick overview :

1) get a summary of your server

The xm info command prints out a whole bunch of information about your system.

Look for the lines below in the output for detail on your systems CPUs.

# xm info
...
nr_cpus                : 8
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 2
cpu_mhz                : 3200
...
This server has a single socket with 4 cores and 2 threads per core. Total of 8 "CPUs". so cpu0..7 really is thread 0..7.

# xm info
...
nr_cpus                : 12
nr_nodes               : 1
cores_per_socket       : 6
threads_per_core       : 2
...
This server has a single socket with 6 cores and 2 threads per core. Total of 12 "CPUs".

2) get the cpu topology

The xenpm command lets you print out the thread/core/socket topology on a given server :

# xenpm get-cpu-topology
CPU     core    socket  node
CPU0     0       0       0
CPU1     0       0       0
CPU2     1       0       0
CPU3     1       0       0
In the above example, it's a single socket machine with 2 cores and 2 threads per core. cpu 0 is thread 0 of core 0, cpu 1 is thread 1 of core 0, cpu 2 is thread 0 of core 1, cpu 3 is thread 1 of core 1. If you were to have cpus="0,1" in the VM config file, you would be running on core 0. If you were to have cpus="0-3" in the VM config file, you would actually run a VM on both cores.

# xenpm get-cpu-topology
CPU     core    socket  node
CPU0     0       0       0
CPU1     0       0       0
CPU2     1       0       0
CPU3     1       0       0
CPU4     2       0       0
CPU5     2       0       0
CPU6     3       0       0
CPU7     3       0       0
In the above example, you have a single socket server with 4 cores and 2 threads per core. cpu 0 -> thread 0 of core 0, cpu 1 -> thread 1 of core 0 etc.If you were to have cpus="4-7" in the VM config file, you would run the VM on cores 2 and 3.

# xenpm get-cpu-topology
CPU     core    socket  node
CPU0     0       0       0
CPU1     0       0       0
CPU2     1       0       0
CPU3     1       0       0
CPU4     2       0       0
CPU5     2       0       0
CPU6     8       0       0
CPU7     8       0       0
CPU8     9       0       0
CPU9     9       0       0
CPU10    10      0       0
CPU11    10      0       0
In the above example, you see a single socket server with 6 cores hyperthreaded.

3) Show the guest/VM vcpu bindings to physical threads : Use the command xm vcpu-list to list a summary of which virtual CPU is running on which physical CPU.

# xm vcpu-list 1
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
0004fb00000600007c351fa24276c63f     1     0     5   -b-    4673.6 5-6
0004fb00000600007c351fa24276c63f     1     1     5   -b-    4534.0 5-6
If you add the domain ID to the command xm vcpu-list 1, you get the information for just that guest or VM. In the above example, you have a guest with 2 virtual CPUs both running, at this time, on physical cpu (thread in my case) 5. The column CPU Affinity show's 5-6 which means that both virtual CPUs could be running on either thread 5 or 6. This shows that the guest is pinned on those 2 threads. Combined with the information of xenpm get-cpu-topology I can then see that in my case CPU 5 is thread 1 of core 2 and CPU 6 is thread 1 of core 8. So this 2 vcpu guest is pinned to 2 separate physical cores.

# xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
0004fb00000600007c351fa24276c63f     1     0     5   -b-    4676.8 5-6
0004fb00000600007c351fa24276c63f     1     1     5   -b-    4537.0 5-6
Domain-0                             0     0     0   -b-     932.1 any cpu
Domain-0                             0     1     6   -b-    1168.0 any cpu
Domain-0                             0     2     7   -b-    1010.8 any cpu
Domain-0                             0     3    11   -b-     903.0 any cpu
Domain-0                             0     4     8   -b-     494.2 any cpu
Domain-0                             0     5     9   r--     773.8 any cpu
Domain-0                             0     6     1   -b-     522.7 any cpu
Domain-0                             0     7     2   -b-     785.1 any cpu
Domain-0                             0     8     4   -b-     473.8 any cpu
Domain-0                             0     9     3   -b-     728.1 any cpu
Domain-0                             0    10    10   -b-     490.8 any cpu
Domain-0                             0    11     0   r--    1219.6 any cpu
This is the same system but xm vcpu-list without argument. It also shows the dom0 guest. As you can see in this example, dom0 can run on any physical thread and the CPU Affinity is any cpu which implies any virtual CPU can be scheduled on any physical thread so no pinning or partitioning.

4) using ovm_utils ovm_vmcontrol lets a user or admin set and get the cpu/vcpu bindings for a VM through Oracle VM Manager in 3.0.

# ./ovm_vmcontrol -u admin -p Manager1 -h localhost -v apitest -c vcpuget
Oracle VM VM Control utility 0.4.2.
Connected.
Command : vcpuget
Current pinning of virtual CPUs to physical threads : 5,6
In the above example, I have a VM named apitest and using the action vcpuget it shows me that virtual cpus of this guest are bound to threads 5 and 6. This happens to be the VM in the previous example which showed that this was core 2 and core 8 (3).

Now let's bind the vcpu's to core 0, I just run the following command :

# ./ovm_vmcontrol -u admin -p Manager1 -h localhost -v apitest -c vcpuset  -s 0
Oracle VM VM Control utility 0.4.2.
Connected.
Command : vcpuset
Pinning virtual CPUs
Pinning of virtual CPUs to physical threads  '0' 'apitest' completed.
Now, running the xm vcpu-list command, I can see that this immediately took effect (and also will on subsequent startups of the VM) :
# xm vcpu-list 1
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
0004fb00000600007c351fa24276c63f     1     0     0   -b-    4687.6 0
0004fb00000600007c351fa24276c63f     1     1     0   -b-    4547.2 0
The VM now has CPU Affinity 0 for both virtual CPUs.
About

Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

You can follow him on Twitter at @wimcoekaerts

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
9
10
11
12
13
14
15
16
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today