Script to check CPU cores ownership on Oracle's Sun SPARC Enterprise T-Series systems

There are few scripts in my test lab which I am using while running tests. While most of them are very specific, there is one which may be of interest for you. This script is specific to T-series servers (T2 and T2 Plus processors) running Oracle VM Server for SPARC. This script allows you to verify if any CPU core is shared between two or more logical domains. If CPU threads from the same CPU core are assigned to different logical domains, then this can reduce the efficiency of these CPU threads. The reason for this is that single CPU core have memory cache shared between CPU threads belonging to this core. Cache Thrashing occurs when two programs are using cache for different memory pages and the load is very heavy.

Lets check how it works:

First, we have two guest domains configured and each guest domain have 16 CPU threads assigned to it.

# ldm ls
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      24    8000M    0.7%  1d 2h 27m
ldg1             active     -n----  5000    16    15996M   2.4%  1d 17m
ldg2             active     -n----  5001    16    15996M   8.1%  2d 2h 48m

Cores are distributed between domains in the following way as script reports:

# ./check_core_assignment.pl 
Core 0 used by primary 
Core 1 used by primary 
Core 2 used by primary 
Core 12 used by ldg2 
Core 13 used by ldg2 
Core 14 used by ldg1 
Core 15 used by ldg1 

Lets add 1 CPU thread to domain ldg1 and ldg2:

# ldm add-vcpu 1 ldg1
# ldm add-vcpu 1 ldg2
# ldm ls
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      24    8000M    0.4%  1d 2h 47m
ldg1             active     -n----  5000    17    15996M   2.4%  1d 37m
ldg2             active     -n----  5001    17    15996M   7.7%  2d 3h 8m

And check what script will report:

# ./check_core_assignment.pl 
Core 0 used by primary 
Core 1 used by primary 
Core 2 used by primary 
Core 3 used by ldg1 ldg2 MultiUsage detected
Core 12 used by ldg2 
Core 13 used by ldg2 
Core 14 used by ldg1 
Core 15 used by ldg1 

Oops, looks like some threads of core 3 assigned to domain ldg1 and some to domain ldg2. This might lead to performance impact.

To avoid this situation always add/remove CPU threads in multiplies of 8 (number of CPU threads in CPU core on T2 and T2 Plus platform).

The script itself:

# cat check_core_assignment.pl
#!/usr/bin/perl
@AllCores = ();
open(DOM, "ldm ls -p|") || die "failed to get domains";
while (<DOM>)
{
        if ( m/DOMAIN\\|name=([\^\\|]\*)/ )
        {
                $domain = $1;
                open(CPU, "ldm ls-bindings -p $domain|") || die "failed to get cpus for $domain\\n";
                while (<CPU>)
                {
                        if ( m/\\|vid=\\d\*\\|pid=(\\d\*)/ ) {
                                $core = int($1 / 8);
                                push (@AllCores, $core) unless $seen{$core}++;
                                push (@{$Usage[$core]}, $domain) unless $seen{$core, $domain}++;
                        }
                }
        }
}
foreach $c (sort {$a <=> $b} @AllCores)
{
        my $mu = 0;
        print "Core $c used by ";
        foreach $k (sort @{$Usage[$c]}) {
                print "$k ";
                $mu++;
        };
        if ($mu > 1) { print "MultiUsage detected"; }
        print "\\n";
}

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

Roman (pomah) Ivanov, ISV Engineering. Tips how to run Oracle best on Sun. Performance monitoring and tuning for system administrators. OpenSolaris user experience.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today