Wednesday Sep 30, 2009

Querying locality groups

Locality groups are a mechanism that provides Solaris information about how the physical hardware is wired together. A locality group is a bunch of threads that share the same CPU or memory access characteristics. For example a locality group might be all the threads on a single chip.

The command to display the locality group information is lgrpinfo, but this is not on Solaris 10. Here's an example of the output from that command:

% lgrpinfo
lgroup 0 (root):
        Children: 1 2
        CPUs: 0-7
        Memory: installed 16G, allocated 3.8G, free 12G
        Lgroup resources: 1 2 (CPU); 1 2 (memory)
        Latency: 90
lgroup 1 (leaf):
        Children: none, Parent: 0
        CPUs: 0-3
        Memory: installed 8.0G, allocated 1.8G, free 6.2G
        Lgroup resources: 1 (CPU); 1 (memory)
        Load: 0.263
        Latency: 54
lgroup 2 (leaf):
        Children: none, Parent: 0
        CPUs: 4-7
        Memory: installed 8.0G, allocated 2.0G, free 6.0G
        Lgroup resources: 2 (CPU); 2 (memory)
        Load:    0
        Latency: 54

It is possible to access this programmatically:

#include <sys/lgrp_user.h>
#include <stdio.h>
#include <stdlib.h>

void explore(lgrp_cookie_t cookie,lgrp_id_t node,int level)
{
  printf("Lgroup level %i\\n",level);
  
  int ncpus=lgrp_cpus(cookie,node,0,0,LGRP_CONTENT_DIRECT);
  
  processorid_t \* cpus=(processorid_t\*)calloc(ncpus,sizeof(processorid_t));
  lgrp_cpus(cookie,node,cpus,ncpus,LGRP_CONTENT_DIRECT);
  printf("CPUs: ");
  for(int i=0; i<ncpus; i++)
  {
    printf("%i ",cpus[i]);
  }
  printf("\\n");
  
  int nchildren=lgrp_children(cookie,  node, 0,0);

  lgrp_id_t\* children=(lgrp_id_t\*)calloc(nchildren,sizeof(lgrp_id_t));
  lgrp_children(cookie,  node,children,nchildren);

  for (int i=0; i<nchildren; i++)
  {
    explore(cookie,children[i],level+1);
  }
  free(children);
}

void main()
{
  lgrp_cookie_t cookie =lgrp_init(LGRP_VIEW_CALLER); 
  lgrp_id_t node = lgrp_root(cookie);
  explore(cookie,node,0);
  lgrp_fini(cookie);
}

Which provides the following output:

% cc local.c -llgrp
% ./a.out
Lgroup level 0
CPUs:
Lgroup level 1
CPUs: 0 1 2 3
Lgroup level 1
CPUs: 4 5 6 7
About

Darryl Gove is a senior engineer in the Solaris Studio team, working on optimising applications and benchmarks for current and future processors. He is also the author of the books:
Multicore Application Programming
Solaris Application Programming
The Developer's Edge

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
5
6
8
9
10
12
13
14
15
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
The Developer's Edge
Solaris Application Programming
Publications
Webcasts
Presentations
OpenSPARC Book
Multicore Application Programming
Docs