Monday Jan 15, 2007

Processor bindings in Solaris - quick and dirty

A note I had written to...someone...very...close... and - thought would be worthwhile to convert to a blog :-)
Ok, so I remembered our conversation from last week about binding processors to CPUs and seeing which processors these processes are actually bound to.


I wanted to refresh this in my memory and went through this last night and here are my notes.

I am doing this on a Solaris 10 server with 2 CPUs, but am assuming that this should work on Solaris 8 as well. The only caution I have is around not being completely sure whether the 'ps' command was actually enhanced to take '-o psr' as one of the 'format' arguments to it back in Solaris 8 or not. (You can validate this just by running '/usr/bin/ps -o' alone on the command line and looking for a line that includes 'psr' and/or 'pset' in the output.


Before I start, the output should look like the following:


root@ultra2cd:/export/home/isaac: 23:35 > /usr/bin/ps -o
/usr/bin/ps: option requires an argument -- o
usage: ps [ -aAdeflcjLPyZ ] [ -o format ] [ -t termlist ]
        [ -u userlist ] [ -U userlist ] [ -G grouplist ]
        [ -p proclist ] [ -g pgrplist ] [ -s sidlist ] [ -z zonelist ]
  'format' is one or more of:
        user ruser group rgroup uid ruid gid rgid pid ppid pgid sid taskid ctid
        pri opri pcpu pmem vsz rss osz nice class time etime stime zone zoneid
        f s c lwp nlwp psr tty addr wchan fname comm args projid project pset


Now, onto the experiment. The goal is to find out whether, given a certain process that is running (and you knowing its PID) we need to understand which CPU it is bound to (if any, in fact) ?

On my system, I have 2 CPUs: CPU 0 and CPU 1, as seen from the output of 'mpstat 1' below:

root@ultra2cd:/export/home/isaac: 23:29 > mpstat 1
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0    0   0  112   401  301   49   11    2    0    0    16   34   0   0  65
  1    0   0    0   114  102   54   12    2    0    0    15   34   0   0  65
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0  140   0  273   623  524   90    2   10    5    0   374    2   4   0  95
  1  489   3  215   112  106   57    4    4    6    0   364    2  14   0  84
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0  381  11  183   590  490  106    5    8   63    0  1318   14  21   0  66
  1   83   0  186   110  105   98    2   12   64    0   365    1  20   0  79
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0  209   7  366   596  490  110    8   10  121    0   864    5  10   0  85
  1  601  11  100   116  111   99    4   11  122    0  2132   23  23   0  54
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0   41   1  290   638  535   99    2    9   73    0   263    1   5   0  94
  1  407   8  103   117  108   59    7    6   68    0  2467   18  34   0  49
\^C


Now, in my environment, I don't YET have a process bound to a CPU, so I will pick a random process and bind it to CPU #0. I pick my 'tcsh' process, and note its PID (16029).



root@ultra2cd:/export/home/isaac: 23:29 > ps
   PID TTY         TIME CMD
 16014 pts/3       0:00 sh
 16029 pts/3       0:00 tcsh
 16130 pts/3       0:00 ps

Then, I run 'pbind -b 0 16029' (after glancing over 'man pbind')

root@ultra2cd:/export/home/isaac: 23:30 > pbind -b 0 16029
process id 16029: was not bound, now 0

This tells me that process was not bound and now is bound to CPU 0.


I can check that with 'pbind -Q', which will tell me all bindings.



root@ultra2cd:/export/home/isaac: 23:30 > pbind -Q
process id 16029: 0
process id 16331: 0

Notice that now I have another process, 16331. What is that ?

That's the 'pbind -Q' itself, which the shell forks to execute the query.

Notice if I keep doing it, the PID of this "mysterious" process will change. That is because there's a new process ID being created every time I invoke 'pbind -Q'.

root@ultra2cd:/export/home/isaac: 23:30 > pbind -Q
process id 16029: 0
process id 16353: 0

So now I have a process bound to a CPU, so this should be somewhat equal given your original question of 'how can I see what CPU a process is bound to?'.
Armed with a PID of a process of interest, I feed it to 'ps' and ask 'ps' to shape its output format to include the actual processor, and I get my answer below:

root@ultra2cd:/export/home/isaac: 23:31 > ps -p 16029 -o psr
PSR
  0

Looking at the various additional formats that 'ps' is capable of arranging, I can include things like the pid itself, or the fname, or even the memory address of the process (see '/usr/bin/ps -o' for the additional types of formats supported).


root@ultra2cd:/export/home/isaac: 23:31 > ps -p 16029 -o psr,pid,fname
PSR   PID COMMAND
  0 16029 tcsh

root@ultra2cd:/export/home/isaac: 23:31 > ps -p 16029 -o psr,pid,fname,pset,addr
PSR   PID COMMAND  PSET             ADDR
  0 16029 tcsh       -      300021ab7f8


Now, what if I had a number of processes that I kicked off from the current shell, and the list was too numerous to provide ? I could simply omit the '-p PID' option to 'ps' and here's what I'd get.



root@ultra2cd:/export/home/isaac: 23:31 > ps -o psr,pid,fname,pset
PSR   PID COMMAND  PSET
  - 16014 sh         -
  0 16029 tcsh       -
  0 16527 ps         -

(Note that 'pset' is a processor set, something that I have not yet configured on this system, hence the corresponding output is simply a hyphen)
What if I wanted the address ?


root@ultra2cd:/export/home/isaac: 23:31 > ps -o psr,pid,fname,pset,addr
PSR   PID COMMAND  PSET             ADDR
  - 16014 sh         -      3000219afc0
  0 16029 tcsh       -      300021ab7f8
  0 16558 ps         -      300021bb350
root@ultra2cd:/export/home/isaac: 23:32 >


What if I wanted more things, including user, time, scheduling class, priority ?

root@ultra2cd:/export/home/isaac: 23:48 > ps -p 16029 -o psr,pid,fname,pset,addr,time,class,pri,user
PSR   PID COMMAND  PSET             ADDR        TIME  CLS PRI     USER
  0 16029 tcsh       -      300021ab7f8       00:00  FSS  59     root
root@ultra2cd:/export/home/isaac: 23:48 >

root@ultra2cd:/export/home/isaac: 23:48 > ps  -o psr,pid,fname,pset,addr,time,class,pri,user
PSR   PID COMMAND  PSET             ADDR        TIME  CLS PRI     USER
  - 16014 sh         -      3000219afc0       00:00  FSS  11     root
  0 16029 tcsh       -      300021ab7f8       00:00  FSS  52     root
  0 18453 ps         -      300021c1348       00:00  FSS  59     root
root@ultra2cd:/export/home/isaac: 23:49 >

I think you get the point.


One last comment: notice how the 'sh' process has a '-' hyphen in the PSR column. That is because the 'sh' process is a parent of 'tcsh' and, since I only bound 'tcsh' itself, only tcsh's children will be bound, not the parents. Processor bindings are inherited from the parent.

Hope this was interesting and helpful.

About

Isaac Rozenfeld is a Product Manager for Oracle Solaris; current responsibilities include the portfolio of networking and installation technologies in Solaris, with a focus on easing the overall application deployment experience


You can follow Isaac on Twitter @izfromsun

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
News
Blogroll
Tech Reference

No bookmarks in folder