Mutex Contention vs number of cpus

I've had a few cases recently that have brought this issue to the fore.

It's amazing how many people think that the answer to all performance issues is to simply throw more cpu at the problem.

Let's work through this thought (and this holds true for other queuing type locks too).

  1. On Solaris, if the mutex holder is on proc, the waiter spins instead of blocking and sleeping
  2. If we have sufficient threads wanting the same mutex, we can quickly utilize all cpus in a box
  3. The more threads we have in the queue for a mutex, logically the longer any given thread will take to progress through this queue to obtain it.

What does this tell you about what is likely to happen if you add more cpus into the mix?

It's relatively obvious now, isn't it.

  1. More cpus stuck spinning in kernel space
  2. Longer mutex queues
  3. Longer average time to obtain mutex for each thread that wants it.

The obvious consequence being that adding more cpus can actually have the effect of making the problem worse.

Sigh.

Comments:

Post a Comment:
Comments are closed for this entry.
About

* - Solaris and Network Domain, Technical Support Centre


Alan is a kernel and performance engineer based in Australia who tends to have the nasty calls gravitate towards him

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Links
Blogroll

No bookmarks in folder

Sun Folk

No bookmarks in folder

Non-Sun Folk
Non-Sun Folks

No bookmarks in folder