Thursday Mar 26, 2009

When to add a membar (an example)

I was recently having a discussion on one of the OpenSolaris lists on the topic of when to use the volatile keyword, and when it was necessary to use membars.

So volatile is a clue to the compiler to load the variable from memory and immediately store it back to memory. What it does not do is to tell the hardware anything. So the application can perform the store, but that store may not be immediately visible to the rest of the system. Most of the time this is not a problem - so long as the store is visible to the processor on which the thread is executing it's fine. Variability of when the store is visible to other processors may also be fine. There is one clear situation where the ordering of store operations could be a problem - and that's unlocking mutexes.

The problem here is best illustrated by the following scenario. I lock some data structure, then store new values into it, then unlock the structure. Immediately another thread comes along and uses the values in that structure. Not an uncommon situation. Unlocking a mutex is often just a case of storing a value (of zero) into the mutex structure. And here's the potential problem. In some weaker ordering architectures there is no guarantee that other processors see the stores in the same order that they are performed. So if you have Store A followed by Store B it may be possible for other processors to observe the change in the value of B before they see the change in the value of A. In the case of mutex unlock, the store of B would be the action that unlocked the mutex, enabling other threads to access the variable A... and there could be problems if they see the old value of A.

The solution to this is to put a membar in before the store that unlocks the mutex. You can see this happening in the OpenSolaris code:

     41 /\*
     42  \* lock_clear(lp)
     43  \*	- clear lock.
     44  \*/
     45 	ENTRY(_lock_clear)
     46 	membar	#LoadStore|#StoreStore
     47 	retl
     48 	  clrb	[%o0]
     49 	SET_SIZE(_lock_clear)

The membar ensures that all the pending stores are visible to other processors before the store that releases the lock becomes visible to them.

Tuesday Feb 24, 2009

OpenSolaris Bible

I've just finished reading The OpenSolaris Bible. At just over 1,000 pages it's very fortunate that I had a couple of plane journeys during which to read it. The book is six parts:

  1. Introduction to OpenSolaris
  2. Using OpenSolaris
  3. Files ystems, networking, and security
  4. Reliability, availability, and serviceability
  5. Virtualisation
  6. Developing and deploying on OpenSolaris

At its core the book is about using OpenSolaris. It has sections on developing and packaging applications for OpenSolaris, but these are brief in comparison to the rest of the content. There's a lot of content on things like setting up Zones, or configuring ZFS file systems. All of it really useful to have around as a reference. There were also some really nice tidbits of information sprinkled through the book (such as the reason for ypinit), which livened up the text.

Anyway, it's definitely something I'm glad to give shelf space to.

Thursday Sep 25, 2008

OpenSolaris Curriculum Development Resources

There's an excellent set of documents about OpenSolaris, the documents are meant to be used to support OpenSolaris in the curriculum, but they look very useful outside the educational context.

Monday Jul 28, 2008

OpenSolaris presentation in Japan

I'm back from the trip to Japan. I got to visit a number of customers and talk with them about the compilers and tools. However, the highpoint for me was the OpenSolaris event on the Friday evening. Jim Grisanzio has put up a set of photos from the event and the meal afterwards. (Yes, I was wearing a shirt and tie - the outside temperature and humidity was far to great to also wear a jacket.)

You can probably see in the pictures that the room was full - about 70 people turned up, listened, and asked some excellent questions. Keiichi Oono translated for me, and did a superb job, I think I managed to talk in short chunks, but there were a couple of occasions where I probably talked way too much. There's a couple of pictures of me using the whiteboard, and this turned out to be quite a burden to translate - I plan to do a proper write up in the next day or so.

Hisayoshi Kato did a nice talk (with live demos on a V490) of various performance tools, including some dtrace. I must admit that since Hisayoshi's talk was in Japanese I didn't actually attend all of it, and instead chatted to Jim and Takanobu Masuzuki.

Friday Jul 18, 2008

OpenSolaris Hot Topics talk 25th July - Tokyo!

The last few weeks have been quite manic, which is why I've managed to write fewer blog entries than I'd hoped. Unfortunately that's likely to continue for a while longer - I'm traveling to Japan next week for some customer visits. Due to some fortuitous timing, there's an OpenSolaris meeting on Friday the 25th. I've been invited to talk for half an hour.

Thursday May 22, 2008

Tonight's OpenSolaris User Group presentations

Slides for tonight's presentations are now available:

Wednesday May 21, 2008

OpenSolaris Users Group presentation topics

As I wrote earlier, I'm planning on a number of short presentations rather than a single long one. I don't know whether I'll manage all four of the sets of slides that I've prepared - I rather hope that there will be more discussion and I'll end up only doing one or two sets. Anyway the topics I've prepared are:

  • A deck of slides on my book.
  • A quick run through what I consider to be the important compiler flags, and the associated gotcha's.
  • Compiler support for parallelisation.
  • An overview of OpenSPARC.

Presenting at OpenSolaris Users group tomorrow

Tomorrow I'll be presenting at the Silicon Valley OpenSolaris Users group. Alan DuBoff has asked me to try and avoid the monolithic presentation, so I'll be aiming to have a couple of short presentations. The idea is to push the balance towards communications rather than presentations.

Friday May 02, 2008

OpenSolaris Summit

I'm stepping in to present at the OpenSolaris Summit on Sunday. The presentation is titled "Optimizing for OpenSolaris", and I believe someone else has already prepared the slideset - we'll see. Anyway, I'm looking forward to going, the attendees list contains many familiar names.

Thursday Jan 31, 2008

Win $20,000!

Sun has announced a Community Innovation Awards Programme - basically a $1M of prize money available for various Sun-sponsored open source projects. There is an OpenSPARC programme, and the one that catches my eye is $20k for:

vi. Best Adaptation of a single-thread application to a multi-thread CMT (Chip Multi Threaded) environment

My guess is that they will expect more than the use of -xautopar -xreduction or a few OpenMP directives :) If I were allowed to enter (unfortunately Sun Employees are not) I'd be looking to exploit the features of the T1 or T2:

  • The threads can synchronise at the L2 cache level - so synchronisation costs are low
  • Memory latency is low

The upshot of this should be that it is possible to parallelise applications which traditionally have not been parallelisable because of synchronisation costs.

Funnily enough this is an area that I'm currently working in, and I do hope to have a paper accepted for the MultiExpo.


Darryl Gove is a senior engineer in the Solaris Studio team, working on optimising applications and benchmarks for current and future processors. He is also the author of the books:
Multicore Application Programming
Solaris Application Programming
The Developer's Edge
Free Download


« July 2016
The Developer's Edge
Solaris Application Programming
OpenSPARC Book
Multicore Application Programming