Wednesday Mar 18, 2009

The Developers Edge

Darryl Gove just announced the availability of a a new book, The Developers Edge over on his blog. The book is a collection of various entries on sites such as, and I have the privilege of having one of my past entries, STREAM and the performance impact of compiler optimization included.

The book is available via Safari, and if your looking for tips tricks and insights to get that little extra edge I'd highly recommend dipping in.

Monday Mar 09, 2009

DTrace, Fortran & Sun Studio

Just read a really, really cool article on a newly evolving feature of the Sun Studio Fortran Compiler, a -dtrace option, which will allow you to automatically generate dtrace probes in your Fortran code. Must go play when I get an opportunity... this is wickedly cool.

Monday Nov 10, 2008

Adventures in MacBook land...

... or getting Solaris running on a Macbook Pro. I do of course have an instance running in Sun xVM VirtualBox, but I like having a native environment to work in as well.

A lot of people have gone down this path before me, so I'll focus specifcally here on getting the ndis wrapper working for the onboard broadcom card, however before going there, a quick list of the places I found info at (and the odd credit as well).

First off I used rEFIt, suggested to me by Nicky as an alternative to Bootcamp, although I did use BootCamp to do the initial partioning of the disk up. Before installing you need to do a bit of fiddling around with EFI id's, the details of which I found from an entry by Paul Mitchell, How to Dual Partition a MacBook Pro with MacOS and Solaris.

After that it was a straight install from a dvd, then onto setting up the required networking. The ethernet nic is a Marvell Yukon, scanpci details area

pci bus 0x000c cardnum 0x00 function 0x00: vendor 0x11ab device 0x436a
 Marvell Technology Group Ltd. Marvell Yukon 88E8058 PCI-E Gigabit Ethernet Controller

This is supported by Masayuki Murayama's myk driver (ported from FreeBSD's msk) from and can be downloaded from his driver page. Details are all in the readme. In my case I went with the i386_gcc compile as we need to boot in 32bit mode for the ndis wrapper we will get to in a moment.

Now the awkward part, getting the wireless. The wireless nic in this macbook is a Broadcom, details below

pci bus 0x000b cardnum 0x00 function 0x00: vendor 0x14e4 device 0x4328
 Broadcom Corporation BCM4328 802.11a/b/g/n

The folks over at the laptop community on have a detailed page on useing ndis. You need to pull down 1.2 ndis kit, and the relevant driver files (search for 14E4,4328). Now with the arrival of Fast Boot (PSARC/2008/382, flag day) in build 100 of Solaris Express we have a new driver entry point which has yet to be reflected in the ndis code, the fix is a simple change to the if_ndis.c (thanks to Michael Li for the correct quiesce option).

+++ if_ndis.c   Mon Nov 10 05:39:52 2008
@@ -159,7 +159,7 @@

 uint32_t identify;
 DDI_DEFINE_STREAM_OPS(ndis_dev_ops, nulldev, nulldev, ndis_attach, ndis_detach,
-    nodev, NULL, D_MP, NULL);
+    nodev, NULL, D_MP, NULL, ddi_quiesce_not_supported);

 static struct modldrv ndis_modldrv = {
        &mod_driverops,         /\* Type of module.  This one is a driver \*/

Guidelines for the rest of the process are listed on on the ndis page mentioned above. Once you have built the modules and copied them in place add the ndis driver with

        add_drv -i '"pci14e4,4328"' bcmndis

Plumb up the interface and you should be ready to go. The trade off here is that you have to use a 32bit kernel, so make sure you've booted into one.

The final item to get working is a right click emulation, covered in detail in Pradhaps entry Solaris Nevada / OpenSolaris Mac book right-click. Anyway after all of that, unless you really want to have Solaris running natively on your macbook I'd suggest using Sun xVM VirtualBox, and OpenSolaris.

Monday Jul 28, 2008

Solaris Nevada 94 on Acer Aspire 5610

I've been a bit lax on my Nevada upgrades for my laptop recently, but I got a chance today to bring myself back up to date with snv_94. If your going to install this on an Acer Aspire 5610 you will run into a panic on boot due to some recent ACPI changes.

Straight off on boot you will get a panic similar to below, you will however need to boot in kmdb mode to see this properly (just add -k to your grub options).

ddi_create_minor_common+0x51(0, f97a5538, 2000, 400, f97a5720, 0)
ddi_create_minor_node+0x25(0, f97a5538 , 2000, 400, f97a5720, 0)

The work around is to add -B disable-acpi_drv=true to your grub options, my snv_94 grub entry reads

title snv_94
root (hd0,2,a)
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B disable-acpi_drv=true -B acpi-user-options=8
module$ /platform/i86pc/$ISADIR/boot_archive

There is a fix in place for snv_95, and the relevant bugs are

  • 6726425 - Dell XPS M1330 panic in 32 bit kernel.
  • 6727562 - acpi_drv: snv_94 panics on 3 different laptops.

Technorati Tag(s) : ,

Wednesday Jul 02, 2008

The LinkedIn Cloud

An interesting blog which passed over my desk recently on Linked In's architecture. Well worth a read. Solaris, Java & MySQL - seems like the perfect software stack to me ;).

Tuesday Jun 17, 2008

Blogging bosses...

My new boss, Amy O'Connor, (well my bosses boss) got a mention over on the On The Record blog last week, but for a slightly less formal introduction, and a lot more background, take a look at her blog.

Amys first post in her role as VP of Services Marketing is highlighting our OpenSolaris Essentials and OpenSolaris Production Subscriptions. Changing my hat from an OpenSolaris user (and yes still a core contributor to the test community) to a member of the Services marketing team, these subscriptions offer you an unprecedented level of support with an open source OS, so go forth and deploy with confidence. Oh our multiple acronyms get a mention as well, although they are not quite up there with Tom Waits and PEHDTSCKJMBA.

Now we just have to get the direct boss blogging :).

Wednesday May 07, 2008

uperf - opensource network benchmark

I noted that the availability of uPerf, has just been announced over on perf-discuss. I had the opportunity to play with uperf a little bit in my previous role, its an extremely powerful and useful network benchmarking tool. If you need to measure networking performance with "real world" type workloads (and lets face it thats what you need to be doing) uPerf is well worth checking out.

Friday Dec 21, 2007

DTrace ip Network Provider

I see that Brendan Gregg has posted a draft PSARC case for the first part of the DTrace Network Provider over on DTrace-discuss. I can't even begin to describe how useful this will be.
Technorati Tag(s) : ,

Wednesday Dec 05, 2007

DTrace and the case of the slow login...

An interesting little problem came my way earlier on today which yet again let me show just how useful DTrace is. Way back when I did a bit of work with a customer around a small webapp they migrated to Solaris from Linux. Anyway I got a mail from them earlier today, they split out the webserver onto a seperate box a while back, and recently the initial connections to the database tier have gotten very slow, for no obvious reasons.

They were looking into things like large logs and slow writes etc, but nothing obvious was popping up. So we went through a couple of initial steps to see if anything obvious jumped out - the various \*stat tools showed everything running normally so time to reach for DTrace. A quick oneliner showed up exactly where the problem was, and while its a simple fix I thought I'd share it as an example here. The key point here is that this isn't a resource starvation issue, which is when people generally reach for DTrace, rather its a nasty side effect of a simple change that doesn't manifest itself in an obvious manner. Right, enough typing, the oneliner was...

dtrace -n 'syscall:::entry { @sc[ustack()] = count(); } tick-5sec { printa(@sc); }'

Looking at the output the first thing that I spotted were some stacks similar to`_so_send+0x7
Hmmm, so nscd is busy. The next question was "have there been any network changes?", they weren't aware of any but on some digging there was some reconfiguration work done recently on the DNS server they use. Anyway the temporary solution that they have put in place until the DNS issues are resolved is to add the new webserver machine directly to /etc/hosts.

Without DTrace the root cause would have been a lot more painful to find, instead this exchange was a couple of e-mails in the background while I got on with the day job. Gotta love DTrace ;).

Technorati Tag(s) : ,

Tuesday Nov 20, 2007

IPS @ 16th IE-OSUG

Tim has been busy organising the next Irish OpenSolaris Users Group meeting. The announcement mail is here, but for your ease of viewing..
I'm happy to announce the 16th Irish OpenSolaris User Group meeting:

Topic           Image Packaging System (IPS)
Date            Tuesday November 27th
Time            7:00pm onwards
Location        DIT, Kevin Street, room KE: 1-008

This month, Michal Pryc has kindly offered to give a talk on the Image
Packaging System[1] - a new package management system for OpenSolaris
that will ease the pain of package installation and upgrade.

The project is still in it's early stages, but you can try it out now by
downloading and experimenting with Indiana[2]. Come along to the meeting
to find out more!

Michal plans to give a quick outline of:

  \* IPS structure
  \* client/server side tools
  \* limitations (some would say benefits ;-)
  \* portability

as well as a GUI tool he's been working on. He's asked if people have
specific questions about IPS before the meeting, to feel free to post
them to the list and he'll try to prepare them for the meeting.

This should be a pretty interesting talk, and I'd definitely encourage
you to come along – please feel free to pass this message around and
spread the word.

As always, we'll also give a quick run down of OpenSolaris news
over the past month, and will be recording the talk and adding it to our
podcast feed as well as posting the slides here, in case you can't make
it to the meeting.

As always, many thanks to the nice folks at DIT for offering to host
our meeting!

A link to this announcement is:

Look forward to seeing you there!   There's a poster on my blog if you'd
like to help us publicise this meeting.



Thursday Sep 20, 2007

Ldoms @ IE-OSUG

The fourteenth Irish OpenSolaris Users Group meeting is on next Tuesday. Liam Merwick will be presenting on Logical Domains. If you have any machines with Coolthreads technology, this will definately be of interest to you.

The good folks at the Dublin Institute of Technology are hosting the meeting.

Logical Domains @ IE-OSUG
DIT, Kevin St, @ 7.00 PM, September 25th 2007

Technorati Tag(s) : ,

Thursday Sep 13, 2007

Stream and the performance impact of compiler optimization


A couple of weeks back there was a discussion on the perf-discuss alias over on around memory bandwidth/throughput benchmarks, and as is customary in such a discussion stream got a mention. During the discussion a couple of suggestions were made regarding various compiler options to use with stream. Seeing as we gather some of this data on an ongoing basis as part of Suns Performance Lifestyle, I decided at the time to throw in a couple of extra experiments so that we can demonstrate the kind of impact that various compiler options can have.

Now in order to keep the test in the realms of reproducability for most people (and of course within the bounds of not annoying other people by taking up valuable test cycles on some of our larger machines, I would have liked to give one of our Sun Fire X4600's a run with all of these experiments, but they are incredibly highly utilised by the various engineering groups we work with) the numbers which I based this post on were generated on a Sun Fire X2100 M2 Server, which is a single cpu, dual core amd64 box. The ARRAY_SIZE for our problems was set using the l2 cache size script mentioned previously.

Secondly, as mentioned before, my group does not generate benchmark numbers for publication, in general we run stream with -xopenmp -fast as a more out of the box type compilation, however here we are taking a baseline with no compiler options passed in, which consists of twenty iterations of stream, and calling that our 100% point, and finally expressing our results as a percentage of our baseline. Now with all of that aside, lets move onto the experiments.

Experiment Details

The rig, as mentioned above is a Sun Fire X2100, 1 x 2400Mhz M2 chip, 1Gb of ram. The OS used is the latest version of Solaris Developer Express (Nevada 70b for those following from OpenSolaris). The compiler is Sun Studio 12, 2007/05.

The compiler options used here do not represent an exhaustive comparison of the various compiler options, rather a more general indications of the kind of optimization that Studio 12 can do, and the impact that your compiler can have on the performance of your application. It should be noted though that Stream is a benchmark which is highly suited to be being optimized. For a more accurate set of results we use OpenMP. We have two tables of results below, one with OMP_NUM_THREADS set to the core count (two in this case) and one with OMP_NUM_THREADS set to the physical processor count (one in this case).

The Results

The data here is pretty self explanatory, the darker the green the better the number. Of the experiments we did here, the most optimal options were -fast -xopenmp -xvector=simd -xprefetch -xprefetch_level=3 and running with our environment set to include OMP_NUM_THREADS=2.

1 OpenMP Thread
metric no options -fast -fast -xopenmp -fast -xopenmp
-fast -xopenmp
-xvector=simd -xprefetch
-fast -xvector=simd
-xprefetch -xprefetch_level=3
add 100.00% 118.50% 117.04% 130.08% 188.93% 187.18%
copy 100.00% 126.39% 124.94% 217.72% 214.05% 214.68%
scale 100.00% 123.97% 122.48% 214.20% 210.21% 212.20%
triad 100.00% 118.34% 116.67% 129.84% 186.03% 188.86%

2 OpenMP Thread's
metric no options -fast -fast -xopenmp -fast -xopenmp
-fast -xopenmp
-xvector=simd -xprefetch
-fast -xvector=simd
-xprefetch -xprefetch_level=3
add 100.00% 118.50% 177.71% 245.01% 301.24% 187.33%
copy 100.00% 126.44% 181.02% 323.97% 324.35% 214.55%
scale 100.00% 124.10% 175.78% 319.18% 319.12% 212.20%
triad 100.00% 118.41% 177.07% 245.63% 298.91% 188.94%

Compiler Options

The compiler options used here represent a small subset of what you can do with Sun Studio 12, and are covered in a lot more detail in the extensive documentation. Most of the options used above are self explanatory, but the two that maybe of interest are -xvector and -xprefetch.

-xvector=simd Instructs the compiler to use SIMD, Single Instruction Multiple Data. Basically this allows us to deal with several chunks of data in one operation rather than multiple ones. Stream is heavily vector orienated, so this gives us a sizeable performance gain.
-xprefetch -xprefetch_level=3 This option enables prefetching, at the highest level the compiler supports. Prefetching is a mechanism by which data is speculatively is fetched from memory into the cpu cache. Certain processor architectures (ie Sparc, and in this case amd64) will do an amount of prefetching, but you can instruct the compiler to insert even more prefetch instructions. In the case of stream we are processing large arrays which lends itself very well to this kind of optimization, but its one that should be used with some caution.

Further Reading

The compiler folks are continuously publishing new articles which contain various tips and suggestions on how to get the most out of your compiler which are well worth reading, and its also worth signing up to the Sun Developer Netowrk to get the free downloads of Studio 12.
Technorati Tag(s) : ,

Tuesday Aug 14, 2007

DTrace for #!/bin/sh

Now this is seriously handy - Alan Hargreaves has released a DTrace provider for sh over on No more hiding under the excuse "its the shell" - bad shell scripts will now be exposed :).

Thursday Aug 02, 2007

13th Irish OpenSolaris User Group Meeting

Tim has been busy organising the next Irish OpenSolaris users group meeting -

To: [ug-ieosug] 13th Irish OpenSolaris User Group Meeting
From: Tim Foster Tim.Foster at Sun.COM
Date: Wed Aug 1 08:42:29 PDT 2007

Hi all,

I'm happy to announce the 13th Irish OpenSolaris User Group meeting:

Topic		Discovering Open High Availability Cluster
Date		Tuesday	August 14th
Time		7:00pm onwards
Location	TBD

This month, we're lucky to have Nick Solter, from the Open HA Cluster
community visiting Dublin who will give us a talk about Open HA Cluster,
which forms the basis of one of the newer communities on

Solaris Cluster, Sun's high-availability cluster software, is going
open-source! This presentation will first give an overview of High
Availability software and Solaris Cluster, and demonstrate how Open HA
Cluster can be used with OpenSolaris and the Apache webserver to provide
higher availability for web services.

Nick's talk will next describe the open-source roadmap, give an overview
of the source code, and walk through the steps to download, compile,
install, and play with Open HA Cluster and Cluster Express. The talk
will conclude with a description of the contribution process and some
sample projects for community participation.

As usual we'll also be doing the roundup of news since our last talk,
and will be adding audio from this meeting to our podcast feed.

More info at:

We \*don't\* yet have a venue for this meeting, so if you'd like to host
it, please drop a mail to this list [ I'm actually out on vacation from
Monday for 2 weeks, but will of course be at the meeting on the 14th and
will try to monitor this list in the meantime: so please could someone
finalise the venue for the group - otherwise, I'll assume we're at
meeting at the Sun offices in East Point as per last month. ]

I'd also like to take this opportunity to poll for talks or speakers for
our September meeting - if you'd like to talk about anything
OpenSolaris-related on Sep 25th, we want to hear from you.

If you're looking for an excuse to visit Ireland, here's your chance -
we'd love to have you present a talk to the user group, and will do our
best to show some Irish hospitality!

See you all then!

Technorati Tag(s) : ,

Wednesday Jul 25, 2007

compiling lame

I was compiling up lame earlier on this evening, and encountered the following error
brhist.c: In function `brhist_disp':
brhist.c:432: error: syntax error before '->' token
brhist.c:459: error: invalid type argument of `->'
brhist.c:465: error: invalid type argument of `->'

Anyway the good folks over at pkgsrc have a work around documented.




« July 2016