Friday Mar 20, 2009

OpenSolaris 2009 Elections & Constitution

Just cast my ballot in this years OpenSolaris OGB Ballot, and voted yes for the new OpenSolaris constitution. I strongly believe that if your entitled to vote on something and didn't then you shouldn't be complaining about it later, so if you are a core contributor you should go vote now. Polls close at Mon Mar 23 23:59:59 2009 PDT.

Friday Mar 06, 2009

Analyzing Performance Benchmark Data

One of my colleagues back in my days in performance, Eoin Lawless did a very interesting blog post on Analyzing Performance Benchmark Data a few weeks back. Well worth a read.

Thursday Jan 22, 2009

Irish Open Solaris Groups Meetup

Tim sent an announcement out yesterday to on the 18th IE-OSUG meeting, 7:30 PM @ The Vaults next Thursday Jan 29th.

For those interested in hearing about my adventures in mac book land, be it with OpenSolaris native on the mac or via Sun xVM Virtualbox I'll have the macbook along so you can take a look. You can of course pull me aside to talk about our OpenSolaris service & support offerings if so inclined, but its Thursday evening so it might cost you a beer.

Wednesday Dec 10, 2008

Layers of cool tech....

or tech for tech's sake in some eyes..... (go on click on the image)

yes thats me getting the best of multiple worlds with OpenSolaris running in VirtualBox on my MacBook (running OS X which includes Sun innovations such as DTrace and ZFS) and installing NetBeans. Adventures in my development environment to follow...

And if you want to use OpenSolaris in production with the peace of mind that a support contract brings your boss take a look at our various OpenSolaris subscription services.

Monday Nov 10, 2008

Adventures in MacBook land...

... or getting Solaris running on a Macbook Pro. I do of course have an instance running in Sun xVM VirtualBox, but I like having a native environment to work in as well.

A lot of people have gone down this path before me, so I'll focus specifcally here on getting the ndis wrapper working for the onboard broadcom card, however before going there, a quick list of the places I found info at (and the odd credit as well).

First off I used rEFIt, suggested to me by Nicky as an alternative to Bootcamp, although I did use BootCamp to do the initial partioning of the disk up. Before installing you need to do a bit of fiddling around with EFI id's, the details of which I found from an entry by Paul Mitchell, How to Dual Partition a MacBook Pro with MacOS and Solaris.

After that it was a straight install from a dvd, then onto setting up the required networking. The ethernet nic is a Marvell Yukon, scanpci details area

pci bus 0x000c cardnum 0x00 function 0x00: vendor 0x11ab device 0x436a
 Marvell Technology Group Ltd. Marvell Yukon 88E8058 PCI-E Gigabit Ethernet Controller

This is supported by Masayuki Murayama's myk driver (ported from FreeBSD's msk) from and can be downloaded from his driver page. Details are all in the readme. In my case I went with the i386_gcc compile as we need to boot in 32bit mode for the ndis wrapper we will get to in a moment.

Now the awkward part, getting the wireless. The wireless nic in this macbook is a Broadcom, details below

pci bus 0x000b cardnum 0x00 function 0x00: vendor 0x14e4 device 0x4328
 Broadcom Corporation BCM4328 802.11a/b/g/n

The folks over at the laptop community on have a detailed page on useing ndis. You need to pull down 1.2 ndis kit, and the relevant driver files (search for 14E4,4328). Now with the arrival of Fast Boot (PSARC/2008/382, flag day) in build 100 of Solaris Express we have a new driver entry point which has yet to be reflected in the ndis code, the fix is a simple change to the if_ndis.c (thanks to Michael Li for the correct quiesce option).

+++ if_ndis.c   Mon Nov 10 05:39:52 2008
@@ -159,7 +159,7 @@

 uint32_t identify;
 DDI_DEFINE_STREAM_OPS(ndis_dev_ops, nulldev, nulldev, ndis_attach, ndis_detach,
-    nodev, NULL, D_MP, NULL);
+    nodev, NULL, D_MP, NULL, ddi_quiesce_not_supported);

 static struct modldrv ndis_modldrv = {
        &mod_driverops,         /\* Type of module.  This one is a driver \*/

Guidelines for the rest of the process are listed on on the ndis page mentioned above. Once you have built the modules and copied them in place add the ndis driver with

        add_drv -i '"pci14e4,4328"' bcmndis

Plumb up the interface and you should be ready to go. The trade off here is that you have to use a 32bit kernel, so make sure you've booted into one.

The final item to get working is a right click emulation, covered in detail in Pradhaps entry Solaris Nevada / OpenSolaris Mac book right-click. Anyway after all of that, unless you really want to have Solaris running natively on your macbook I'd suggest using Sun xVM VirtualBox, and OpenSolaris.

Monday Aug 11, 2008

VDBench Release....

Just back from some quite enjoyable vacation time, and I noticed that Henk has released VDBench to the general public.

In my previous role with the Perf folks here in Dublin I used vdbench very extensively. We found (and the guys still find) more than a few bugs with it, and I can't praise it highly enough as a benchmark. In a world where some benchmarks are completely unrealistic and others are just plain pointless (yep dd is a benchmark tool, honest guv) vdbench allows you to stimulate realistic workloads quickly and easily. If you benchmark disks at all you should be aware of vdbench.

Technorati Tag(s) : ,

Wednesday May 07, 2008

uperf - opensource network benchmark

I noted that the availability of uPerf, has just been announced over on perf-discuss. I had the opportunity to play with uperf a little bit in my previous role, its an extremely powerful and useful network benchmarking tool. If you need to measure networking performance with "real world" type workloads (and lets face it thats what you need to be doing) uPerf is well worth checking out.

Friday Jan 18, 2008

17th Irish OpenSolaris Users Group Meeting

Tim have been busy planning the next IE-OSUG meeting
Topic           News from December/January, Lightning talks
		& generally being social!

Date            Thursday January 31st
Time            7:00pm onwards
Location        DIT, Kevin Street, room KE: 1-008
	        (unless I hear otherwise from the nice folks at DIT)

This month, we don't actually have a main speaker organised, so we're
just going to wing it.

Along with presenting the OpenSolaris news which we didn't get to talk
about in December[1], we'll also cover January's news items, which are
already starting to mount up!

In addition, we could use the extra time to host some ad-hoc Lightning
Talks[2] - these worked pretty well the last time we did these, and we
think the casual atmosphere was a good one:
If you've slides in advance that you want to send on, drop Tim a mail,
or feel free to just bring them along on the night.

We'll update this month's web page at:

with any Lightning Talk topics we get in advance -- all volunteers

As always, we'll be recording the talk and adding it to our
podcast feed as well as posting the slides here, in case you can't make
it to the meeting.

Note this month, we're holding the meeting on a Thursday this month -
we've had a few people say that Tuesdays don't suit, so perhaps it would
be better switching the day the meetings on each month to better
accommodate people ?

Many thanks to the nice folks at DIT for hosting our meeting, please let
me know if the change of date doesn't suit?

A link to this announcement is:
there's a PDF poster on the page you can use to further publicise the

Friday Dec 21, 2007

DTrace ip Network Provider

I see that Brendan Gregg has posted a draft PSARC case for the first part of the DTrace Network Provider over on DTrace-discuss. I can't even begin to describe how useful this will be.
Technorati Tag(s) : ,

Wednesday Dec 05, 2007

DTrace and the case of the slow login...

An interesting little problem came my way earlier on today which yet again let me show just how useful DTrace is. Way back when I did a bit of work with a customer around a small webapp they migrated to Solaris from Linux. Anyway I got a mail from them earlier today, they split out the webserver onto a seperate box a while back, and recently the initial connections to the database tier have gotten very slow, for no obvious reasons.

They were looking into things like large logs and slow writes etc, but nothing obvious was popping up. So we went through a couple of initial steps to see if anything obvious jumped out - the various \*stat tools showed everything running normally so time to reach for DTrace. A quick oneliner showed up exactly where the problem was, and while its a simple fix I thought I'd share it as an example here. The key point here is that this isn't a resource starvation issue, which is when people generally reach for DTrace, rather its a nasty side effect of a simple change that doesn't manifest itself in an obvious manner. Right, enough typing, the oneliner was...

dtrace -n 'syscall:::entry { @sc[ustack()] = count(); } tick-5sec { printa(@sc); }'

Looking at the output the first thing that I spotted were some stacks similar to`_so_send+0x7
Hmmm, so nscd is busy. The next question was "have there been any network changes?", they weren't aware of any but on some digging there was some reconfiguration work done recently on the DNS server they use. Anyway the temporary solution that they have put in place until the DNS issues are resolved is to add the new webserver machine directly to /etc/hosts.

Without DTrace the root cause would have been a lot more painful to find, instead this exchange was a couple of e-mails in the background while I got on with the day job. Gotta love DTrace ;).

Technorati Tag(s) : ,

Tuesday Nov 20, 2007

IPS @ 16th IE-OSUG

Tim has been busy organising the next Irish OpenSolaris Users Group meeting. The announcement mail is here, but for your ease of viewing..
I'm happy to announce the 16th Irish OpenSolaris User Group meeting:

Topic           Image Packaging System (IPS)
Date            Tuesday November 27th
Time            7:00pm onwards
Location        DIT, Kevin Street, room KE: 1-008

This month, Michal Pryc has kindly offered to give a talk on the Image
Packaging System[1] - a new package management system for OpenSolaris
that will ease the pain of package installation and upgrade.

The project is still in it's early stages, but you can try it out now by
downloading and experimenting with Indiana[2]. Come along to the meeting
to find out more!

Michal plans to give a quick outline of:

  \* IPS structure
  \* client/server side tools
  \* limitations (some would say benefits ;-)
  \* portability

as well as a GUI tool he's been working on. He's asked if people have
specific questions about IPS before the meeting, to feel free to post
them to the list and he'll try to prepare them for the meeting.

This should be a pretty interesting talk, and I'd definitely encourage
you to come along – please feel free to pass this message around and
spread the word.

As always, we'll also give a quick run down of OpenSolaris news
over the past month, and will be recording the talk and adding it to our
podcast feed as well as posting the slides here, in case you can't make
it to the meeting.

As always, many thanks to the nice folks at DIT for offering to host
our meeting!

A link to this announcement is:

Look forward to seeing you there!   There's a poster on my blog if you'd
like to help us publicise this meeting.



Wednesday Oct 17, 2007

Solaris Developer Express on Acer 5613 AWLMi

This post is a bit out of date as there have been several more builds of Solaris Express since I originally started to create this entry, but it should provide a good starting point for anyone with a similar laptop. While installing Solaris Developer Express on an Acer 5613 I ran into a couple of issues that took a bit of time to resolve, so rather than having someone else having to do the same digging I've listed the relevant details here. The installation discussed here was done with a Solaris Express Developer Edition 9/07 dvd.

Initial Boot

The intial bootup from the dvd goes through fine until we start to configure /dev. Now it turns out that Acer bioses have a history of being a bit funky in regards to ACPI, and the workaround is documented in 6505915. You need to add the following options to your Grub menu options before starting the installation.
-B acpi-user-options=8
If your curious about ACPI on Solaris Dana Myers has a nice blog posting from way back on how to configure Solaris ACPI at boottime. In the above workaround we have just switched our acpi mode to legacy.


The onboard nic is a BCM4401-B0 Broadcom, which is supported by the bfe driver.

Wireless networking proved to be problematic, initially I had suspected something odd with the chipset, but it is supported. I logged 6614097 which after some analysis with the help of the wifi nic folks was shown to be a side effect of to the ACPI problems noted above.

This problem has been root caused to an inability to switch on the RF switch from Solaris due to having to disable ACPI, unfortunately there is no workaround for this. My solution has been to purchase an atheros based pcmcia wireless card which works perfectly.

I/O Resource Warnings

On the initial boot we get a warning message similar to
	WARNING: out of I/O resources on bridge: bus 0x20, dev 0x3, func 0x0, for secondary bus 0x23
Which is 6573171 - unnecessary I/O resource warnings on some machines. A fix for this issue integrated into Nevada 75, but its nothing to worry about anyway.

Grub Entries for Windows & ACPI Workaround

I decided to keep the Vista partition on this laptop, and while the installer picked up the windows partition, it didn't add it to my grub menu. As far as I am aware this has been fixed, but just in case I added the following to my /boot/grub/menu.lst under the unknown partition id section.
# Unknown partition of type 6 found on /dev/rdsk/c0d0p0 partition: 2
# It maps to the GRUB device: (hd0,1) .
title Windows Vista
rootnoverify (hd0,1)
chainloader +1
We also need to pass the acpi option listed above into Solaris, which gave me the following
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris Express Developer Edition 9/07 snv_70b X86
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B acpi-user-options=8
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris failsafe
kernel /boot/platform/i86pc/kernel/unix -s -B acpi-user-options=8
module /boot/x86.miniroot-safe
#---------------------END BOOTADM--------------------
Technorati Tag(s) : ,

Thursday Sep 20, 2007

Ldoms @ IE-OSUG

The fourteenth Irish OpenSolaris Users Group meeting is on next Tuesday. Liam Merwick will be presenting on Logical Domains. If you have any machines with Coolthreads technology, this will definately be of interest to you.

The good folks at the Dublin Institute of Technology are hosting the meeting.

Logical Domains @ IE-OSUG
DIT, Kevin St, @ 7.00 PM, September 25th 2007

Technorati Tag(s) : ,

Thursday Sep 13, 2007

Stream and the performance impact of compiler optimization


A couple of weeks back there was a discussion on the perf-discuss alias over on around memory bandwidth/throughput benchmarks, and as is customary in such a discussion stream got a mention. During the discussion a couple of suggestions were made regarding various compiler options to use with stream. Seeing as we gather some of this data on an ongoing basis as part of Suns Performance Lifestyle, I decided at the time to throw in a couple of extra experiments so that we can demonstrate the kind of impact that various compiler options can have.

Now in order to keep the test in the realms of reproducability for most people (and of course within the bounds of not annoying other people by taking up valuable test cycles on some of our larger machines, I would have liked to give one of our Sun Fire X4600's a run with all of these experiments, but they are incredibly highly utilised by the various engineering groups we work with) the numbers which I based this post on were generated on a Sun Fire X2100 M2 Server, which is a single cpu, dual core amd64 box. The ARRAY_SIZE for our problems was set using the l2 cache size script mentioned previously.

Secondly, as mentioned before, my group does not generate benchmark numbers for publication, in general we run stream with -xopenmp -fast as a more out of the box type compilation, however here we are taking a baseline with no compiler options passed in, which consists of twenty iterations of stream, and calling that our 100% point, and finally expressing our results as a percentage of our baseline. Now with all of that aside, lets move onto the experiments.

Experiment Details

The rig, as mentioned above is a Sun Fire X2100, 1 x 2400Mhz M2 chip, 1Gb of ram. The OS used is the latest version of Solaris Developer Express (Nevada 70b for those following from OpenSolaris). The compiler is Sun Studio 12, 2007/05.

The compiler options used here do not represent an exhaustive comparison of the various compiler options, rather a more general indications of the kind of optimization that Studio 12 can do, and the impact that your compiler can have on the performance of your application. It should be noted though that Stream is a benchmark which is highly suited to be being optimized. For a more accurate set of results we use OpenMP. We have two tables of results below, one with OMP_NUM_THREADS set to the core count (two in this case) and one with OMP_NUM_THREADS set to the physical processor count (one in this case).

The Results

The data here is pretty self explanatory, the darker the green the better the number. Of the experiments we did here, the most optimal options were -fast -xopenmp -xvector=simd -xprefetch -xprefetch_level=3 and running with our environment set to include OMP_NUM_THREADS=2.

1 OpenMP Thread
metric no options -fast -fast -xopenmp -fast -xopenmp
-fast -xopenmp
-xvector=simd -xprefetch
-fast -xvector=simd
-xprefetch -xprefetch_level=3
add 100.00% 118.50% 117.04% 130.08% 188.93% 187.18%
copy 100.00% 126.39% 124.94% 217.72% 214.05% 214.68%
scale 100.00% 123.97% 122.48% 214.20% 210.21% 212.20%
triad 100.00% 118.34% 116.67% 129.84% 186.03% 188.86%

2 OpenMP Thread's
metric no options -fast -fast -xopenmp -fast -xopenmp
-fast -xopenmp
-xvector=simd -xprefetch
-fast -xvector=simd
-xprefetch -xprefetch_level=3
add 100.00% 118.50% 177.71% 245.01% 301.24% 187.33%
copy 100.00% 126.44% 181.02% 323.97% 324.35% 214.55%
scale 100.00% 124.10% 175.78% 319.18% 319.12% 212.20%
triad 100.00% 118.41% 177.07% 245.63% 298.91% 188.94%

Compiler Options

The compiler options used here represent a small subset of what you can do with Sun Studio 12, and are covered in a lot more detail in the extensive documentation. Most of the options used above are self explanatory, but the two that maybe of interest are -xvector and -xprefetch.

-xvector=simd Instructs the compiler to use SIMD, Single Instruction Multiple Data. Basically this allows us to deal with several chunks of data in one operation rather than multiple ones. Stream is heavily vector orienated, so this gives us a sizeable performance gain.
-xprefetch -xprefetch_level=3 This option enables prefetching, at the highest level the compiler supports. Prefetching is a mechanism by which data is speculatively is fetched from memory into the cpu cache. Certain processor architectures (ie Sparc, and in this case amd64) will do an amount of prefetching, but you can instruct the compiler to insert even more prefetch instructions. In the case of stream we are processing large arrays which lends itself very well to this kind of optimization, but its one that should be used with some caution.

Further Reading

The compiler folks are continuously publishing new articles which contain various tips and suggestions on how to get the most out of your compiler which are well worth reading, and its also worth signing up to the Sun Developer Netowrk to get the free downloads of Studio 12.
Technorati Tag(s) : ,

Tuesday Aug 21, 2007

Quick grab of L2-Cache sizes on x64 for stream

One of the benchmarks we run on a regular basis is stream, which needs you to set up some feasible values based on the size of your L2 cache. Following a recent discussion over on perf-discuss I figured a bit of background and sharing was called for. Now we get access to a lot of random hardware, both new and old, particularly in the x64 space, and you generally just want to get things up and running without thinking about every small config detail, so enter smbios(1M). As an example the L2-Cache entry on a box I'm looking at currently is

6     27   SMB_TYPE_CACHE (processor cache)

  Location Tag: L2-Cache

  Level: 2
  Maximum Installed Size: 1048576 bytes
  Installed Size: 1048576 bytes
  Speed: Unknown
  Supported SRAM Types: 0x10
        SMB_CAT_PBURST (pipeline burst)
  Current SRAM Type: 0x10 (pipeline burst)
  Error Correction Type: 5 (single-bit ECC)
  Logical Cache Type: 5 (unified)
  Associativity: 5 (4-way set associative)
  Mode: 2 (varies by address)
  Location: 0 (internal)
  Flags: 0x1
        SMB_CAF_ENABLED (enabled at boot time)

Now smbios is incredibly useful, but it still doesn't give us a generic solution that works most of the time for the task at hand, so enter Perl, and getl2cachesize. Its short and sweet, but if your running stream it will give you a good starting point for your array sizes. A couple of other pointers, I'll leave compiler optimizations for your on reading, but as a warning stream is very variant, so get a lot of runs, generally we get twenty iterations, average that as a single run, reboot, and get at least five runs.




« July 2016