Tuesday Jun 23, 2009

Sun Studio 12 Update 1

Sun Studio 12 Update 1 went live yesterday. It's still a free download, and it's got a raft of new features. Many people will have been using the express releases, so they will already be familiar with the improvements.

It's been about two years since Sun Studio 12 came out, and the most obvious change in that time is the prevalence of multicore processors. I figured the easiest way to discern this would be to look at the submissions of SPEC CPU2006 results in that time period. The following chart shows the cummulative number of SPEC CPU2006 Integer speed results over that time broken down by the number of threads that the chip was capable of supporting.

Ok, the first surprising thing about the chart is that there's very few single threaded chips. There were a few results when the suite was launched back in 2006, but nothing much since. What is more apparent is the number of dual-thread chips, that was where the majority of the market was. There were also a number of quad-thread chips at that point. If we fast-forward to the situation today, we can see that the number of dual-thread chips has pretty much leveled off, the bulk of the chips are capable of supporting four threads. But you can see the start of a ramp of chips that are capable of supporting 6 or 8 simultaneous threads.

The relevance of this chart to Sun Studio is that Sun Studio has always been a tool that supports the development of multi-threaded applications. Every release of the product improves on the support in the previous release. Sun Studio 12 Update 1 includes improvements in the compiler's ability to automatically parallelise codes - afterall the easiest way to develop parallel applications is if the compiler can do it for you; improvements to the support of parallelisation specifications like OpenMP, this release includes support for the latest OpenMP 3.0 specification; and improvements in the tools and their ability to provide the developer meaningful feedback about parallel code, for example the ability of the Performance Analyzer to profile MPI code.

Footnote SPEC and the benchmark names SPECfp and SPECint are registered trademarks of the Standard Performance Evaluation Corporation. Benchmark results stated above reflect results posted on www.spec.org as of 15 June 2009.

Tuesday Nov 11, 2008

New SPEC CPU search programme announced

SPEC has announced the search programme for the follow up to the CPU2006 benchmark suite. They are looking for compute intensive codes, real production apps, not microkernels or artificial benchmarks, that will cover a broad range of subject domains. They need to have both the code and workloads. There are financial rewards for completing the various selection hurdles.

Friday Jul 18, 2008

SIPEW 2008 conference slides online

Bunch of slides from the SIPEW 2008 conference are now up on line. Interesting one using performance counters to look at the correspondence between SPEC CPU2006 and a selection of commercial workloads.

Tuesday May 27, 2008

CPU2006 monitor wrapper for instrumentation

The monitor_wrapper infrastructure in SPEC CPU2006 is one of the best kept secrets of the suite. It allows you to run the benchmarks under various analysis tools (such as spot). With the X.X release of the kit, the monitor_wrapper infrastructure has finally been documented. An example of using it to run spot is:

monitor_wrapper=/opt/SUNWspro/extra/bin/spot -o${benchmark}.${size} ${command}

Which runs the command under spot, and puts the result in a directory which has the name benchmarkname.workloadsize. I also used the monitor_wrapper infrastructure to gather the data necessary for the papers on the correspondence between the training and reference workloads.

Thursday Mar 27, 2008

SPEC CPU2006 discussions

I recently read a couple of posts about SPEC CPU2006. As you can tell from the papers linked on this blog, I was quite busy helping prepare the suite - which was considerable fun. The first post is by Tom Yager, where he praises the suite for raising awareness of the components of a system that actually contribute to performance: "I added a practical angle to my scientific understanding of compiler optimizations, processor scheduling, CPU cache utilization".

On the other hand Neil Gunther (second time I've mentioned him) condemns "bogus SPECxx_rate benchmarks which simply run multiple instances of a single-threaded benchmark". I hope he's joking, but taking his comments at face value...

Interestingly he suggests SPEC SDM as a good choice. I'd not heard of this suite, but reading up on it it looks like it tests the impact of multiple users typing and executing commands on the system at the same time, and it's not been touched in over 10 years. I guess SDM would be a good match for the SunRay that I use daily, but I'm certain that the suite doesn't include the 22 copies of firefox that I currently see running on the server I'm using. On only slightly less rocky ground he talks about TPC-C, which appeared in 1992!

CPU2006 represents the CPU intensive portion very well, but deliberately tries of avoid hitting the disk or network. Since disk and network do play significant roles in system performance, I probably wouldn't recommend getting a machine purely on its SPECcpu2006 or SPECcpu2006_rate scores. However, the mix of apps in the benchmark suite is representative of most of the codes that are out there (and I believe some are codes that appeared less than 10 years ago;). So what ever app is being run on a system, there is probably a code in CPU2006 which is not that dissimilar.

Tackling his core beef with the suite that the rate metric "simply" runs multiple copies of the the same ecode, this is actually harder work for the system than running a heterogeneous mix of applications. So I'd suggest that it is a better test of system capability than running some codes that stress memory bandwidth together with some other codes that are resident in the L1 cache. So IMO far from being "bogus", specrate is a very good indicator of potential system throughput.

Wednesday Apr 11, 2007

Papers on CPU2006

The March issue of Computer Architecture News has a section on the SPEC CPU2006 suite. SPEC has conveniently put together a set of the submitted papers. There are 12 papers, of which 8 are at least co-authored by Sun folks.


Darryl Gove is a senior engineer in the Solaris Studio team, working on optimising applications and benchmarks for current and future processors. He is also the author of the books:
Multicore Application Programming
Solaris Application Programming
The Developer's Edge
Free Download


« February 2017
The Developer's Edge
Solaris Application Programming
OpenSPARC Book
Multicore Application Programming