Monday Feb 15, 2010

SPECtacular awards - Java

Last December SPEC released the SPECjEnterprise2010 benchmark, the third generation of Java Enterprise Edition performance tests from the Standard Performance Evaluation Corporation (SPEC) consortium. Steve RealmutoThe new benchmark tests Java EE 5.0's significantly expanded and simplified programming model, with a realistic workload stressing the entire system including JVM, middleware, Database, CPU, disk and servers. Yet although the benchmark test is much broader, it is simpler than ever to run because it takes advantage of Java EE 5.0 features and because it uses the open source Faban general purpose driver.

One of the most enjoyable duties of SPEC President is thanking the people from all the member institutions who make SPEC's success possible. I fear that it is all too easy for SPEC people's achievements to miss recognition, in our environment where their successes are most visible to their competitors rather than to their own management. So each January at the annual meeting we present awards to SPECtacular contributers. And now I write here to give them a bit of public recognition. I start this year with award recipients from the Java committee. Awards were presented by Alan Adamson (on right in photo, presenting award to Steve Realmuto). Alan is a SPEC Awards Committee member, member of the Board of Directors, and former chair of the Java committee.

Akara Sucharitakul - Oracle

The silent partner. Akara developed Faban, made it available to SPEC, and implemented new features needed to facilitate its use in SPECjEnterprise2010, so that the benchmark can be broader in scope and still be simpler to run.

Anil Kumar - Intel

The greenie. Although SPECjEnterprise2010 version 1.0 does not include an energy metric, the code is there thanks to Anil, awaiting adoption of suitable run and reporting rules.

Anoop Gupta - Oracle

The quiet achiever. Anoop seldom took part in the committee's sometimes raucous debates, because he was busy working on the code, making sure the workload is correct and correctly balanced.

Bernhard Riedhofer - SAP

Mr Specification. Berhnard speaks very quietly and politely , but the development group learned early on that when Bernhard speaks , it pays to listen. Of his many code contributions, those to make the database loader run truly parallel are going to be most appreciated by folks running large submissions.

David Keenan – Oracle

The chair. The job of chairing one of SPEC's largest and sometimes fractious committees is not an easy one, particularly while running results review for 4 benchmarks plus development of two new benchmarks. David combines a soft touch with firm determination to get the job done.

John Stecher - IBM

The closer. At then end when "only" a few tough action items stood in the way of benchmark release, John got additional resources committed from IBM to "get this thing done," pitching in for plenty of the closing work himself too.

Rahul Biswas - Oracle

Mr WebServices. Rahul provided most of the web services code, plus the code and ant scripts to integrate the benchmark into the Faban harness.

Robert Wisniewski - IBM

The reporter. Rob wrote the reporter code and built the test kits. He also served as secretary: taking good notes is vital to an open development process of multiple (competing) vendors.

Saraswathy Narayan - Oracle

The architect. Sara took the time to make a deep study of the entire benchmark from a transactional and data flow perspective, ensuring the correctness and function of the database schema and benchmark data partitioning.

Steve Realmuto - Oracle

The chief of police. Steve contributed much to the organisation of the benchmark development effort, and helped the team follow SPEC policies. He was editor of the run rules, driving the review and ensuring correctness.

Tom Daly – Oracle

The instigator. Tom was a relentless leader in driving the project forward, and a tireless worker in helping to push at every stage. The benchmark became a much richer and diverse test of Java middleware because of Tom’s influence.

Congratulations to these outstanding engineers, and to the entire SPEC Java team!

Friday Apr 11, 2008

autonomous robots come to San Diego

The International Autonomous Robotics Competition is coming to San Diego in June, at the Del Mar Fairgrounds! Thanks to the San Diego Java Users Group and Wintriss Technical Schools, kids can compete in building and programming robots. The kids will use Sun SPOT which are - of course - the open source robot tool kits. They'll program the robots in Java. Eric Arseneau writes all about the robot competition. I think my boy is a bit young to be writing software, but he's taken me by surprise many times before. Contest or not, he wants me to get him a robot kit, which he thinks I must be able to pick up in the office any day.


Thursday Feb 21, 2008

SPEC award recipients, Web Server

More SPECtacular awards given at SPEC's 2008 annual meeting in San Francisco, these to members of the Web Committee. As before, I won't post anyone's name without permission, but you know who you are and SPEC is grateful for your contributions.

It's a bit more complicated to tell you what these engineers did to earn this recognition. In 2007 SPEC released a maintenance version 1.20 to SPECweb2005 which included a lot of improvements. Most notable to me are new code that extends the level playing field of comparison, and tightened standards compliance rules that ensure real world applicability.

In order to focus on performance of the web server, to simplify the configurations tested and so reduce the cost of benchmarking, SPECweb2005 uses a simulated back-end database, BESIM, instead of the database server typically found with real world web applications. The great majority of the computing load is, by design, on the web server. Therefore the database component being synthetic code has not been an issue with fair comparison across platforms. However we discovered that when using an ISAPI implementation, two Ethernet packets were exchanged per HTTP response instead of the single packet with other implementations. No such results had been published, so there were no unfair comparisons - yet. But the problem needed to be fixed in order that ISAPI solutions could be measured on a level playing field. The solution was to adapt BESIM to use ISAPI v2.0 interface based on the Windows IIS implementation (with permission from Microsoft), so now different solutions behave alike, and they behave like real world web servers.

Another potential risk to fair comparison was in the Java Server Pages (JSP) which is typically used in benchmark results in preference to other slower scripting languages like PHP. We added a requirement to the run rules that software products prove their standards compliance by passing the relevant test suite. In the case of JSP that means that web servers which say they implement JSP must really implement JSP, not a subset needed to run the benchmark but omiting some slower features needed to implement real world web applications. Again, we don't know of any published results which did gain a performance advantage by cutting corners on standards conformance. Now that the rules are tightened we know there won't be any.

Thursday Feb 14, 2008

SPEC award recipients, Java

More SPECtacular awards given at SPEC's 2008 annual meeting in San Francisco, these to members of the Java committee. As before, I won't post anyone's name without permission, but you know who you are and SPEC is grateful for your contributions.

SPEC's Java committee maintains, reviews, and publishes benchmark results for SPECjvm98, the first industry standard benchmark of Java client performance; SPECjAppServer2004, the benchmark of Java application server performance; SPECjbb2005, the benchmark of Java server performance; and the newest SPECjms2007, the benchmark of Java Messaging Service performance.

SPECjvm is the only pre-Y2K benchmark in that list. Think how much CPU performance has improved and how much the Java platform has grown in scope from 1998 to today, and it's clear that SPECjvm98 is due for update which should be announced this quarter. Messaging is an increasingly crucial part of the Internet infrastructure, and of enterprise applications, but until the release of SPECjms2007 there was no standardized way to quantify the performance of various solutions.

Sam Kounev of the Technical University of Darmstadt, Germany, was cited for his leadership as chair of SPEC's JMS team in driving and bringing SPECjms2007 to market. Key developers on the project included Kai Sachs from TU Darmstadt, Lawrence Cullen from IBM, and his team of messaging developers at IBM Hursley laboratory, UK, including Tim Dunn and Martin Ross. An engineer from Sun, Silicon Valley, was cited for specification and design of the benchmark and the run and reporting rules.

Stefan Sarne of BEA Systems, Stockholm, and Evgeniya Maenkova from Intel, St. Petersburg, Russia, were cited for coding, testing, and finalizing the new Java client benchmark.

Leading such a large committee with so many projects and so many company interests has not been an easy job, and we were fortunate in 2007 to have a very capable engineer from Intel as our Java chairman.  He always did what he saw was right, putting SPEC's interests foremost.  He is moving on to other assignments this year, and will be terribly missed at SPEC.

Friday Jan 04, 2008

more on 64-bit Linux

I've found good information by searching blogs, but never before had the information just come to me - in the form of comments on my blog. Thanks to everyone who offered advice! Mikael Gueck's fix sounds intriguing, but after blowing away one installation I'm hesitant to mess with the 64-bit libraries.

 Georgez is right that I'm not at all doing a feature to feature comparison of Solaris and Linux. I'm just trying to use my PC. And yes, my problems are mostly due to proprietary software.You guessed right that the VPN software is Cisco. I just hadn't wanted to single them out for criticism. It's their right to decide not to open source their software, and to decide which platforms to support and not to support. And yes, it was their decision not to support Solaris that moved me onto punchin, for which I'm grateful because I like it a lot better. I'll investigate vpnc to see if it's compatible with the VPN servers on the other end. I guess it ought to be if they're both implementations of a standard protocol (ipsec), but sometimes there's a gap between theory and practice.

I did try installing the non-Adobe flash (gnash) and the non-Sun Java plugin (Icedtea). Gnash plays youtube videos but the controls don't work; I have to close the tab or window to make it stop. I haven't made any Java applet work at all. I haven't looked yet inside Sun to see what the problem(s) are with providing 64-bit plugin support, and when I do I won't be able to write about it, assuming anything inside to be company confidential. But since Java is open source I have to think somebody must have gotten the idea they don't need to wait for Sun, and all the development process is open, so I bet that over in the Java forums I'll find lots of public information about the issues.

Perhaps the most promising answer to my 64-bit problems is a question. Dmitri asks why anyone would need a 64-bit browser. I'm pretty sure I don't. That's just what Ubuntu offered to install. But what about running a 32-bit Firefox on 64-bit Ubuntu? I don't see any such thing looking in Synaptic, or on 

Searching around I found this posting by Rhaurison. He runs not only Firefox in 32-bit mode but also Opera, Skype, and other applications. He avoids conflicts by keeping 32-bit applications in a chroot environment, and points to this posting by Crad explaining how to set up such an environment.

When/if I try it I'll write about it. And if you've already done it, please post a comment. I love it when good information just comes right to me. Thanks again, people!

Linux 64-bit woes

13 years ago I switched from 32-bit Solaris to 64-bit and everything just worked. So when I installed Ubuntu 7.10 on my home PC I never considered installing anything except the "best" 64-bit version. As I noted, most things work, even if you have to beat the package manager into submission, insisting that 64-bit x86 is upwards compatible with 32-bit x86. But you have to be really careful with that --force-architecture flag because Linux doesn't seem to allow multiple versions of system libraries, and if you overwrite a critical library routine with an incompatible version, you're hosed.

It turns out that amd64 compatibility on Linux has long been a hot topic, leading some people to debate the merits of falling back to 32-bit Linux. Flash player won't work. Java browser plugin and web start won't work. (I'm embarrassed by that one!) My VPN client software won't work. As someone commented about flash on a discussion forum:
Stop saying something is available on Linux when what you mean is that it is available on Linux, with Intel processor, in 32-bit mode.
But wait, isn't open source supposed to remove all compatibility issues? Well, after compiling a lot of packages I'd add to the advice above, please say if you mean it works on Red Hat distro only, or if you mean kernel 2.6.9 or later, or if you mean uni-processor only.

I found a great workaround for flash. Simply install the Windows version of Firefox using wine. On Ubuntu 7.10 I didn't even have to use the command line instructions listed in the howto. The installer just made a desktop icon for me, which I renamed from "Firefox" to "Windows Firefox" and I just click it to run. I haven't yet found a workaround for the Java plugin, which prevents me from using many web applications like WebEx.

Like flash, the VPN software is proprietary closed source code, so compatibility updates are under sole control of the owner. Using Solaris on the same PC the VPN worked flawlessly. Open source Solaris includes punchin implementing ipsec. So rather than wait for the proprietary VPN vendor to decide amd64 is worthy of their time to do a compilation, I might do better to investigate Linux implementations of ipsec.

So for things I want to use at home, Linux is looking less "open" than Solaris. (See Jim Laurent's comparison.) But I still love Ubuntu!

Monday Oct 29, 2007

SPEC releases Java Message Service benchmark

SPEC has released SPECjms2007, so there is now an industry standard metric of messaging performance for middleware. Congratulations to the team from vendors, academia, and the open source community who put in long hard work to make this benchmark a reality, including Technische Universität Darmstadt (Germany), IBM, Sun Microsystems, Oracle, BEA, Sybase and the Apache Software Foundation!



I am a software engineer in San Diego, president of the Standard Performance Evaluation Corporation (, formerly a mathematician and a violist.


« April 2014