Wednesday Apr 27, 2011

Java SE 6 Update 25 has been released!

There's some pretty cool performance improvements in JDK 6 Update 25. Check it out and let us know what you think. http://www.oracle.com/technetwork/java/javase/downloads/index.html

Thursday Apr 07, 2011

More Performance World Records for Oracle Java

I'm pleased to announce several new SPECjbb2005 World Records for Oracle Java.

With the launch of the new Intel Xeon Processor E7 famliy, many joint Intel and Oracle partners have published record breaking results using Oracle Java. Congratulations to Cisco, Fujitsu, SGI, and Oracle Systems! Oracle Java now holds the world records on SPECjbb2005 for 2, 4, and 8 Socket servers! And thanks to SGI, we have a massive all around world record result of 32,643,008 SPECjbb2005 bops. Wow!

These results are the fruit of the labor of the Oracle Java Performance and JVM teams through our performance collaboration with Intel. Great work team!

Server Performance Summary from Intel
Press release for Oracle Systems result: 2,703,740 SPECjbb2005 bops



SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4470 M2 results submitted to SPEC. Other results as of 02/17/2011 on www.spec.org. Sun Fire X4470 M2 (4x Intel Xeon E7-4870 CPU, MS Windows, Oracle Java 6 u25) SPECjbb2005 bops = 2,703,740, SPECjbb2005 bops/JVM =675,935.

Thursday Feb 18, 2010

Dagastine -> Keenan

So who's David D. Keenan? Its me, the same Java performance dude, with a new last name. My full name is now David Dagastine Keenan.

Wednesday Jun 03, 2009

Come to my talks, Intel is raffling 2 Netbooks!

Come to TS-5254 at 9:45am or TS-7499 at 4:10pm, Intel is giving away 2 netbooks. All the more reason to come to the sessions!

Sunday May 31, 2009

Java One 2009: Come check out my performance sessions

This will be a busy Java One as I have two talks this year.  Both are on Wednesday June 3rd.

9:45 am:  TS-5254 SPEC Java Platform Benchmarks and Their Role in the Java Technology Ecosystem. 

This should be a cool talk.  I'm presenting with Anil Kumar from Intel Corporation, whom I'm worked with for a few years now on SPEC Java benchmark development.  We'll give an overview of each of the SPEC Java benchmarks, how each benchmark stresses JRE and systems performance, and highlight how active competition has influenced and improved the performance of the Java platform as a whole.

4:10 pm: TS-7499 Maximizing Java Technology-Based Application Performance on Multicore Platforms

This is another interesting talk that will highlight many of our most recent performance optimizations for multi-core platforms.  It will also go over the performance engineering process, analysis tools, and a neat demo.   I'm presenting with Kingsum Chow from Intel Corportation, this will be our third Java One presenting together. Kingsum is my counterpart at Intel for our Java performance collaboration, its been great working with him and I look forward to presenting together once again. 

Come check out our talks!  And don't hesitate to ask questions!



Monday Apr 06, 2009

Tree Wagers

There has been a lot of public comments about the modifications we've made to Harmony's java.util.TreeMap implementation.  We've been working on it since I announced our use of the code last summer.  Reliability and compatibility are our greatest concerns, and much of the last several months have been spent flushing out obscure compatibility concerns.  Of course, some of the time has been spent in legal reviews for open source contributions.  However, today is the day.

I am very happy to announce that after completing Sun's open source review process, along with many iterations of quality assurance and compatibility testing we are engaging the Harmony community to do as we said we would, give the modifications we made to java.util.TreeMap back to Harmony.

While the modifications made to this TreeMap meet the Java 6 specification there remains behavioral compatibility differences between the TreeMap from Harmony and the TreeMap implementation in OpenJDK. These inconsistencies are continuing to be discovered and addressed --- compatibility is very important to us!  However, it appears to be important to Harmony to receive the modifications immediately.  More important than spending the engineering time to deliver an implementation that is fully compatible, beyond the Java 6 specification, with Java 6 TreeMap or OpenJDK's TreeMap. Hence, we are giving these modifications to Harmony in their current form.

Last but not least, it seems its time for Tim to make his £500 donation to the Woodland Trust.  I expect it would be too much to ask that it be done on behalf of the Sun Java development team :-)   Either way, I expect a bit of head slapping and tree hugging in Hampshire, UK tomorrow.


Thursday Apr 02, 2009

Tim Ellison's misguided remarks

Tim Ellison of the Harmony project has attempted to discredit my blog post on Intel Nehalem.  I'm quite aware of IBM's stellar performance on a "similar" platform, however I wasn't at the liberty to comment on it until a public statement from IBM was made.  Now that has happened...

Congratulations to IBM J9!  Your SPECjbb2005 score is impressive and I look forward to beating it shortly :-)  No, seriously, you've done a great job.  However,  why don't you submit a SPECjvm2008 result?  Actually no company besides Sun has submitted a result!  Why is that?  From my perspective the benchmark is being avoided because no one can beat the Sun result.  If a JVM was truly the fastest on the planet it would step up and beat Sun at SPECjvm2008.  Same applies for single-JVM SPECjbb2005, looking at your current results a single-JVM run IBM J9 should look good.  Come on, make me eat my words.

Look for a blog about SPECjvm2008 on Nehalem in the next few days...

All of our performance optimizations for Intel's latest processor are found in the OpenJDK JVM now.  Sun HotSpot JVM is open source, you can find all the source here.  Sun Java 6 Updates and Performance releases are Sun's distribution of Open JDK.  All performance optimizations make their way into Open JDK, HotSpot optimizations have made it into Open JDK faster than the library changes.  The latest performance release will run on hs15, Open JDK runs on hs15.  To say that we're not open source is simply wrong.

Sun and Open JDK take reliability and stability very seriously.  It is the foundation of a vast majority of Java deployments out there today, let alone its use on the client.  Spouting complaints about how slow we integrate performance optimizations into Open JDK is misleading/misguided.  As for TreeMap, I have to say we're very close.  Its under-review for our outbound open-source contributions.  I look forward to announcing it in the weeks to come, and also look forward to Tim's £500 donation.  Its slow because we are diligent about overall performance impact, and strict when it comes compatibility.  SPECjbb2005 is small potatoes when considering reliability and stability.





Sunday Mar 29, 2009

Java Rocks Intel Nehalem

Today Intel announced its next generation server processors, the Xeon 5500 series, codename Nahalem.

This processor has been on our radar for performance optimizations since 2007, and the Sun and Intel Java VM and Performance teams have pushed the new chip to higher and higher levels.

Today I'm pleased to announce world class SPECjbb2005 results running Sun Java 6 Update 14 64-bit Server Performance Release on a Sun branded system with 2 Intel Xeon X5570 CPUs with a score of 556,882 SPECjbb2005 bops, 278,411 SPECjbb2005 bops/JVM.  Details on the system configuration will be available later this month.

This proves once again that Sun Java powered by the HotSpot Server VM is the fastest, most reliable, and most widely deployed,  open source Java Runtime in the world.  World class reliability and performance and still open source, we can beat the proprietary competition even though they can use all of our source code. 

World Class Performance on 2-Chip Systems: 556,882 SPECjbb2005 bops, 278,411 SPECjbb2005 bops/JVM.





Here's What we've done for Nehalem
  • 64-bit Compressed Oops Performance Optimization: 64-bit now surpasses 32-bit performance.
  • Optimized class libraries to reduce cache misses and decrease path length
  • NUMA-aware Garbage Collection
  • XML library Optimizations
  • SSE 4.2 Support
Congratulations to the Sun Java VM Technologies team and the Intel Collaboration Team!

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4450
results submitted to SPEC. Other results as of 03/30/2009 on www.spec.org
. Sun Branded Intel Xeon X5570 System (2 chips, 8 cores, Sun JDK 6u14-P) SPECjbb2005 bops = 556,882, SPECjbb2005 bops/JVM =278,411.

Monday Oct 27, 2008

Drinking the Cool-aid, and SPECjbb2005

A colleague of mine pointed me to a comment to BMSeer's  by Frank.  I was a bit late reading this comment,
so I'm going to respond with a posting of my own.

http://blogs.sun.com/bmseer/entry/specjbb2005_sun_sparc_enterprise_t5440



Here's the text from Frank's comment:
"Hi BMSeer,


I'm just confused with your statement saying T5440 is the best
specjbb2005 performance machine. In your table I can see total
throughput is 692736 and probably the highest number now. This
benchmark specifically for T5440 uses 32 JVMs and the performance (not
througput) is only 21648/JVM whic is the lowest number in your table.
By comparing to Dell R900 (6 core Xeom) the throughput is 508240 and
performance is 132917/JVM which means use only 4 JVMs. This benchmark
will give us two important information : CPU/Core performance and JVM
performance. Unfortunately you didn't put SUN Intel/AMD machine to be
compared here. However we know that SUN X4600M2 (8 chips, 32 cores)
will give 683542 and the performance is 85443/JVM by runnning 8 JVMs.
So it is very clear nothing so special with T5440 to run Java
application because it has the lowest performance/JVM and SUN JVM
performance on Intel/AMD is lower than Jrockit JVM which is used on
Dell R900 benchmark. The other important thing that we have to know, by
running more JVM means more memory will be needed. By looking at T5440
compared to Dell R900 (32 JVM vs 4 JVM), it is very clear that Intel
wins in Java performance benchmark (the most efficient) and Jrockit is
much faster than SUN JVM."

First, I'd like to thank Frank for his comment to BMSeer's blog.  It give me a chance to talk about Sun SPARC Enterprise T5440, and the Sun Fire X4450.

Now, the golden rule when smearing your competition or making competitive performance claims is to do your homework.  Frank didn't do his homework.

The Sun Fire X4450 is a 4-Socket Intel Xeon X7460 and is very similarly configured to the Dell R900 mentioned by Frank.  The have identical memory and CPU configurations, and are running the same number of JVMs, which more a less allows a head to head software performance comparision. The Sun result is a solid 5% faster than the Dell result.  Why?  The Sun result is running Sun Java 6 Update 6 Performance release, the Dell result is running Oracle JRockit.  

Dell PowerEdge R900: SPECjbb2005 bops = 508240, SPECjbb2005 bops/JVM = 127060

Sun Fire X4450: SPECjbb2005 bops = 531669, SPECjbb2005 bops/JVM = 132917

Seems that Frank's closing comment is a bit unfounded.  Perhaps another colleague from Dell could help Frank out.

Sun also published a single JVM result on the Sun Fire X4450 with a score of 448,262 SPECjbb2005 bops.  Using Frank's logic this would make Sun JDK 6 Update 6-P 3.37x faster than Oracle JRockit!  Clearly this is not a correct comparison, as is Frank's broken AMD v. Intel, Sun v. Oracle comparisons. 


Sun Fire X4450: SPECjbb2005 bops = 448,262, SPECjbb2005 bops/JVM = 448,262


Now as for the Sun SPARC Enterprise T5440 compared to the Dell PowerEdge R900, the proper comparison to gauge system throughput when comparing SPECjbb2005 multi-JVM result is to look at the SPECjbb2005 bops metric.  You're treading on thin ice when attempting to compare SPECjbb2005 bops/JVM using multi-JVM results.  When running multi-JVM the JVM count is used as a tunable configuration option, and the only reason 4-JVMs were chosen when running on the Dell R900 or the Sun Fire X4450 is because it was the fastest configuration.  The previous generation of the Intel Xeon MP ran with twice as many JVMs.  So, comparing the 4-Socket, 24-core Dell PowerEdge R900 to the 4-Socket, 32-core Sun SPARC Enterprise T5440, the Sun T5440 is 36% faster.  We plan to submit a single JVM result on the Sun Enterprise T5440 and it seems that we may need to push a little harder to get that out.   Stay tuned.

Sun Fire T5440: SPECjbb2005 bops = 692,736 SPECjbb2005 bops/JVM = 21,648


Last word.  Do your homework when trashing your competition.  You're making it to easy.

 


Disclosure Statement:


SPECjbb2005
Sun SPARC Enterprise T5440 (4 chips, 32 cores) 692736 SPECjbb2005 bops, 21648 SPECjbb2005 bops/JVM.   Dell R900 (4 chips, 24 cores) 508240 SPECjbb2005 bops, 127060 SPECjbb2005 bops/JVM. 
Sun X4450 (4 chips, 24 cores) 531669 SPECjbb2005 bops, 132917 SPECjbb2005 bops/JVM.
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of 10/27/08



Thursday Jul 10, 2008

First SPECjvm2008 Result Published!

The first ever SPECjvm2008 result has been published on the SPEC website. The Sun result stands alone, with no other hardware or software vendor stepping forward. This is the example of the confidence we have in HotSpot JVM and OpenJDK, and their abilities to strive in a broad range computing environments.

Also note that this is a SPECjvm2008 Base submission, with No JVM Tuning Required. SPECjvm2008 consists of 11 sub-components, some of which have multiple sub-test, such as crypto, scimark, and xml. As I've said before, this is a bear of a JVM benchmark, so improving the composite score requires significant improvement of its parts. Sun has not only submitted the first SPECjvm2008 result, but did so by submitting a SPECjvm2008 base submission only, hopefully setting the tone for competitive comparison moving forward.


Confident Performance. It has a nice ring to it.


Monday Jul 07, 2008

Apache Harmony: Thanks for the TreeMap Work!

I'd like to thank Apache Harmony for their JDK library performance efforts. We were given a tip that the Harmony folks were doing some interesting work with the TreeMap collection class, and low and behold they were. The work is surrounding fat TreeMap nodes where each node contains several TreeMap entries. This greatly improves in order traversal of the TreeMap entries, and since SPECjbb2005 traverses over a TreeMap I can see why they did this. Controlled measurements showed the "fat" TreeMap significantly faster with in order traversals, and improved our SPECjbb2005 score by a solid 3-5% depending on the platform.

Considering the potential performance gain we started the effort of porting the Apache Harmony TreeMap code to JDK 6. We ran into a few performance snags in our performance regression testing, but with a bit more work we were able to develop a solution that was able to realize the performance gains without any negative impact on other workloads. We are now making the necessary steps to give back our code changes to Apache Harmony.

The new TreeMap is included in JDK 6 Update 6 Performance Release which is available for download at http://java.sun.com/performance.

Thanks again Harmony! Keep the JDK performance optimizations coming.







JDK 6 Update 6 Performance Release is here!

I'm please to announce the release of JDK 6 Update 6 Performance Release on SPARC platforms. This is the latest of our performance releases and is the culmination of our optimization efforts over the last year. With this JDK Sun achieved many world records on SPECjappserver2004, SPECweb2005, SPECjbb2005, and the first ever SPECjvm2008 submission. Please try it for yourself, download it at http://java.sun.com/performance.

Where are the x64 bits? We believe in reliable performance, not just top line performance on targeted workloads. To achieve this we run a lot of tests, and our testing back end is often much longer than the development cycle for any given release. Long story short we found an issue that needed another build, so the x64 binaries will be delivered at a slight delay. Since the performance of an unreliable JDK is zero, we thought you'd understand. I'll update everyone once the x64 binaries are posted.

Finally, I'd like to thank the Apache Harmony project for their JDK optimizations efforts. No, you're not seeing things, I'm thanking Apache Harmony. More details to come in my next blog posting.




Tuesday May 06, 2008

SPECjvm2008 Is Here!

SPEC has release SPECjvm2008 and ....

Its Free!!

The new benchmark is the replacement to SPECjvm98, the first SPEC Java benchmark and the beginning of a family of SPEC Java Benchmarks including SPECjbb2005, SPECjappserver2004, SPECjms2007, SPECpower_ssj2008, and a bit of SPECweb2005.

SPECjvm2008 leverages a wide range of workloads including Scimark, Compilation, SunFlow, XML, Derby, Startup, and many more, and its a fine Java benchmark for competition and research. Download it and give it a try.

Congratulations to the development team and a big thanks to our many contributors from the community.

Sun Java on Intel Delivers Again!

Just in time for JavaOne, I'm pleased to announce two new SPECjbb2005 World Records on Sun Intel systems. The Sun Fire X4450, powered by Intel Xeon MP CPUs and Java SE 6 Update 6-P, now hold the 4 Chip Multi-JVM World Record and the Single JVM x86 world record.

World Record Performance on 4-Chip Systems running 8-JVMs: 464,355 SPECjbb2005 bops, 58,044 SPECjbb2005 bops/JVM.


World Record Performance on x86 Systems running a single JVM: 389,208 SPECjbb2005 bops, 389,208 SPECjbb2005 bops/JVM.




The single JVM result actually beats all single JVM results with 16 CPUs or less. Wow. Thanks again to the HotSpot JVM and Performance teams, great work guys. Thanks also to the fine engineers at Intel. Lets keep on trucking.


SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4450
results submitted to SPEC. Other results as of 05/6/08 on www.spec.org
. Sun Fire X4450 (4 chips, 16 cores, Sun JDK 6u6-p) SPECjbb2005 bops = 464355, SPECjbb2005 bops/JVM = 58044. SPECjbb2005 bops = 389208, SPECjbb2005 bops/JVM = 389208.

Monday May 05, 2008

JDK 6 Update 5-P is Alive!

I'm pleased to announce the release of our second performance release, JDK 6 Update 5-P. Its available to download at http://java.sun.com/performance. This is our fastest JDK to date, released just in time for JavaOne.

Thanks to the entire VM Technologies team!

Wednesday Apr 09, 2008

Java and CMT Triple Crown

Java, Solaris and CMT Systems have achieved a Java performance triple crown with winning results in 3 key industry benchmarks.

SPECjbb2005 2 CPU Multi-JVM World Record: Sun SPARC Enterprise T5240

SPECweb2005 World Record: Sun SPARC T5220

SPECjAppServer2004 World Record Single Application Server: Sun SPARC Enterprise T5240

All running the upcoming JDK 6 Update 6 Performance release.  Nice.

Java == Performance

Fast! Java on UltraSPARC T2 Plus

Today Sun announced the next generation chip multi-threading (CMT) systems with the powerhouse UltraSPARC T2 Plus Processor and again Java is ready.  We've spent the last several months optimizing the Sun JDK and HotSpot JVM for UltraSPARC T2 Plus and the results are impressive.  Today, I again have the privilege to announce another Sun World Record, this time taking the lead with the new Sun SPARC Enterprise T5240 achieving a World Record 2 Socket SPECjbb2005 Result of 373,405 SPECjbb2005 bops, 23,338 SPECjbb2005 bops/JVM.

  • 1.94x scaling with  2 x 1.4Ghz UltraSPARC T2 Plus compared to 1 x 1.4Ghz UltraSPARC T2
  • Faster than all 4-Socket SPECjbb2005 results from IBM, including the IBM System p 570 with 4 x 4.7Ghz Power 6 CPUs !
  • Faster than all 4-Socket SPECjbb2005 results from HP (Actually faster than all results from HP outside of their 64-way Itanium2 systems)


Wow.  The UltraSPARC T2 Plus and Sun Java powered by the HotSpot JVM are a sweet combination, and this result was made possible with the highly optimized Java SE 6 Update 6-P which will be available in July 2008.  Take a look at http://java.sun.com/performance for the latest performance release.

Many, many engineers (and dare I say a few managers) had a role delivering these results, and I'd like to personally thank the following folks whose passion and determination made this possible.  Thanks for all the hard work!

Paul Hohensee
Tom Rodriguez
Vladimir Kozlov
Brian Doherty
Charlie Hunt
Alejandro Murillo
Xiaobin Liu
Coleen Phillimore
Igor Veresov
Peter Kessler
John Coomes
Tony Printezis
Rob Strout
Dave Cox
Jim Melvin
John Pampuch
Kirill Soshalsky

SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC.  Other results as of 09/28/07 on www.spec.org.  T5240 results submitted to SPEC for review. 
Other results as of 04/09/2008 on www.spec.org.  Sun SPARC Enterprise T5240 (2 chip, 16 cores, 128 threads) 373,405 SPECjbb2005 bops, 23,338 SPECjbb2005 bops/JVM. IBM p570 (4 chips, 8 cores, 16 threads) SPECjbb2005 bops = 346,742, SPECjbb2005 bops/JVM = 86,686. 
IBM System x3650 (2 chips, 8 cores, 8 threads) SPECjbb2005 bops = 323,172, SPECjbb2005 bops/JVM = 80,793.  HP ProLiant DL360 G5 (2 chips, 8 cores, 8 threads) SPECjbb2005 bops = 301,321, SPECjbb2005 bops/JVM = 75330. Dell PowerEdge 2950 III (2 chips, 8 cores, 8 threads) SPECjbb2005 bops = 305,411, SPECjbb2005 bops/JVM = 76,353
Technorati Tags: , , , , , , , ,

Tuesday Mar 04, 2008

My New Role at SPEC

 I have been elected to chair of the SPEC OSG Java Subcommittee, which is responsible for Java benchmarks such as SPECjbb2005, SPECjappserver2004, and SPECjms2007 (plus a few in development).  The first month has been entertaining to say the least, but its been a lot of fun so far.  Thanks to everyone in OSG Java for helping me take this on.  I've got big shoes to fill and I can't help feeling intimidated from time to time.  The folks on the subcommittee are great and their patience and support have been a big help.  Thanks!




Powered by ScribeFire.

Intel Covers SPECjbb2005 Record Result

Intel has posted a "Chip Shot" about the Sun SPECjbb2005 World Record on Intel systems.  The text is a bit familiar, but heck, its coverage.

http://www.intel.com/pressroom/chipshots/chipshots.htm


Powered by ScribeFire.

Thursday Feb 28, 2008

SPECjbb2005 World Records live on SPEC Website

The SPECjbb2005 World Record results on Sun Intel systems are now live on the SPEC website.

World Record Performance on 2-chip x86 Systems running 4-JVMS: 303,297 SPECjbb2005 bops,  75,824 SPECjbb2005 bops/JVM

World Record Performance on x86 Systems with a Single JVM: 277,585 SPECjbb2005 bops

The Sun JDK powered by the HotSpot JVM is the the most widely deployed, scalable, and reliable JVM on the planet, but we need your help to continue to improve.  Please participate at OpenJDK and the performance forum at Java.net, let us know what your application needs for reliable performance.  You can even dive in and help.  Competitive benchmarks are fun, but at the end of the day its our customer's application that really matter, so let us know what we can do to help!


SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4150 results submitted to SPEC.  Other results as of 02/15/08 on www.spec.org.  Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM.  Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 277,585 SPECjbb2005 bops, 277,585 SPECjbb2005 bops/JVMDell 2950 III (2 chips, 8 cores, BEA JRockit 6.0 P27.4.0) 303,130 SPECjbb2005 bops , 75,783 SPECjbb2005 bops/JVM.




Powered by ScribeFire.

Friday Feb 22, 2008

What's Different about the Sun SPECjbb2005 Result on Intel?

Some rather ridiculous comments from "Rick Jones" on the Bmseer's blog prompted me to write this.   Rick stated:

"If that four JVM figure wasn't run over and over again,
one is left wondering just how bad it was before. Given
the same CPUs at the same frequency, one would expect
more from "major impact" performance changes."


We have made huge performance improvements over the last year.  Yes, our previous capabilities on the Intel platform were less than what we can do now.  I thought that would be obvious. 

But you know what the biggest difference with our SPECjbb2005 World Record?  Sun Hardware + Sun Solaris + Sun HotSpot JVM.  Its not a result from any old box vendor who runs with the latest and greatest OS and JVM and claims the result as their own.  This is a Sun World Record, through and through.  Who else can deliver the entire stack, its a short list isn't it?

Hmm, Think of the possibilities for customers.  Who do you call when you have a JVM issue?  Sun.  OS issue?  Sun. Hardware issue?  Sun.  What if you need a specific feature which requires a tailored approach from hardware to OS to JVM?  Sun.  Thats the difference.




Powered by ScribeFire.

Thursday Feb 21, 2008

SPECjbb2005 World Record Web Coverage

Our latest SPECjbb2005 World Record on the Sun Fire X4150 is getting some coverage on Sun web pages and blogs. 

As always, the bmseer has something to say, check out his post on the SPECjbb2005 World Record here.

The Sun marketing folks posted a page as well, check it out here.

We're very excited about these results, and hope our customers are too.  Let us know if you have any questions!







Powered by ScribeFire.

Friday Feb 15, 2008

Slam! SPECjbb2005 World Record on Intel!

It was one year ago that Sun and Intel kicked off a collaboration to optimize Solaris and the HotSpot JVM for Intel Xeon systems.  The last year has been a fun ride, working together we've made our performance steadily increase, and finally today is the day.

Without further ado,

Sun HotSpot JVM, Solaris, and the Sun Fire X4150 with 2 Intel X5460 quad-core processors now hold 2 SPECjbb2005 World Records!


World Record Performance on 2-chip x86 Systems
303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM running 4 JVMs!





World Record Performance on x86 Systems with a Single JVM
277,585 SPECjbb2005 bops, 277,585 SPECjbb2005 bops/JVM





Thats right, Sun is fastest on Intel DP, inching past BEA to take the lead.  Game on JRockit.


We submitted two results, one with 4 JVMs, and another with 1 JVM.  Why?  When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations.  With both single-JVM and multi-JVM results a more complete picture can be drawn of the performance capabilities a software/hardware stack.   If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results.  If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results.  I encourage my colleagues from other JVM vendors to submit both single JVM results and multi-JVM results.  We, and our competitors owe it to the Java community to show the whole truth about JVM performance.  It's the Java community which will benefit from the active competition and drive JVM development forward.  I've written about this before, take a look here for more background.



The JRE we used to achieve the world record results is Java 6 Update 5 Performance Release (Java 6 Update 5-p).  This is our second performance release, and will be available mid-May 2008.  Our first performance release Java 6 Update 4-p, is available on our performance page at java.sun.com/performance.  Java 6 Update 4-p is a step along the way to our current high scores, and is only ~10% behind Java 6 Update 5-p in the general case.  Go ahead, give it a try and let us know what you think.

Congratulations to the Sun and Intel Java performance collaboration team.  For the last year we've worked together to deliver some grand performance improvements.  Great work everyone, let's keep up the pace!


SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun Fire X4150 results submitted to SPEC.  Other results as of 02/15/08 on www.spec.org.  Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 303,297 SPECjbb2005 bops, 75,824 SPECjbb2005 bops/JVM.  Sun Fire X4150 (2 chips, 8 cores, Sun JDK 6u5-p) 277,585 SPECjbb2005 bops, 277,585 SPECjbb2005 bops/JVMDell 2950 III (2 chips, 8 cores, BEA JRockit 6.0 P27.4.0) 303,130 SPECjbb2005 bops , 75,783 SPECjbb2005 bops/JVM.



Tuesday Oct 09, 2007

US-T2 and HotSpot JVM: An Example of Scalability

Single JVM scaling has been ignored in the realm of competitive benchmarking.  Since the release of SPECjbb2005 and its support for multiple JVM configurations most JVM and hardware vendors have ignored the single JVM configuration.  Which begs my first question, "Why not submit using a single JVM?"  Simply put, SPECJbb2005 multi-JVM results tends to be significantly higher than single JVM results on the same hardware.  Which leads to my second question, "Why are SPECjbb2005 multi-JVM configurations faster than SPECjbb2005 single-JVM configurations?".  Now that's a good question.

Multi-JVM configurations tend to be faster for two general reasons.  First, you can run many 32-bit JVMs instead of a single 64-bit JVM.  32-bit JVMs are anywhere from 15 - 20% faster than true 64-bit JVMs when running memory bound compute intensive applications that can't leverage the large heap available with a 64-bit JVM.  Second, multi-JVM configurations allow the end user to bind or set affinity of a given process to a physical processor or cores.  For example, lets say a given processor runs very well when the SPECjbb2005 JVM process is kept on a single socket or a few cores on a socket, but suffers when a single SPECjbb2005 JVM process runs across many sockets or cores.  In this situation, it would be advantageous to run multiple JVMs each bound to a set of cores that are all or a subset of the cores available in a single CPU.

The example above is very prevalent in SPECjbb2005 results today.  Many, many vendors submit multi-JVM results, even on 1U and 2U servers with 2 sockets or less.  Sun has done the same.  Many of the processors available in 1U and 2U form factors have processors that show significantly different performance characteristics when comparing single-JVM and multi-JVM SPECjbb2005 configurations, and since multi-JVM is faster,  vendors  chose to publicize that result.  Until now...

When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations, and from this point forward I plan to advocate that Sun continue as it has done today and submit both single-JVM and multi-JVM configurations on new hardware platforms.   With both single-JVM and multi-JVM results a more complete picture of the performance capabilities a software/hardware stack can drawn.   If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results.  If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results. 

Unfortunately the competitive submissions on the SPECjbb2005 website are woefully incomplete with many vendors only submitting multi-JVM results.  I implore all vendors to submit SPECjbb2005 single-JVM results along with the multi-JVM results.  Sure they're not as fast, but at least you're not hiding behind the multi-JVM configuration, and then hardware and JVM vendors will work together to solve any problems.  Its the best decision for JVM innovation, our customers, and the Java platform.

Below is our most recent SPECjbb2005 single JVM submission on a Sun SPARC Enterprise T5120 powered by the UltraSPARC-T2 processor.  This result fully utilizes the 64 hardware threads available on the US-T2, up to 8X more threads than our 1U and 2U competition, with nearly picture perfect scaling to boot.  This result surpasses all single-JVM submission from HP and Dell, and is faster than all 8-core or less single-JVM submissions from IBM.  And just to sweeten it a bit more, this 64-bit result is only 13% behind the 32-bit 8-JVM SPECjbb2005 submission on the SE T5120 @ 192,055 SPECjbb2005 bops!





SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. 
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM.



Technorati Tags: , , , , , , ,

Powered by ScribeFire.

Java Screams on Sun UltraSPARC T2


The UltraSPARC T2 has arrived and the Sun HotSpot JVM is ready.  We've spent the last several months optimizing Sun's JVM for US-T2 and the day has finally come to share what we've been working on.

Two new SPECjbb2005 world records! 
  • Sun SPARC Enterprise T5120 and T5220 servers delivered a World Record SPECjbb2005 Single Socket result of 192,055 SPECjbb2005 bops, 24,007 bops/JVM.
    • 9% faster than IBM p570 with 2 x 4.7Ghz Power 6 Processors!
    • 4X less space, 2X less power!
  • Sun SPARC Enterprise T5120 and T5220 servers delivered a World Record Single JVM result for all systems with 8-cores or less.
    • Beats all single JVM results from Dell and HP!
Record Breaking Perfomance

The Sun SPARC Enterprise T5120 and  Sun SPARC Enterprise T5220 servers equipped with a single UltraSPARC T2 processor at 1.4GHz, delivered a World Record single-chip result of 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM.  An average of 464 Watts of power was used to obtain this result.  This result beats the previous SPECjbb2005 single-chip world record  by 71%.






Record Breaking Scalability

The Sun SPARC Enterprise T5120 and Sun SPARC Enterprise T5220 servers delivered a World Record  single-chip, single-JVM result of 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM.    This result beats all single-JVM SPECjbb2005 results from Dell and HP, and all 8-core or less single-JVM results from IBM.  To top it off, the  64-bit single-JVM SPECjbb2005 result is within 13% of the performance of the 32-bit multi-JVM result of 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM, highlighting the flexibility of the Ultra SPARC T2 and Sun HotSpot JVM technology.





Reliable Performance: Sun's HotSpot JVM


Rock solid reliable performance is at the heart of HotSpot JVM development at Sun.  Our dedicated effort has allowed us to deliver performance optimizations into JDK 6 update releases in a highly reliable fashion, with our first ever JDK 6 performance release coming your way in December 2007.   Stay tuned for more information on the performance release and all of the cool optimizations we're working on for all of our supported platforms.


SPEC Disclosure Statement

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC.  Other results as of 09/28/07 on www.spec.org.  Sun SPARC Enterprise T5220 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM.  Sun SPARC Enterprise T5220 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM. IBM p570 (2 chips, 4 cores) SPECjbb2005 bops = 175474, SPECjbb2005 bops/JVM = 87737.  Dell 2950 (2 chips, 8 cores) SPECjbb2005 bops = 148,436, SPECjbb2005 bops/JVM = 74,218. HP rx2660 (2 chips, 4 cores) SPECjbb2005 bops = 158,174, SPECjbb2005 bops/JVM = 39,544. IBM p505Q (1 chip, 4 cores) SPECjbb2005 bops = 63544, SPECjbb2005 bops/JVM = 31772. HP rx2660 (2 chips, 4 cores) SPECjbb2005 bops = 80884, SPECjbb2005 bops/JVM = 80884



Technorati Tags: , , , , , , , ,

Powered by ScribeFire.

Wednesday Sep 19, 2007

Sun HotSpot JVM Talk at Intel IDF

I'll be presenting tomorrow at Intel Developer Forum in San Francisco with an esteemed colleague from Intel, Kingsum Chow.

The session is SSGS003:  "How to Get the Most Performance from Sun\* JVM\* on Intel® Multi-Core Servers".

Take a look at this link for more information, just search by last name.

The conference is rather interesting.  I've run into a lot of Sun folks working on Solaris, and I've seen many folks our competitors in blue.  I've looked for Henrik and friends, but haven't found them yet.  Interesting, considering all the marketing babble I expected to see them here. 

Here's what we hope you'll walk away with from my talk tomorrow (straight from the abstract):
  •  An overview of how Sun\* JVM\* has evolved to take advantage of current and future Intel® Multi-Core processor-based servers since Sun and Intel teams started working on JVM performance improvements in early 2007
  •  A good understanding how harnessing the best Intel® Xeon® processor features with Sun JVM will start producing immediate benefits
  •  A quick overview of Intel Multi-Core architectures 
  •  An understanding of how Sun JVM takes advantage of Intel® architectures on three major operating systems (Linux\*, Windows\*, Solaris\*) 
  •  A look at how to select JVM parameters for your application to get the most performance, and at employing VM parameters tuning as a last resort 
  •  A few common options for a list of application classes








Powered by ScribeFire.

Tuesday May 08, 2007

New SPECjbb2005 World Record!

The 72-way Sun Fire E25K with 1.80Ghz US-IV+ processors and powered by Java SE 6 Update 2 now holds the Single Instance SPECjbb2005 World Record with a result of 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM.

This result surpasses the previous high score held by Fujitsu PowerQuest 580 running BEA Jrockit and Itanium 2 processors by 4%. To top it off, this result was accomplished without using the fastest US-IV+ processors available (the fastest available is 1.95Ghz!) This makes the Sun Fire E25K running Solaris 10 and Java SE 6 Update 2 the fastest, most scalable computing platform for Java applications on the planet!

SPECjbb2005 Performance (ordered by performance bops : SPECjbb2005 Business Operations per Second, bigger is better)

System Date Processors Performance
(Chips, Cores, Threads) GHz Type bops JVMs bops/JVM
Sun Fire E25K 5/07 72, 144, 144 1.80 US-IV+ 1,149,100 1 1,149,100
Fujitsu PowerQuest 580 3/07 32, 64, 128 1.6 Itanium 2 1,105,465 1 1,105,465
Azul 7280B 1/07 16, 768, 768 ? 872,972 1 872,972
Fujitsu PP2500 1/06 128, 128, 128 2.08 SPARC64 V 811,607 1 811,607

Sun results have been submitted to SPEC for review and are on track for publication.

Disclosure Statement:

SPECjbb2005 Sun Fire E25K (72 chips, 144 cores, 144 threads, 1.95 GHz) 1,149,100 SPECjbb2005 bops, 1,149,100 SPECjbb2005 bops/JVM submitted for review; Fujitsu PQ580 (32 chips, 64 cores, 128 threads, 1.6 GHz) 1,105,465 SPECjbb2005 bops, 1,105,465 SPECjbb2005 bops/JVM; Azul 7280B (16 chips, 768 cores,768 threads, ? Ghz) 872,972 SPECjbb2005 bops, 872,972 SPECjbb2005 bops/JVM; Fujitsu PP2500 (128 chips, 128 cores, 128 threads 2.08 GHz) 811,607 SPECjbb2005 bops, 811,607 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 5/8/07 on www.spec.org

High Performance Java Technology in a Multi-core World

Paul Hohensee and I are doing a session this week at Java One, TS-2885: High Performance Java Technology in a Multi-core World. We'll talk about current and upcoming multi-core computer architectures ,JVM technologies and optimizations , and application and JVM tuning . Finally I'll do a quick demo showing our recent performance optimizations coming soon in Java SE 6.0_02, identification of heap tuning opportunities and common application bottlenecks.

Come see our talk on Thursday at 4:10pm in Gateway 102/103. See you at Java One!

Tuesday Dec 19, 2006

Update: Java 6 Leads Out of Box Server Performance

Based on feedback from my esteemed colleagues and readers I've updated this entry with a table of results. The tables are at the bottom after the charts.

Java 6 is finally here. Its our fastest, most reliable release and specifically targets out-of-box performance. What does this mean? Simply put it means no tuning options are needed for the JVM to achieve optimal performance. Looking at the bigger picture it means much more. No longer will you spend hours pouring over cryptic JVM tuning parameters to determine the optimal configuration for your application. No more expensive re-qualification of your application for special command line tuning. Java 6 makes performance tuning easy.

Java 6, powered by the recently open-sourced HotSpot JVM is impressive. Here's a summary:

  • On SPECjbb2005 the numbers are impressive. Java 6 out of the box is more than 40% ahead of the competition on Intel Core, and 30% ahead on AMD Opteron.
  • On Scimark Java 6 continues to show solid performance leading the performance of the competition by more than 40%.
  • On Volano, Java 6 improves performance by more than 20% over the most recent update of the JDK 5.
The out of box performance of a Java application is an intriguing and difficultengineering problem. The requirements of client and server applications couldn't be more different. On one hand client apps want fast startup and low footprint, on the other hand server applications want highly optimized code, throughput and low pause times; while both want reliability and compatibility.

Out of box performance is the right goal for JVM development, and future Java benchmarks should reflect that goal. Delivering optimizations quickly to allow high benchmark results is fun but it doesn't help customers unless they become part on the default runtime behavior of the JVM.

Just to be clear and to reiterate once again, the intention of the data charts below is to highlight the importance of customer experience and out-of-box performance to Sun Java Engineering. These are not meant to be high performance benchmark results. Hand tuning can change the results significantly.

The following is an out-of-box performance comparison on a Dell 2950 and a Sun Fire X4200. The Dell system is configured with 2 dual-core Intel 5160 processors (2 CPUs, 4 cores @ 3.0Ghz) and 16GB of RAM. The Sun system is configured with 2 dual-core Opteron 280 processors (2 CPUs, 4 cores, 2.4 Ghz) and 8GB of RAM. The Operating System installed on both systems is Red Hat EL 4.0 AS Update 4. The kernel version is unmodified from the base install, which is 2.6.9-42.ELsmp. The only variable in this configuration is the JVM.

The JVM distributions and versions tested were the latest versions publicly available at the time of testing. The BEA JRockit JVMs tested are downloaded from their main GA website and their 64-bit performance update website. The IBM JVM is the latest available on the IBM developer website.

  • Sun JDK 1.5.0_10
    • 32-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) Server VM (build 1.5.0_10-b03, mixed mode)
    • 64-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_10-b03, mixed mode)
  • Sun Java SE 6 build 98
    • 32-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) Server VM (build 1.6.0-rc-b99, mixed mode)
    • 64-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-b105, mixed mode)
  • IBM JDK 5.0 SR3
    • 32-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxi32dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20061001 (JIT enabled) J9VM - 20060915_08260_lHdSMR JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002
    • 64-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxa64dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 j9vmxa6423-20061001 (JIT enabled) J9VM - 20060915_08260_LHdSMr JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002
  • BEA JRockit 5.0_06 R26.4
    • 32-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-ia32, )
    • 64-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build P26.4.1-12-67782-1.5.0_06-20061003-1620-linux-x86_64, )
As stated above and in the title no JVM tuning options were used for these results. The results below are statistical comparisons. No less than 10 samples were performed, and a T-test (single-tailed) was used to ensure confidence in the result. The data is normalized to the 32-bit IBM JDK 5 SR3 result.

The first set of charts reflect performance on Intel's latest Core 2 micro-architecture. The results below, particularly the SPECjbb2005 results, strongly highlight a core difference in philosophy between Sun HotSpot and its competitors. If you look at highly tuned competitive submissions of our competitors, BEA JRockit in particular, have impressive numbers on the new chip. Our competitors have chosen to quickly deliver platform specific performance optimizations for the purpose of benchmark submissions but require the use of several tuning parameters to achieve that level of performance. Unfortunately this is quite misleading for customers. Yes, the benchmark numbers are good, but can a customer jump right in and use these features? If they were thoroughly tested and ready for prime time shouldn't they be enabled by default on the platforms that require them? We think so, and we have chosen differently, and thats the difference with HotSpot.

The first chart is SPECjbb2005. SPECjbb2005 is SPEC's benchmark for evaluating the performance of server side Java. It evaluates server side Java by emulating a three-tier client/server system (with emphasis on the middle tier). It extensively stresses Java collections, BigDecimal, and XML processing. The cool thing about SPECjbb2005 is that optimizations targeted for it also show performance gains in other competitive benchmarks, such as SPECjappserver2004, and a broad range of customer workloads. The benchmark results below are run in single instance mode.

SciMark 2.0 is a Java benchmark for scientific and numerical computing and is a benchmark where Sun's JVMs have continued to shine. Its a decent test of generated code, particularly for tight computational loops. However it is particularly sensitive to alignment issues and can show some level of variance from run to run, mostly in a bimodal fashion. The test has three modes of exectution; small, large, and default. This is the size of the data under test, more details can be found at the scimark website. All in all its a good set of microbenchmarks.

Note that the 32-bit JVMs in all cases are faster than the 64-bit JVMs when running on the Intel Core system. This is quite different than the AMD Opteron system further down the page where 64-bit is significantly faster. Since the Scimark 2.0 test is using the large dataset, its likely that the added pressure of 64-bit pointers on the memory subsystem increases bandwidth enough to impede performance, however this is just a hypothesis.

Volano is a popular Java chat server. The benchmark is quick and involves both a client and server instance. From a JVM perspective the workload is heavily dominated by classic Java socket I/O which is a bit long in the tooth, an NIO version would be quite interesting. That being said, some customers have found this benchmark quite useful so we continue to test it, however it is by no means our favorite benchmark as my friends at BEA have suggested. Running Volano the performance gaps are not as large, most likely because this benchmark has very little garbage collection overhead. BEA JRockit is showing good performance here with a result thats 19% over the baseline. Sun Java SE 6 shines as well with a result thats nearly 22% over baseline.

The second set of charts are run on a Sun Fire X4200 with AMD Opteron 280 CPUs. This is the identical system used in my previous blog articles on this subject, this time with updated JVM releases from Sun and IBM. I'm sure someone will be curious why I didn't compare the Intel and AMD based systems directly. The primary reason is simple, I'm writing about JVM performance, not CPU performance. That being said, I didn't have the latest AMD CPUs readily available. In short, Intel is faster when running some of these benchmarks, while AMD is faster on others. In general the memory subsystem differences between these platforms is prevalent when comparing the performance of Java benchmarks. Sun Java 6 is showing impressive results running SPECjbb2005 with a result 30% over baseline and ~15% faster than J2SE 5.0_10.

Scimark 2.0 is impressive on AMD Opteron as well. The large dataset is an interesting workload as its effect on cache can highlight memory subsystem limitations. If your application crunches on a large dataset, take a look at the large dataset of Scimark when comparing JVMs and system architectures.

Last but not least is Volano on AMD Opteron (and again, no this is not our favorite benchmark!). Java 6 shows a strong improvement of with results more than 20% greater than 5.0_10, pulling ahead of 64-bit BEA JRockit. Nice.

SPECjbb2005 Result Disclosure
Single Instance Run. SPECjbb2005 bops = SPECjbb2005 bops/JVM
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit SPECjbb2005 bops 64-bit SPECjbb2005 bops
IBM 5.0 SR3 43,575 32,617
BEA JRockit 5.0_06 R26.4 26,071 26,092
Sun J2SE 5.0_10 49,308 46,080
Sun Java SE 6 62,246 56,488

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.
JVM Version 32-bit SPECjbb2005 bops 64-bit SPECjbb2005 bops
IBM 5.0 SR3 30,500 23,998
BEA JRockit 5.0_06 R26.4 19,309 19,185
Sun J2SE 5.0_10 35,297 31,096
Sun Java SE 6 39,973 34,975

SciMark 2.0 Result Disclosure
Large Dataset. Score is in SciMark MFlops
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 171.49 207.21
BEA JRockit 5.0_06 R26.4 278.15 276.37
Sun J2SE 5.0_10 321.85 292.89
Sun Java SE 6 357.72 336.58

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.
JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 175.02 180.46
BEA JRockit 5.0_06 R26.4 230.85 231.53
Sun J2SE 5.0_10 300.23 332.23
Sun Java SE 6 320.42 343.74

VolanoMark 2.5.0.9 Result Disclosure
Loopback performance test
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 121,747 111,826
BEA JRockit 5.0_06 R26.4 128,185 146,012
Sun J2SE 5.0_10 120,048 116,959
Sun Java SE 6 149,198 142,602

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.
JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 64,218 60,802
BEA JRockit 5.0_06 R26.4 73,627 76,675
Sun J2SE 5.0_10 66,955 64,316
Sun Java SE 6 80,592 75,156

SPEC(R) and the benchmark name SPECjbb(TM) are trademarks of the Standard Performance Evaluation Corporation. Competitive benchmark results stated above reflect experiments performed by Sun Microsystems, Inc. For the latest SPECjbb2005 benchmark results, visit http://www.spec.org/osg/jbb2005.

Sunday Dec 10, 2006

Java 6 Leads Out of the Box Server Performance

Java 6 is finally here. Its our fastest, most reliable release and specifically targets out-of-box performance. What does this mean? Simply put it means no tuning options are needed for the JVM to achieve optimal performance. Looking at the bigger picture it means much more. No longer will you spend hours pouring over cryptic JVM tuning parameters to determine the optimal configuration for your application. No more expensive re-qualification of your application for special command line tuning. Java 6 makes performance tuning easy.

Java 6, powered by the recently open-sourced HotSpot JVM is impressive. Here's a summary:

  • On SPECjbb2005 the numbers are impressive. Java 6 out of the box is more than 40% ahead of the competition on Intel Core, and 30% ahead on AMD Opteron.
  • On Scimark Java 6 continues to show solid performance leading the performance of the competition by more than 40%.
  • On Volano, Java 6 improves performance by more than 20% over the most recent update of the JDK 5.
The out of box performance of a Java application is an intriguing and difficult engineering problem. The requirements of client and server applications couldn't be more different. On one hand client apps want fast startup and low footprint, on the other hand server applications want highly optimized code, throughput and low pause times; while both want reliability and compatibility.

Out of box performance is the right goal for JVM development, and future Java benchmarks should reflect that goal. Delivering optimizations quickly to allow high benchmark results is fun but it doesn't help customers unless they become part on the default runtime behavior of the JVM.

Just to be clear and to reiterate once again, the intention of the data charts below is to highlight the importance of customer experience and out-of-box performance to Sun Java Engineering. These are not meant to be high performance benchmark results. Hand tuning can change the results significantly.

The following is an out-of-box performance comparison on a Dell 2950 and a Sun Fire X4200. The Dell system is configured with 2 dual-core Intel 5160 processors (2 CPUs, 4 cores @ 3.0Ghz) and 16GB of RAM. The Sun system is configured with 2 dual-core Opteron 280 processors (2 CPUs, 4 cores, 2.4 Ghz) and 8GB of RAM. The Operating System installed on both systems is Red Hat EL 4.0 AS Update 4. The kernel version is unmodified from the base install, which is 2.6.9-42.ELsmp. The only variable in this configuration is the JVM.

The JVM distributions and versions tested were the latest versions publicly available at the time of testing. The BEA JRockit JVMs tested are downloaded from their main GA website and their 64-bit performance update website. The IBM JVM is the latest available on the IBM developer website.

  • Sun JDK 1.5.0_10
    • 32-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) Server VM (build 1.5.0_10-b03, mixed mode)
    • 64-bit: java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_10-b03, mixed mode)
  • Sun Java SE 6 build 98
    • 32-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) Server VM (build 1.6.0-rc-b99, mixed mode)
    • 64-bit: java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-b105, mixed mode)
  • IBM JDK 5.0 SR3
    • 32-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxi32dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20061001 (JIT enabled) J9VM - 20060915_08260_lHdSMR JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002
    • 64-bit: Java(TM) 2 Runtime Environment, Standard Edition (build pxa64dev-20061002a (SR3) ) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 j9vmxa6423-20061001 (JIT enabled) J9VM - 20060915_08260_LHdSMr JIT - 20060908_1811_r8 GC - 20060906_AA) JCL - 20061002
  • BEA JRockit 5.0_06 R26.4
    • 32-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-ia32, )
    • 64-bit: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build P26.4.1-12-67782-1.5.0_06-20061003-1620-linux-x86_64, )
As stated above and in the title no JVM tuning options were used for these results. The results below are statistical comparisons. No less than 10 samples were performed, and a T-test (single-tailed) was used to ensure confidence in the result. The data is normalized to the 32-bit IBM JDK 5 SR3 result.

The first set of charts reflect performance on Intel's latest Core 2 micro-architecture. The results below, particularly the SPECjbb2005 results, strongly highlight a core difference in philosophy between Sun HotSpot and its competitors. If you look at highly tuned competitive submissions of our competitors, BEA JRockit in particular, have impressive numbers on the new chip. Our competitors have chosen to quickly deliver platform specific performance optimizations for the purpose of benchmark submissions but require the use of several tuning parameters to achieve that level of performance. Unfortunately this is quite misleading for customers. Yes, the benchmark numbers are good, but can a customer jump right in and use these features? If they were thoroughly tested and ready for prime time shouldn't they be enabled by default on the platforms that require them? We think so, and we have chosen differently, and thats the difference with HotSpot.

The first chart is SPECjbb2005. SPECjbb2005 is SPEC's benchmark for evaluating the performance of server side Java. It evaluates server side Java by emulating a three-tier client/server system (with emphasis on the middle tier). It extensively stresses Java collections, BigDecimal, and XML processing. The cool thing about SPECjbb2005 is that optimizations targeted for it also show performance gains in other competitive benchmarks, such as SPECjappserver2004, and a broad range of customer workloads. The benchmark results below are run in single instance mode.

SciMark 2.0 is a Java benchmark for scientific and numerical computing and is a benchmark where Sun's JVMs have continued to shine. Its a decent test of generated code, particularly for tight computational loops. However it is particularly sensitive to alignment issues and can show some level of variance from run to run, mostly in a bimodal fashion. The test has three modes of exectution; small, large, and default. This is the size of the data under test, more details can be found at the scimark website. All in all its a good set of microbenchmarks.

Note that the 32-bit JVMs in all cases are faster than the 64-bit JVMs when running on the Intel Core system. This is quite different than the AMD Opteron system further down the page where 64-bit is significantly faster. Since the Scimark 2.0 test is using the large dataset, its likely that the added pressure of 64-bit pointers on the memory subsystem increases bandwidth enough to impede performance, however this is just a hypothesis.

Volano is a popular Java chat server. The benchmark is quick and involves both a client and server instance. From a JVM perspective the workload is heavily dominated by classic Java socket I/O which is a bit long in the tooth, an NIO version would be quite interesting. That being said, some customers have found this benchmark quite useful so we continue to test it, however it is by no means our favorite benchmark as my friends at BEA have suggested. Running Volano the performance gaps are not as large, most likely because this benchmark has very little garbage collection overhead. BEA JRockit is showing good performance here with a result thats 19% over the baseline. Sun Java SE 6 shines as well with a result thats nearly 22% over baseline.

The second set of charts are run on a Sun Fire X4200 with AMD Opteron 280 CPUs. This is the identical system used in my previous blog articles on this subject, this time with updated JVM releases from Sun and IBM. I'm sure someone will be curious why I didn't compare the Intel and AMD based systems directly. The primary reason is simple, I'm writing about JVM performance, not CPU performance. That being said, I didn't have the latest AMD CPUs readily available. In short, Intel is faster when running some of these benchmarks, while AMD is faster on others. In general the memory subsystem differences between these platforms is prevalent when comparing the performance of Java benchmarks. Sun Java 6 is showing impressive results running SPECjbb2005 with a result 30% over baseline and ~15% faster than J2SE 5.0_10.

Scimark 2.0 is impressive on AMD Opteron as well. The large dataset is an interesting workload as its effect on cache can highlight memory subsystem limitations. If your application crunches on a large dataset, take a look at the large dataset of Scimark when comparing JVMs and system architectures.

Last but not least is Volano on AMD Opteron (and again, no this is not our favorite benchmark!). Java 6 shows a strong improvement of with results more than 20% greater than 5.0_10, pulling ahead of 64-bit BEA JRockit. Nice.

SPECjbb2005 Result Disclosure
Single Instance Run. SPECjbb2005 bops = SPECjbb2005 bops/JVM
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit SPECjbb2005 bops 64-bit SPECjbb2005 bops
IBM 5.0 SR3 43,575 32,617
BEA JRockit 5.0_06 R26.4 26,071 26,092
Sun J2SE 5.0_10 49,308 46,080
Sun Java SE 6 62,246 56,488

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.
JVM Version 32-bit SPECjbb2005 bops 64-bit SPECjbb2005 bops
IBM 5.0 SR3 30,500 23,998
BEA JRockit 5.0_06 R26.4 19,309 19,185
Sun J2SE 5.0_10 35,297 31,096
Sun Java SE 6 39,973 34,975

SciMark 2.0 Result Disclosure
Large Dataset. Score is in SciMark MFlops
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 171.49 207.21
BEA JRockit 5.0_06 R26.4 278.15 276.37
Sun J2SE 5.0_10 321.85 292.89
Sun Java SE 6 357.72 336.58

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.
JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 175.02 180.46
BEA JRockit 5.0_06 R26.4 230.85 231.53
Sun J2SE 5.0_10 300.23 332.23
Sun Java SE 6 320.42 343.74

VolanoMark 2.5.0.9 Result Disclosure
Loopback performance test
System: Dell 2950, 2 X Intel 5160 (2 CPUs, 4 cores @ 3.0Ghz), 16GB of RAM.

JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 121,747 111,826
BEA JRockit 5.0_06 R26.4 128,185 146,012
Sun J2SE 5.0_10 120,048 116,959
Sun Java SE 6 149,198 142,602

System: Sun Fire X4200, 2 X AMD Opteron 280 (2 CPUs, 4 cores @ 2.4Ghz), 16GB of RAM.
JVM Version 32-bit Score 64-bit Score
IBM 5.0 SR3 64,218 60,802
BEA JRockit 5.0_06 R26.4 73,627 76,675
Sun J2SE 5.0_10 66,955 64,316
Sun Java SE 6 80,592 75,156

SPEC(R) and the benchmark name SPECjbb(TM) are trademarks of the Standard Performance Evaluation Corporation. Competitive benchmark results stated above reflect experiments performed by Sun Microsystems, Inc. For the latest SPECjbb2005 benchmark results, visit http://www.spec.org/osg/jbb2005.
About

dagastine

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today