US-T2 and HotSpot JVM: An Example of Scalability

Single JVM scaling has been ignored in the realm of competitive benchmarking.  Since the release of SPECjbb2005 and its support for multiple JVM configurations most JVM and hardware vendors have ignored the single JVM configuration.  Which begs my first question, "Why not submit using a single JVM?"  Simply put, SPECJbb2005 multi-JVM results tends to be significantly higher than single JVM results on the same hardware.  Which leads to my second question, "Why are SPECjbb2005 multi-JVM configurations faster than SPECjbb2005 single-JVM configurations?".  Now that's a good question.

Multi-JVM configurations tend to be faster for two general reasons.  First, you can run many 32-bit JVMs instead of a single 64-bit JVM.  32-bit JVMs are anywhere from 15 - 20% faster than true 64-bit JVMs when running memory bound compute intensive applications that can't leverage the large heap available with a 64-bit JVM.  Second, multi-JVM configurations allow the end user to bind or set affinity of a given process to a physical processor or cores.  For example, lets say a given processor runs very well when the SPECjbb2005 JVM process is kept on a single socket or a few cores on a socket, but suffers when a single SPECjbb2005 JVM process runs across many sockets or cores.  In this situation, it would be advantageous to run multiple JVMs each bound to a set of cores that are all or a subset of the cores available in a single CPU.

The example above is very prevalent in SPECjbb2005 results today.  Many, many vendors submit multi-JVM results, even on 1U and 2U servers with 2 sockets or less.  Sun has done the same.  Many of the processors available in 1U and 2U form factors have processors that show significantly different performance characteristics when comparing single-JVM and multi-JVM SPECjbb2005 configurations, and since multi-JVM is faster,  vendors  chose to publicize that result.  Until now...

When using SPECjbb2005 to compare software and hardware configurations, it's critical to run and compare both single-JVM and multi-JVM configurations, and from this point forward I plan to advocate that Sun continue as it has done today and submit both single-JVM and multi-JVM configurations on new hardware platforms.   With both single-JVM and multi-JVM results a more complete picture of the performance capabilities a software/hardware stack can drawn.   If your application is horizontally distributed, but is configured to run 1 JVM per system, then look at the single JVM results.  If you plan to configure each system with processor sets and/or core affinity and will run multiple JVMs, look at the multi-JVM results. 

Unfortunately the competitive submissions on the SPECjbb2005 website are woefully incomplete with many vendors only submitting multi-JVM results.  I implore all vendors to submit SPECjbb2005 single-JVM results along with the multi-JVM results.  Sure they're not as fast, but at least you're not hiding behind the multi-JVM configuration, and then hardware and JVM vendors will work together to solve any problems.  Its the best decision for JVM innovation, our customers, and the Java platform.

Below is our most recent SPECjbb2005 single JVM submission on a Sun SPARC Enterprise T5120 powered by the UltraSPARC-T2 processor.  This result fully utilizes the 64 hardware threads available on the US-T2, up to 8X more threads than our 1U and 2U competition, with nearly picture perfect scaling to boot.  This result surpasses all single-JVM submission from HP and Dell, and is faster than all 8-core or less single-JVM submissions from IBM.  And just to sweeten it a bit more, this 64-bit result is only 13% behind the 32-bit 8-JVM SPECjbb2005 submission on the SE T5120 @ 192,055 SPECjbb2005 bops!





SPEC Disclosure Statement
SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Sun SPARC Enterprise T5120 results submitted to SPEC. Other results as of 09/28/07 on www.spec.org
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 170,153 SPECjbb2005 bops, 170,153 SPECjbb2005 bops/JVM. 
Sun SPARC Enterprise T5120 (1 chip, 8 cores) 192,055 SPECjbb2005 bops, 24,007 SPECjbb2005 bops/JVM.



Technorati Tags: , , , , , , ,

Powered by ScribeFire.

Comments:

Post a Comment:
Comments are closed for this entry.
About

dagastine

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today