Solaris 11 provides smooth, scalable performance on SPECjbb 2013

Oracle released SPEC Benchmark results for the T5-2 and X2-4 processor using the SPECjbb 2013 benchmark. Who would be interested in SPECjbb performance? According to SPEC:

The SPECjbb2013 benchmark has been developed from the ground up to measure performance based on the latest Java application features. It is relevant to all audiences who are interested in Java server performance, including JVM vendors, hardware developers, Java application developers, researchers and members of the academic community. 

Jeff Victor has posted an excellent comparison of the T5 SPECjbb performance to our competitors on a per core basis.  To me, the charts tell the biggest part of the story,  Oracle's Solaris 11 on both SPARC and X86 shows smooth scaling with excellent response times over a wide range of transaction counts.

First, let's look at the results for the SPARC T5-2 server with 2 CPU sockets and 32 cores.  The vertical access marks "response time" so a lower number is better.  The horizontal axis is the number of Java operations being performed.  The blue dots indicate the median response time at each level of operations being processed.  Notice how Solaris 11 and the SPARC hardware provide smooth, predictable performance up through 60,000 jOPS.

(Note: You may not be able to see the full chart width on this page.  Right-click and open image in new tab to see the full chart.) 

T5-2 Chart

 Now let's look at Oracle's X2-4 Intel based system also running Solaris 11.  The X2-4 has 4 CPU chips with 40 total cores.  Here Solaris 11 also provides smooth scaling of performance.

X2-4 chart

For comparison, I've also selected HP's most powerful Intel based server the DL980 with 8 CPUs and 80 cores.  This system, however is running Red Hat Enterprise Linux 6.3.  On this chart you will see that RHEL 6 takes a dive in median response time shortly after 27,000 jOPS. Response time drops from 10 milliseconds to 100 milliseconds at around 27,000 jOPS.  Oracle's T5-2 stays below 100 milliseconds all the way to about 62,000 jOPS. Also note how the minimum response times fall apart at around 20,000 jOPS where the T5-2 stays consistent through 57,000 jOPS.

While admittedly, the 80 core DL980 reaches a higher total MaxjOPS throughput number than the 32 core T5-2, the Solaris 11 based system provides smoother scalability in a 2 socket system that requires only three rack units of space.  If that's not enough horsepower, we also offer a T5-4 and T5-8 system.  Need more?  Our M5-32 data center server scales to 32 sockets, 192 cores and 1536 threads. The M5-32 also supports up to 32 TB of RAM. All support our no cost Logical Domains virtualization capability.

HP DL980 Chart

Summary:

 If you want a proven, enterprise class, scalable OS for SPARC (from Oracle or Fujitsu) or X86 based platforms (from Oracle or many third party vendors), choose Solaris 11.  Predictability in response time is important to your enterprise customers.

All Oracle servers under Premier Support for systems include:

  • 7 x 24 on-site hardware support
  • Solaris (SPARC or X86), Oracle Linux (x86 only) and Oracle VM support (SPARC or X86)
  • Integrated Lights out Management
  • Oracle Enterprise Manager Ops Center support 

For more information on recent SPARC T5 world records, see https://blogs.oracle.com/BestPerf/.

  • SPEC and the benchmark name SPECjbb are registered trademarks of Standard Performance Evaluation Corporation (SPEC). Results as of 4/22/2013, see http://www.spec.org for more information.
  • SPARC T5-2 75,658 SPECjbb2013-MultiJVM max-jOPS, 23,334 SPECjbb2013-MultiJVM critical-jOPS. Sun Server X2-4 65,211 SPECjbb2013-MultiJVM max-jOPS, 22,057 SPECjbb2013-MultiJVM critical-jOPS. Sun Server X3-2 41,954 SPECjbb2013-MultiJVM max-jOPS, 13,305 SPECjbb2013-MultiJVM critical-jOPS. SPARC T4-2 34,804 SPECjbb2013-MultiJVM max-jOPS, 10,101 SPECjbb2013-MultiJVM critical-jOPS. HP ProLiant DL560p Gen8 66,007 SPECjbb2013-MultiJVM max-jOPS, 16,577 SPECjbb2013-MultiJVM critical-jOPS. HP ProLiant ML350p Gen8 40,047 SPECjbb2013-MultiJVM max-jOPS, 12,308 SPECjbb2013-MultiJVM critical-jOPS. Supermicro X8DTN+ 20,977 SPECjbb2013-MultiJVM max-jOPS, 6,188 SPECjbb2013-MultiJVM critical-jOPS. HP ProLiant ML310e Gen8 12,315 SPECjbb2013-MultiJVM max-jOPS, 2,908 SPECjbb2013-MultiJVM critical-jOPS. Intel R1304BT 6,198 SPECjbb2013-MultiJVM max-jOPS, 1,722 SPECjbb2013-MultiJVM critical-jOPS.

Comments:

I have difficult in understanding these charts.
What's, for instance, 90-th percentile?
Also, in "Also note how the minimum response times fall apart at around 20,000 jOPS where the T5-2 stays consistent through 57,000 jOPS." I can't see where in the chart you see this.
Please, help us better understand your point so we can offer support.
Thanks!

Posted by guest on April 25, 2013 at 09:47 AM EDT #

Guest,

Let me answer your questions with two examples. In the first chart (T5-2/Solaris) see the green triangle (90th percentile) at the intersection of the vertical line at 30,000 jOPS and the horizontal line for 1000 microseconds. This means that while the system is processing 30,000 Java Operations per second, 90% of all operations take 10 milliseconds or less (9.7 milliseconds to be exact.) At the SPEC Site, you can right-click on the chart to view the actual numerical source for the charts.

By comparison, the third chart (HP/RHEL) at 30,000 jOPS has a green triangle at about the 269 millisecond line. At the same level of Java Operations being process, 90% of the HP/RHEL ops take over 27 times longer than the T5/Solaris system.

The vertical scale is logarithmic so the actual drop off would appear MUCH more severe if it were charted linearly.

The blue dots represent the median result which means that 50% of the transactions take longer and 50% of the transactions take less time than marked by the blue dot.

My point of the blog is that RHEL 6 performance drops of dramatically (as measured by longer response times) at a very early point in the scale up of the testing (before 30,000 jOPS) when compared to Solaris on either SPARC or Intel based systems.

Posted by Jim Laurent on April 25, 2013 at 01:17 PM EDT #

Why not also do this test with Solaris 11 on the DL980 ? The DL980 is a complex x86 archtecture yet there are no mention of any hardware optimization (DIMM configuration, PCI option cards placement, BIOS configs, NUMA awareness, power mgmt, etc). I'd be interested in knowing if it was fine tuned and optimized or simply tested straight out of the box without any tuning.

Posted by CharlesQc on June 05, 2013 at 10:02 AM EDT #

CharlesQc,

Industry standard benchmarks are resource and time intensive. They are funded by marketing departments to make specific points about our products. It costs money to acquire the hardware, run the tests, get them audited and posted on the SPEC site. For this reason, it's unlikely that Oracle will publish results on an HP DL 980 or that HP will publish results using Solaris. We tend to only publish results that we know are superior.

You can assume that each system is configured for optimum performance by the vendors. No one wants to publish a sub-optimal benchmark result. Compiler options are documented in the benchmark test reports.

Posted by Jim Laurent on June 05, 2013 at 11:42 AM EDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Jim Laurent is an Oracle Sales consultant based in Reston, Virginia. He supports US DoD customers as part of the North American Public Sector hardware organization. With over 17 years experience at Sun and Oracle, he specializes in Solaris and server technologies. Prior to Oracle, Jim worked 11 years for Gould Computer Systems (later known as Encore).

Search

Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today