New SPECjAppServer2004 813.73 JOPS result using Glassfish 2 and Postgres

Sun SPECjAppServer2004 813.73 JOPS@Standard result improves performance and lowers price (again) using Glassfish V2 and Postgres 8.2 DB on T2000.



Sun has submitted a new benchmark result of 813.73 SPECjAppServer2004 JOPS@Standard which improves on the performance of the recent 778 JOPS number and at a reduced price for the tested system. The improvements are due to better database performance and a huge boost to the application server performance which enabled the number of application server machines in the tested configuration to be reduced from 3 to 2.

As I suggested in my comments regarding the recent Sun 778 JOPS result, at Sun we are in the early stages of working with open source communities such as the PostgreSQL community so really we have only just started to demonstrate  how far these open source products will scale and how much money they can save your organisation.


Highlights : -

  • Approximately 4% better performance compared to the earlier 778 JOPS number

  • Decreased the price of the system.
    By using Glassfish V2, we were able to reduce the cost of the system by about $US8K as we only needed 2 application server machines instead of 3.
    The total purchase cost of the tested configuration is now only $US 57, 425

  • Improved price/performance from $US 84.98/JOPS to $US 70.57/JOPS (see below for full pricing of this result)

  • All open source software result so the purchase cost of the software (operating system, application server and database) is zero!

  • This system is supporting more than 6000 simulated concurrent users
    note that SPECjApp2004 run rules ensure that during the test there is no (significant) single point of failure so these virtual users are running in a simulated production quality environment. E.g. log disks and database are mirrored and striped, the StorageTek 2540 array has battery backed cache in case of power fail etc.

Cost comparison against HP result (costing details for HP result)









 

 

 

 

Pricing calculations for Sun 813.73 JOPS result

  • I have used the bill of materials (bom) from the benchmark result to price this result.

  • All hardware prices come directly from the Sun US website as at 20th July 2007
    go to http://www.sun.com/servers , choose a server and use the “get it” tab to be taken to the Sun store where you will find the pricing information

  • I have omitted minor items from the pricing like a keyboard etc

  • The pricing doesn't include support but if it did the result would look even better compared to results using commercial (non open source) software!



 

SPEC required disclosure : -

SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of 22nd Aug 2007

All comparisons are based on the SPEC SPECjAppServer2004JOPS@Standard metric from www.spec.org or on pricing made using the bill of materials included in each SPECjAppServer2004 result
Sun 813.73 SPECjAppServer2004 JOPS@Standard (Sun Fire X4200, 4 chips / 8 cores, T2000 1 chip / 8 cores)
HP 874.17 SPECjAppServer2004 JOPS@standard (rx2660 , 4 cores, 2 chips)



Comments:

I don't have a lot of trust in this investigation because it is an internal investigation. You could have taken your best configuration against the worst configuration of your competitors. Why haven't you tested with PostgreSQL on HP? Or with Oracle on Sun?

Posted by Erik van Ingen on August 21, 2007 at 05:04 PM PDT #

This analysis is with PUBLISHED results from "http://spec.org". HP has not published with PostgreSQL.

Posted by Glenn Fawcett on August 22, 2007 at 01:11 AM PDT #

Have you tried using an x4200 as the database server instead of the T2000? Would X2200 app servers be sufficient instead of the X4200s? What appears to be the current bottleneck on the database tier?

Posted by Bill Hathaway on August 22, 2007 at 03:45 AM PDT #

That's the real question. It seems to me that at this juncture, the DB tier isn't the bottleneck here, but it's the app servers, correct?

The new benchmark hardware is very similar to the old GFv1 benchmark, specifically on the DB tier, but they got better performance with one less AS machine and the new software.

So, was the DB tier saturated at this point, or were the appservers? Was the goal this time "how fast can we go on this hardware" or "can we do the same thing we did before with less"?

What if you used the 4500 instead of the T2000 and the StorageTek?

Either way, it's still a heck of a system.

It would be great to see the entire software stack used (save the actual SPEC code, which is the only Non-OSS piece here). Specifically the Postgres builds, the GF build, any patches from the mainline, and all of the OS, AS, and DB tweaks.

Posted by Will Hartung on August 22, 2007 at 05:59 AM PDT #

G'Day Eric,

Further to what Glenn has already pointed out this is an industry standard benchmark and indeed the results are published just click on the links in the post to go to the www.spec.org site or go to the footnote to go directly to the result. Also please have a look at the result because it has lots of tuning information, in fact the idea is that there should be enough information in the result so allow a 3rd party to run the test and achieve the same score. Also a small plug for the work we do at SPEC. There is a version of SPECjAppServer2004 available called EStress that is very cost effective (about $US200 I believe) and you can purchse this and try very similar tests for yourself. You can't compare EAStress and SPECjAppServer results but you can use it to do sizing and performance tuning. I hope you will see that this is far from internal testing.

regards
Tom Daly

Posted by Tom Daly on August 22, 2007 at 02:13 PM PDT #

G'Day Bill / Will,

One of the rules of the SPECjAppServer benchmark is that benchmark submitters and SPEC members can't quote benchmark scores unless they have been submitted and accepted by SPEC as meeting the run rules. This is one of the many quality measures that help to make SPECjAppServer numbers useful and hopefully high quality. So for this reason I can't talk about what scores we have achieved with the Postgres DB running on X2100 or X2200 etc. I can however refer you to the published MySQL number we did which was 720.56 JOPS and I have written some details about this on this blog see the post at http://blogs.sun.com/tomdaly/entry/pricing_for_latest_sun_glassfish

Will, you are correct , in this particular test the Postgres DB is not the bottleneck we need a more than 2 appserver machines to get the Postgres DB running to it's maximum, especially as we are continuing to see improvements in Postgres performance. This result was intended to show great price/performance and demonstrate the performance improvements of Glassfish V2.

As for wanting to see the entire software stack, well once again the SPEC rules and process ensure this is all available to you. Please go ahead and have a look at the result on the SPEC web site it contains all of the configuration information including all of the software and hardware details for the result. This is is a great source of tuning information for Glassfish and Postgres!

regards
Tom

Posted by Tom Daly on August 22, 2007 at 08:09 PM PDT #

Post a Comment:
Comments are closed for this entry.
About

tomdaly

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today