Thursday Nov 06, 2008

Sun Demonstrates Compelling Price Performance

Sun demonstrates compelling price/performance advantage of enterprise supported Open Source.

Using Glassfish V2 U2 / MySQL 5.0/ OpenSolaris on commodity Sun Intel hardware (i.e. SunFire X4150 Intel based servers)

Sun has achieved a result of 1197.50 SPECjAppServer2004JOPS@Standard , you can see a lot more detail on this result on the Sun benchmarks page and also a nice write up by the bmseer and also checkout the Sun ISV blog here

The Summary :







Why this is interesting/important to organisations running web applications.

  1. Demonstrates the huge cost savings available from using Open Source software for the entire application stack !

    The table and graphs above highlight an important fact. The Sun solution for hardware and software costs less than one CPU's worth of database license as used on the Dell or HP results and this isn't true just of the results above but in fact true of most of the SPECjAppServer2004 submissions that use proprietary (application server or database) software. You can check out the details of the pricing here

  2. Demonstrates Open Source solution competing directly against proprietary solutions

    One very useful and important feature of the SPECjAppServer2004 benchmark is that it is written in java and this makes it possible to run the same code unchanged on a variety of different operating systems and hardware platforms, in fact there is a benchmark rule that prohibits any code changes being made to the SPECjAppServer2004 java code. This means that no matter what platform , j2ee application server, operating system or database the application is run on the application itself is unchanged. This characteristic makes SPECjAppServer2004 and ideal workload for making comparisons between  different environments and Sun is utilising this fact to provide a comparison between the proprietary software based Dell and HP results with the an all open source based result. In addition to running the same java code and hence accessing the same database data, the open source results must fulfill the benchmark requirements for enterprise products such as , correctness, support, durability ACID compliance etc.

    Of course there are limitations with all benchmarks and comparisons must be made carefully, however this result clearly demonstrates open source doing the same job as proprietary but at a fraction of the cost. This coupled with the continuing performance improvements I mention in 3. below seem to be adding weight to Allan Packers thoughts on the topic of the future of proprietary and open source databases.

  3. Demonstrates continuing and outstanding performance improvements in the Open Source stack and in the MySQL DB in particular

    In fact it turns out that the Sun result of 1197.10 JOPS demonstrates the best published “commodity hardware” result for per core database performance. The Sun benchmark uses just 4 Intel cores on one chip in the X4150 database server and this means that the combination of OpenSolaris/MySQL 5 and X4150 delivers per core JOPS of 299.28 JOPS/database core. The only results that demonstrate superior per core performance for the database are some IBM power architecture chips (which are definitely not commodity priced).  If you want to verify this claim please download the spreadsheet of results available via the search feature at http://www.spec.org/jAppServer2004/results/

    Also it turns out that the MySQL database requires far less disk resources that competing products, this is demonstrated by the fact that the internal disks of the X4150 were used to support the database throughput and only 6 of them in a raid 0+1 partition. If you look through the SPECjAppServer2004 results you will see most vendors need to use expensive disk arrays to achieve the required disk performance (and most likely expensive extra database options).

  4. Just the start of performance improvements for MySQL , Glassfish and Open Solaris

    This result is based on the first release/update of OpenSolaris and uses MySQL version 5.0. and the performance of both of these platforms is going to improve in the near future for instance with  MySQL 5.0 we simply have not yet seen  the results of the impressive gains the Sun Performance Technology team in conjunction with the open source community and open source focussed companies such as Google are bringing to MySQL.  We will see these performance gains start to turn up with MySQL 5.1 and beyond (see Allan Packers article on this).  Also stayed tuned as the performance of Glassfish is going to continue to impress especially on commodity 4 and 8 core servers used in conjunction with MySQL. In fact I can't wait to start testing GF V3.

  5. Because this benchmark result can be a pointer to how your organisation can start to save “serious” money by using enterprise supported Open Source software

    Basically you might want to start investigating what web applications in your organisation can run on an enterprise supported open source stack such as the glassfish/MySQL/OpenSolaris stack used for this benchmark result.
    There are some steps you can start talking now and I am going to detail a strategy for doing just this in an upcoming blog.


Required disclosure : SPEC and SPECjAppServer are registered trademarks of Standard Performance Evaluation Corporation. Results from www.spec.org as of 11/05/2008. 2xSun Fire X4150 (8 cores, 2chips) and 1xSun Fire X4150 (4 cores, 1 chip) 1197.10 SPECjAppServer2004 JOPS@Standard; Best result with 8 cores in application-tier of the benchmark: 1xHP BL460c (8 cores,2chips) and 1xHP BL480c (8 cores,2 chips) 2056.27 SPECjAppServer2004 JOPS@Standard; Best result with 2 systems in application-tier of the benchmark: 2xDell PowerEdge 2950 (8 cores, 2 chips) and 1xDell PowerEdge R900 (4 chips, 24 cores) 4,794.33 SPECjAppServer2004 JOPS@Standard.



Tuesday Nov 04, 2008

Price Substantiation for SPECjAppServer2004 results

This blog entry is intended to disclose  and document the price of a number of selected SPECjAppServer2004 submissions and to demonstrate the big price advantages to consumers when using enterprise supported Open Source software in as much of the (enterprise) application stack as possible.  The SPECjAppServer2004 benchmark (despite it's name) is a full system benchmark and tests most system components i.e. application server, JVM, hardware, network, operating system and (especially) the database. Pricing SPECjAppServer2004 submissions therefore assists end users and developers to compare the costs of a system benchmark using proprietary software and using Open Source software running the same (unchanged) applications.

Notes on pricing

  1. For all submissions priced here the "Bill of Materials" (BOM) available in the SPECjAppServer2004 Full Disclosure report has been used as the basis for pricing. 
  2. List prices have been used because a) list prices are most easily comparable b) they are the prices advertised by vendors and c) are conservative i.e. it is unlikely a customer would pay higher than a vendor advertised list price.
  3. Although all tested components in SPECjAppServer2004 are required to be supported,  the pricing below does not include support support prices as support prices vary/ are not available for some hardware components listed in the BOM in some SPECjAppServer2004 submissions. [ note including support for the Open Source based Sun benchmark would result in an even lower price for the Sun benchmark compared to the proprietary solutions also note MySQL support is actually included in the Sun hardware purchase cost ].
  4. The prices quoted come predominately from the vendors websites with a small number coming from online stores.  All Sun components have been priced from the sun online store and prices included in the totals. Where prices can't be found for other vendors component then we have simply omitted the price or quoted the next cheapest and most equivalent component from the vendor website i.e. we are trying to make a genuine attempt to conservatively yet realistically quote competitors prices. The price/performance advantage for Open Source is so great understating the proprietary system cost by a little does not influence the analysis a great deal.

Table 1



Table 2


Table 3


Price/Perf claim Sun MySQL 5.0 SPECjAppServer result demonstrates the best price/performance of any of the published SPECjAppserver2004 results.

How can we be so sure ?

Well largely because the Sun submission is sooo much less expensive than the competition, just glancing through the other submissions shows that the software purchase/license costs of other non open source submissions are all going to be larger than the system cost of the Sun hardware and software combined in fact the Sun submission in most cases is less expensive than the database license for a single CPU.

Consider : There really are no other submissions that come close, to demonstrate this lets consider a couple.

Result / candidate

JOPS

Costs involved

$US/JOPS for (partial) costing

Conclusion

Sun GlassFish Enterprise Server v2 Update 2, SunFire X4150 Cluster with MySQL 5.0 on OpenSolaris 2008.05

1197

Hardware and Software costs = $16,110 as detailed above

$US 13.46 / JOPS@Standard

Note again this is the full system cost not partial

Sybase has published a low JOPS and potentially cheap and therefore good price/performance candidate.

Sybase Enterprise Application Server 6.0.2 Advanced Edition, 4 cores, 2 chips http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00074.html

653


From the Sybase web site using the buy button ...

Sybase EAServer Enterprise Edition 6.1 (6.0.2 no longer seems available)   = 7,500 / CPU = $US 15,000

SQL Anywhere 10.0.1 = $2499/CPU  = $4998


From www.microsoft.com

http://www.windowsmarketplace.com/Details.aspx?itemid=3267250


Microsoft Server 2003 (64 Bit) Standard Edition x64 = $US 594   \* 2

=  $US 1188


So software total

15,000

4,998

1,188

======

21,186

$US 32.44 / JOPS@Standard

So just of software costs alone the Sybase result is significantly more expensive.

The IBM result of WebSphere 6.0.2.3 Application Server on IBM System p5 550 Cluster http://www.spec.org/jAppServer2004/results/res2005q4/jAppServer2004-20051026-00020.html

This result shows 365.13 JOPS/DB core (and 8 DB cores) and so could potentially deliver good price performance (note that the Sun/MySQL result is not far behind at 299.28 JOPS/DB core)

2921

So looking at the IBM Processor Value Unit Calculator and DB2 price at https://www-112.ibm.com/software/howtobuy/buyingtools/paexpress/Express?P0=E9

It appears that we need 120 PVUs per core or a total of 960 PVUs and IBM DB2 Enterprise Server Edition Processor Value Unit (PVU) License + SW Subscription & Support 12 Months = $US 386
=> database license of 386 \* 960 = $US 370,560

$US 126.86 / JOPS@Standard

without even considering application server software, operating system or hardware.


Conclusion : The current Sun GlassFish Enterprise Server v2 Update 2, SunFire X4150 Cluster with MySQL 5.0 on OpenSolaris 2008.05 is by far the best price/performance published SPECjAppServer2004 result.

SPEC required disclosure : - SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 5th Nov 2008 All comparisons are based on the SPEC SPECjAppServer2004JOPS@Standard metric from www.spec.org or on pricing made using the bill of materials included in each SPECjAppServer2004 result

PricingSubstantiationForSPECjAppServer2004Results

some content
About

tomdaly

Search

Archives
« November 2008 »
SunMonTueWedThuFriSat
      
1
2
3
5
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
      
Today