MySQL Benchmark UltraSPARC T2 beats Xeon on Consolidation of OLTP & Web

Recently we put together a consolidation benchmark to see how an open-source stack performs against the proprietary stack from Microsoft. Solaris, MySQL, and Sun Web Server running on the open-source UltraSPARC T2 processor were pitted against a Microsoft SW stack running on a 4-socket QC Xeon server. This benchmark highlights the continued trend to incorporate MySQL open-source databases and how it works under virtualization (Solaris Zones).

The Sun SPARC Enterprise T5220 (1.4 Ghz UltraSPARC T2 processor) and Solaris Containers managing a consolidation of Open-Source Software components (MySQL Database and Sun Java System Web Server) provided 2.4 times better performance than the HP DL580 system (four Xeon quad-core processors) and a major virtualization software, Microsoft Windows 2003 Server EE, Microsoft SQLserver database and Microsoft IIS webserver.

The Sun SPARC Enterprise T5220 using the MySQL database in Solaris zones is over 3.2 times faster performance than HP DL580 using Microsoft SQLserver database running with the leading virtualization software.

The Sun SPARC Enterprise T5220 using the Sun Java System Web Server with Solaris Zones is 53% faster than than HP DL580 using Microsoft IIS webserver running with the leading virtualization software.

The database performance and the web performance were normalized and equally weighted for an overall performance metric.

The Sun SPARC Enterprise T5220 (1.4 Ghz UltraSPARC T2 processor) including hardware and open-source software costs has 3.7 times better price/performance than the HP DL580 system (four Xeon quad-core processors) with the Microsoft software stack. The Sun solution has 56% less price than the HP/Microsoft/virtualization configuration.

The Sun SPARC Enterprise T5220 (1.4 Ghz UltraSPARC T2 processor) had a 4.1 times better watt/performance than the HP DL580(four Xeon quad-core processors). The Sun system used 73% fewer watts total.

Sun's solution requires half of the rack space of the HP DL580. Sun has an 8.1 SWaP advantage over the HP DL580 (SWaP = Perf /[ Space (RU) x Watts ] )

The Sun SPARC Enterprise T5220 used open-source MySQL 5.0 database software and Sun Java System Web Server 7 which are both free for download to obtain these results.

Performance Comparison:

System Configuration (both 64GB Memory)
CPU RUs OS DB SW Web-
server SW
Sun SPARC Enterprise T5220 1.4GHz 1 chip
8 cores
2 RU Solaris 10 8/07 MySQL 5.0.51a Sun Java System Web
Server 7
HP DL580 G5 2.93GHz 4 QC Xeon
16 cores
4 RU MS Server 2003 EE MS SQL-
server 2005
MS IIS 6.0

System Performance & Cost (server, 64GB, disks, SW)
Web ops/s OLTP txns/m Watts HW+
SW $
Norm
Perf
Watt/
perf
$/perf
Sun SPARC Enterprise T5220 1.4GHz 9546 224K 480W $131k 2.36 203 55
HP DL580 G5 2.93GHz 6250 70K 830W $204k 1.0 830 204
Sun's Advantage 53% 3.2x 73% 56% 2.4x 4.1x 3.7x
Note: Watts are measured server watts during the run.

Benchmark Description

The Full-Stack Consolidated workload consists of 2 primary components: 1) a Web workload consisting of static HTTP requests, and 2) an OLTP database workload using a mix of common SQL transactions executed on a mix of tables in an RDBMS.

The Web workload generates static requests for web pages using a mix of file sizes that range from 100 bytes to 900K bytes with an average file size of 16K bytes. The metrics from this workload includes the number of web operations per second. In this Web workload, a set of 6 client systems emulating thousands of users, generated HTTP requests to all 3 webservers in parallel running on the server.

This performance comparison used an Ad-Hoc OLTP workload, called iGEN OLTP 1.6, which was developed from a realistic customer workload. iGen OLTP avoids problems that plague other OTLP benchmarks like TPC-C. In particular, it is completely random in table row selections and thus is difficult to use artificial optimizations. iGen OLTP stresses process and thread creation, process scheduling, and database commit processing. The database has 10 million customers residing in it, and is approximately 15 GB in size. at time of creation. The transactions are comprised of various SQL transactions: read-only selects, joins, inserts and update operations. For this OLTP workload, 1 client system was used to emulate hundreds of users, generating SQL transactions to all 3 database instances in parallel running on the server.

On the Sun SPARC Enterprise T5220, the 3x Webserver instances and 3x MySQL database instances where isolated on 6 Containers. Each webserver zone shared a single storage array, but separate Gbit network interfaces. The 3x MySQL database zones shared a single network interface but used separate storage arrays for the database files and transaction logs. On the HP DL580, the 3x Webserver instances and 3x SQLserver database instances where isolated on 6 virtual machines. Each webserver VM shared a single storage array, but separate Gbit network interfaces. The 3x SQLserver database VMs shared a single network interface but used separate storage arrays for the database files and transaction logs.

When results listed above were obtained when running both the Web and OLTP workloads concurrently. The performance results for each workload were normalized using the results obtained from the HP DL580 as follows:

Sun SPARC Enterprise T5220 factor = Web + OLTP = 2.36

  1. Web Factor = 9546 ops/sec / (2x 6250 ops/sec) = 0.76
  2. OLTP Factor = 224K txns/min / (2x 70K txns/min) = 1.6

HL DL580 performance factor = Web + OLTP = 1.0

  1. Web Factor = 6250 ops/sec / (2x 6250 ops/sec) = 0.50
  2. OLTP Factor = 70K txns/min / (2x 70K txns/min) = 0.50

The Sun Enterprise T5220 used 3 Solaris Containers to host separate MySQL database instances and 3 Containers to host separate Sun Web Server instances.

Each MySQL and Webserver instance ran in an isolated zone not visible o any processes or users in the other containers.

The combination of OLTP and Web workloads consumed an average of 95% of cpu utilization when running concurrently.

System Configuration & Results Summary

Sun SPARC Enterprise T5220 (1 chip, 8 cores)9546 Web operations/sec, 224K OLTP Txns/min.
HP DL580 (4 chips, 16 cores)6250 Web operations/sec, 70K OLTP Txns/min. Results as of 2/22/2008.

Results
Sun SPARC Enterprise T5220 9546 Web ops/sec, 224K OLTP Txns/min
HP DL580 G5 6250 Web ops/sec, 70K OLTP Txns/min
Reference Date: Feb 22, 2008
Systems: Sun SPARC Enterprise T5220
HP DL580 G5
Processor/GHz of Server: 1x UltraSPARC T2 1.4 GHz,
4x Intel Xeon X7350 2.93 GHz
Operating System/Virtualization: Solaris 10 8/07 & Solaris Containers
Microsoft Windows Server2003 EE & a leading virtualization platform

Sun SPARC Enterprise T5220 with:

  • 1x UltraSPARC T2, 1.4  GHz 8-core processor
  • 64 GB of memory
  • 4x Internal 146GB SAS disks
  • 2x 750W Power Supplies
  • 4x Sun StorageTek ST2540 arrays (12x 146GB disks per array)
  • 2x Sun StorageTek 4Gb Dual-Port Fibre Channel PCI-E HBA (from Emulex)
  • 1x Sun PCI-E Dual Giga Ethernet Adapter UTP
  • Solaris 10 8/07
  • Solaris Containers
  • MySQL 5.0.51a
  • Sun Java System Web Server 7 u1

HP DL580 G5 with:

  • 4x Intel Xeon X7350, 2.93  GHz 4-core processors
  • 64 GB of memory
  • 5x Internal 146GB SAS disks & Smart Array P400i SAS Ctlr
  • 4x 800/1200W Power Supplies
  • 4x Sun StorageTek ST2540 arrays (12x 146GB disks per array)
  • 2x QLogic SANblade QLE2462 PCI-E Dual Port FC adapter
  • 3x HP NC360T PCI Express Dual Port Gigabit Server Adapter
  • Leading Virtualization Platform
  • Microsoft Windows Server 2003 EE 64-bit
  • Microsoft SQLserver 2005
  • Microsoft IIS 6.0 Webserver
Comments:

Hello Ritu,

thank you very much for publishing these numbers. I have some questions about the benchmark's details: what kind of hardware virtualization software/technologie have used on the HP DL580? Have you also performed the benchmarks without any virtualization software involved?
To me it seems with your benchmarking methodology you also (mainly?!) measure the virtualization overhead of the different virtualization approaches (hardware-virtualization vs. logical containment aka zones).

Posted by M.S. on March 03, 2008 at 06:15 AM PST #

Why was there virtualization software? Doesn't sound like apples to apples to me...

Posted by cw on March 06, 2008 at 05:41 AM PST #

Why was there virtualization software? Doesn't sound like apples to apples to me...

Posted by cw on March 06, 2008 at 05:42 AM PST #

I am not sure why you think comparing servers using Solaris 10 and Windows is a useful comparison. All you are prooving is Solaris is faster than Windows. I dare you to publish these benchmarks including Linux.

Posted by Jacob on March 13, 2008 at 12:38 PM PDT #

I love how they throw in VMware for no apparent reason.

bogus

Posted by guest on May 08, 2008 at 10:36 AM PDT #

I don't think it's all that bogus a comparison because this is one of the many coomparisons people make right? You can put them all in one OS with no partitioning/virtualization technology, but most people wouldn't do that because they'd want some level of application isolation if they had the choice as long as the performance is "acceptable". If you were to do that on x86, it'd probably be Solaris Containers, Xen or VMWare. If you were to do that on a T2 based SPARC system, it would be Solaris Conatiners or Logical Domains. So when people talk about going to "cheap commodity hardware," these are one of the comparisons that are made. Each of the different virtualization/isolation technologies add different caveats and overhead. So what would be interesting is to make a comparison of

S10u5, no virtualization on T5220
vs
S10u5, Containers, on T5220
vs
S10u5, LDoms on T5220
vs.
Windows, no virtualization on DL580G5
vs
Windows, VMWare on DL580G5
vs
Windows, Xen on DL580G5
vs
Linux, no virtualization on DL580 G5
vs.
Linux, VMWare on DL580G5
vs
Linux, Xen on DL580G5

Of course, these are a lot of test cases and one would have to have a lot of time to spend running them. However, it would give a really good understanding of OS/hardware vs OS/Virtualization/hardware combinations so as to understand bare metal performance vs virtualization overhead.

Now if you threw in AMD quad-core based systems vs Intel quad-core systems vs UST2 based systems, that throws even more test cases in and probably completely different results depending on each processor's strengths and how well each OS and OS/Virt combo can leverage them.

If you wanted to do a straight performance comparison of an OS, it might be best to run a comparison on the same exact hardware between Solaris 10 u5 x86 vs. your preferred flavor of Linux. Mine would be Ubuntu, especially since it would work in the following test case and serve for a platform comparison.

The other interesting test case for a straight performance of an OS would be Solaris 10 u5 SPARC on T5220 vs Ubuntu on T5220. Running these last two comparisons would be fairer. Of course, it still doesn't factor in OS and Application tuning. Application tuning, especially with respect to the compiler used and compilation options provided can make significant differences on different platforms so you may not necessarily provide the same compiler options, which would be a point of contention by those who want to ensure apples to apples comparisons.

The other variable when comparing OSs, hardware and virtualization is application workloads. How do web, OLTP, batch, single-threaded, heavily-threaded or multi-processed, floating-point intensive, integer intensive, cache-based vs memory or disk based, crypto heavy, network intensive workloads perform. Then factor in the size of the workload as far as how much you plan to scale the workload and you have a lot of workloads. Simply stating that one OS, OS/Virt, processor technology, compiler or some combination of all those is too simplistic. There are waay too many variables and too little time to run all these test cases.

-w

Posted by w on May 21, 2008 at 02:05 PM PDT #

I've tested mysql on a CoolThreads T1000 box about two years ago. We're now testing MySQL on the 128 core latest T5240 CoolThreads box

Typical mysql deployments require high clock speed to achieve performance. MySQL runs really fast on xeons because each core is 3ghz. Sun's architecture is perfect for Postgres, Oracle, Apache, basically anything that runs tons of processes and can take advantage of the 128 cores

Buying MySQL for a billion dollars instead of promoting EnterpriseDB and Postgres was the dumbest thing that sun ever did, because mysql runs horribly on their hardware and getting anything other than a single core chip for mysql is a waste of money

OpenSores is all about buzzwords and trends, and sun made a billion dollar mistake that will probably burry them in the near future unless they do a complete rewrite mysql

The guy that wrote innodb says that mysql runs terribly on multiple cores, there are thousands of posts saying that mysql can't work on multiple cores. So sun buys them, and tries to say that it'll run 10x faster on a 1.2ghz core than it will on a 3.0 xeon

STUPID!!!!

Posted by Ted Osadchuk on July 01, 2008 at 04:54 AM PDT #

Hi Ritu,

For a configuration stated below what would be the maximum number of transactions approximately for the hardware choice below

1. Dual Sun UltraSPARC 3i processors (Sun Fire V245/V445)
2. 4 GB RAM
3. 2 \* 73 GB or 4 \* 73 GB internal HDD in soft mirror configuration

Posted by Pavan on September 09, 2008 at 04:39 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Ritu Kamboj

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today