Sun and Siebel Kick Some Benchmark Butt


Sun and Siebel Systems just kicked some butt with an amazing benchmark of Siebel software running on Sun's platform.  I couldn't be happier about the work that Sun's Market Development Engineering (MDE) folks did in concert with many other participants from around the company.

We will be talking about the results and how we got them at Siebel's big conference "Siebel User Week" in Los Angeles.  You can see the benchmark results for yourself here, but unless you're Francisco Casas from Siebel or Khader Mohiuddin from Sun or one of a few select others in the Siebel benchmark illuminati, you're probably not going to understand what you're looking at.  So I'll summarize the results here then I'll spend some time sharing what I've learned in the last twelve months about benchmarking enterprise applications.

Want the 10-second summary?   Here it is: among all UNIX platforms for Siebel 7.5, we win in terms of both pure performance (we're faster) and in price/performance (we're cheaper).  And we did it with SPARC-based systems.

How about a 60-second summary?  Try this on for size: Siebel is a three-tier application: web tier, application tier, DB tier.  The application tier is wherer a lot of the heavy lifting happens, but you also gotta pay attention to all those DB transactions going on.  According to our calculations (which are based on the published benchmark results on Siebel's web site), we beat the next best Siebel 7.5 UNIX competitor like this (numbers shown are are users per CPU):

The users/cpu comparison is done by extrapolating the total cpu utilized to 100%


HP

Sun

Sun Wins By This Much

Web Tier

1984

2453

24%

Application Tier

385

662

72%

Database Tier

1786

4902

174%


There are those who think that our SPARC systems aren't good enough to win any pure performance races anymore, but this result seems to indicate otherwise.  The benchmark ran on a mixture of UltraSPARC 3 and UltraSPARC 4 machines, and the amazing thing to me is that even the "slow" US-III box (a Sun Fire V440 with 4 CPUs and 16GB of physical memory), still beat HP on the app tier by running at 440 users/CPU (compare above with 385 users/CPU).  That's 14% faster using our "old" stuff that is practically dirt cheap.

Even if we hadn't won in performance, we'd still have won in price/performance.  If you price out the rough list price estimates for the equipment used to do our benchmark and the competition's, it looks like we cost about 1/3 to 1/4 of what the second place UNIX vendor did.  That's just crazy, but I think it's true.

What lesson are we supposed to take away from this?  I think there are a couple of them:

  1. Sun's SPARC-based systems running the Solaris operating system really can scream
  2. Learning how to tune your software and doing it with discipline can get you huge return on the effort you expend doing it
Next, I'll write about what that second lesson is about.  I mean, what could be so difficult about tuning a piece of software, and why did it take Sun almost four years to come up with this benchmark result?  It can't be that hard, now can it?

Oh, yes it can.  Come back to find out why.
Comments:

Post a Comment:
Comments are closed for this entry.
About

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. What more do you need to know, really?

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today