Responsible Testing: comparing performance on Sun
By Eric Reid-ISV Engineering-Oracle on Nov 30, 2007
Letter, we get letters...
Numerous parties inside and outside of Sun ask use 'how does Drupal perform on Solaris vs Linux?' Reasonable question, to be sure, but upon further examination, we find a minefield fraught with potential peril...
- What constitutes a 'typical' Drupal database? Well, it doesn't hurt to start with something along the lines of what Dries Buytaert himself described earlier this year
- What constitutes a 'typical' Drupal workload? Again, Dries and others have some good starting ideas, some of which we've already tried, some of which will take time to implement properly in our framework
- How to conduct a good 'apples-to-apples' performance comparison?
- Most importantly, how to present and defend results in an open and transparent fashion, given we're comparing Sun against 'non-Sun' platforms?
And to answer your question, YES, I'd be interested in your comments on the subject! We are still learning as we go here...
In any event, we needed to start somewhere, so today I'm publishing some very preliminary results showing Drupal 5 performance (scaling, really) on Solaris vs Linux. The intent here is to show how Drupal performs as the number of concurrent users increases, using as close to 'stock' SAMP and LAMP configurations as possible (on identical hardware).
What did we find? Well, it appears that Drupal-on-SAMP performs and scales significantly better than Drupal-on-LAMP. Please note the operative word here -- APPEARS. We are in no way 'declaring a winner' at this point, for several reasons, not the least of which being the need for high certainty on our part and high believability on the reader's part.
Take a look for yourself.. comment.. complain... suggest. We very much want feedback on what we're doing at this point. You'll note that the results published on the Wiki are purely objective observations, with NO interpretation. It's too early for that, because we still need to be SURE of what we're seeing, we need to understand WHY we're seeing it, and we need to what it all really MEANS (as the intent is NOT to sell Sun here - that's the job of others at Sun).
Look for more refined results, as well as other Drupal-on-Sun performance testing, in the weeks to come.