notd - Performance of Servlet Containers - Partial Addendum to Covalent blog
By pelegri on Apr 05, 2007
Covalent recently published some benchmark results on the scalability of 3 popular Servlet containers: Tomcat 6 using the native APR, Jetty and GlassFish. The results are good for the three, but they favor TC 6 when using a very large number of connections.
One danger with performance tests is that is is really hard to make accurate comparisons, despite everybodies' best intentions. For example, inside Sun I've been involved in comparisons between the Web Server and GlassFish and the results are affected by details like how many requests, what response time, what percentage of successful responses, how much memory, what tuning parameters, etc. That is why there are organizations like SPEC and benchmarks like Web2005 and SPECjAppserver (and even those have their own issues).
The Covalent piece is doing a good job in that it is dispelling the myth that Servlet containers do not scale, because it is forcing teams to provide better out of the box performance (Java SE 6 did a great job there) and because I think everybody will be more careful measuring performance. I'd expect improvements on all these fronts in the near future, in the meantime, here are some additional pointers you may want to check out...
Regarding the picture, I chose Mark Twain because (a) he popularized the phrase Lies, Damned Lies, and Statistics and (b) he is such a great guy that we should have him somewhere in TheAquarium, but please do note that I am not saying that the covalent blog are lies! :-)