Dissipating Performance Misconceptions About DPS
By arnaud on Aug 18, 2009
Recently engaged on a project where a Directory Server farm capable of delivering 100,000 searches per second , I was faced with skepticism about the ability for DPS to keep up in such a demanding environment. I had to prove ten times over that our proposed architecture would fit the bill and quantify how much headroom it would provide. Here is some of the things we observed along the way.
Bird's Eye View
In this article I will quickly talk about a theoretical performance of DPS under ideal circumstances, you'll see that the setup is quite far from a real deployment from the hardware to the way the stack is set up.
I won't make you read through the whole thing if you do not want to so here goes:
The box I'm developing on and running tests these days is somewhat unusual and the proper disclaimer is necessary: it is a corei7 (975 EE) very slightly overclocked to 3.9GHz fitted with 12GB of 1600MHz DDR3 RAM. It runs OpenSolaris 2009.06 with ZFS. There is a 10,000 rpm 320GB drive and an Intel Extreme 32GB SSD. All in all a nice rig but not a server either. The closest server if I had to give a comparable would be a beefed up Sun Fire X2270. Keep in mind that my desktop is single socket but about twice faster on the CPU clock rate.
To load this, I use SLAMD. One server, 7 clients (2 Sun Fire T5120, 4 Sun Fire X2100, 1 Sun Fire X4150). For the record, this client set has generated north of 300,000 operations per second in other tests I ran in the past. Clients in this test campaign were always loaded at less than 15% CPU, so results are not skewed by CPU pressure on the client side.
On the software side (middleware I should say) we have an Sun Ultra 27 running OpenDS with a null back-end and my rig running running my workspace of DPS 7 plus my little sauce...
Why a null back-end on OpenDS? that is not representative of real life!
Well, quite true, but the whole point in case here is to push DPS to the limit, not the backend, not the network or any other component of the system. So a null back-end is quite helpful here as it simulates a perfect back end that responds immediately to any requests DPS sends. Quite handy if you come to think of it because what you have as a result is an overview of the overhead of introducing DPS between your clients and your servers under heavy load. The load is actually all the hardware I had could take, the CPU is almost completely used, with idle time varying frantically between 1% and 7%. Keep in mind as well that DPS runs in a JVM and at these rate, garbage collections are almost constant.
Here is how the set up looks like:
That's all I had for you guys tonight!