Taking Throttling To The Next Level

Rationale

After releasing the first version of the throughput throttling, most customers seemed interested in at least kicking the tires and wanted to evaluate it but as it turned out, the fact that throttling was choking traffic across the board, it would have actually required to deploy an extra instance of DPS in the topology for the sole purpose of choking traffic to an acceptable level to the business. While some felt it was simple enough to do, some found it to be a show stopper and therefore, I wrote this new plug-in, leveraging the distribution algorithm API to allow to narrow the scope of traffic throttling per data view, bringing a whole new dimension of flexibility to this feature.

Bird's Eye View

This new wad not only provides a new, more flexible throttling facility to your LDAP Proxy Server, it also comes with a CLI that makes it trivial to configure and change your settings on the fly. The README has some instructions to get you going in no time, but I will provide a quick overview here as well.

The Meat

First things first, you need to unzip the package, which will give your the following files:

$ find Throttling
Throttling
Throttling/throttleadm
Throttling/Throttling.jar
Throttling/README

As you can see, pretty trim.

The CLI will ease mainly 3 things:

  1. Setting up data views to be able to throttle traffic (throttleadm configure)
  2. Configuring the throughput limits to enforce (per data view and operation type, e.g.: throttleadm throttle root\\ data\\ view mod 200 )
  3. Checking the current configuration (throttleadm list)

Here is the complete usage for this little tool

$ ./throttleadm --help
Checking for binaries presence in path
.
.
.
.
throttleadm usage:
  throttleadm list
  throttleadm configure <dataViewName>
  throttleadm unconfigure <dataViewName>
  throttleadm choke <dataViewName>
  throttleadm unchoke <dataViewName>
  throttleadm throttle <dataViewName> <operation> <rate>
for example:
  To list the data views configured for throttling
  throttleadm list

  To set up a data view to use throttling
  throttleadm configure root\\ data\\ view

  To set up a data view back to its original state
  throttleadm unconfigure root\\ data\\ view

  To enable throttling on the default data view (provided the data view has been properly configured)
  throttleadm choke root\\ data\\ view

  To disable throttling on the default data view
  throttleadm unchoke root\\ data\\ view

  To change or set the maximum search throughput on the default data view to 20
  throttleadm throttle root\\ data\\ view search 20

Finally, when you change the settings, you can see them be enforced right away. In the example below, I initially have set the bind throughput limit to 200 per second. The left window has an authrate running and in the right window, while the authrate is running, I lower the throughput limit to 100 for 4 seconds and then set it back to 200. See how that works below:

Finally, here is a quick snap of the output of the CLI for the throttling status.

 $ ./throttleadm -D cn=dpsadmin -w /path/to/password-file list
Checking for binaries presence in path
Will now take action list with the following environmentals:
-host=localhost -port=7777 -user=cn=dpsadmin -password file=/path/to/password-file
VIEW NAME          - THROTTLED -  ADD - BIND -  CMP -  DEL -  MOD - MDN - SRCH
ds-view            -      true -   12 -  200 -   13 -   14 -    1 -  16 -  112

 That's it !

As usual, don't hesitate to come forth with questions, remarks, requests for enhancements.


<script type="text/javascript"> var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www."); document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E")); </script> <script type="text/javascript"> try { var pageTracker = _gat._getTracker("UA-12162483-1"); pageTracker._trackPageview(); } catch(err) {}</script>
Comments:

Arnaud,

Excellent job as usual :)
I was wondering if we could have a feature that would automatically choke the traffic if the resp time starts to rise or hits a certain threshold or if the time the transaction spent in the queue hits a threshold; another thing that would be nice to have is a choke schedule, choke from 7pm to 7am on weekdays then unchoke from 7am to 7pm...
Out of the scope of this topic can we have a transaction time (time the request spent in the system, from when it gets to the listener till the worker thread sends it back to the client) along with etimes?

Again thanks for the great work,

Best regards,
Pierre Charbel

Posted by Pierre Charbel on August 12, 2009 at 05:04 PM MDT #

Pierre that's a very good suggestion indeed, as customary from you. So you are suggesting that the throttling only kick in when etime gets over a certain threshold... How would we link the 2 though?
I mean, it could very well be that the etime is shooting up but traffic is quiet, like a batch of adds for example would send search etime through the roof without necessarily much search traffic, right?
I'll think about it though because I do believe that there are some pretty neat applications of your idea!

Posted by arnaud on August 24, 2009 at 04:43 AM MDT #

Arnaud,

Thank you for looking into this; so if i understand well you mean that if we have 2 clients, one gong throgh 1 view and doing adds that cause the etimes to go through the roof and another one sending a very light search traffic through another view, the etimes of the searches will also be high due to the heavy adds right? unfortunately i don;t have right now a deterministic way to of knowing what traffic is causing the etimes to rise but as a less optimal solution can we do something like this : we first choke the traffic on one view and wait for few minutes, if the etime goes down we are on target else we choke the next view and wait till we hit the threshold and we set some priorities on the views to start with the ones that handle the least critical clients? do you see some benefits in doing that?

Thanks again for the great work; i am planning on testing the ZILs soon (another one of your interesting posts :) )

Posted by Pierre Charbel on September 03, 2009 at 05:32 PM MDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Directory Services Tutorials, Utilities, Tips and Tricks

Search

Archives
« August 2015
SunMonTueWedThuFriSat
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
     
Today