By arnaud on Dec 08, 2009
Every once in a while, an elusive issue may drive you bonkers because you can't quite identify what is going awry, which of course is crucial to solving said issue. In such cases, it's good to go back to the basics and actually analyze what is flowing on the wire. To do so, snoop is the tool. But more important than capturing the data, which is fairly well documented on the web, analyzing properly what the capture contains may put you on track to solving the issue.
Bird's Eye View
In this entry, I'll introduce how wireshark can be used to help investigate LDAP issues or even simply to double check your LDAP product is true in its performance metrics... A word of advice, never trust a single metric at face value, check yourself. The LDAP server (or web or app server for that matter) may report a response time of X, but that really is no good to your client applications if the network card or the TCP layer has trouble getting the message through the wire. Do use snoop to collect data and wireshark to check that everything is fine or dig in the snoop files.
First things first, installing Wireshark.
1- get pkgutil
2- install wireshark
pfexec pkgadd -d http://blastwave.network.com/csw/pkgutil_`uname -p`.pkg
yes|pfexec /opt/csw/bin/pkgutil --install CSWwireshark
3- start wireshark, it is installed in /opt/sfw/bin/wireshark, for example do:
nohup /opt/csw/bin/wireshark 2>/dev/null &
4- Now that wireshark is started up, you can open up you snoop file and what you get is basic Ethernet+IP+TCP decoding., like so:
5- So we will quite simply have to tell wireshark to decode the TCP packets content as being LDAP (wherever applicable). Go to Analyze->Deocde As. A window pops up to allow you to select which decoder to use. Select the "Transport" tab, and then click on LDAP in the list. Parsing through the whole snoop file again may take a while, but once it's done, it will be worth the wait.
6- Once data is properly interpreted as LDAP, we can see that those TCP packets with an LDAP payload will now be advertised as LDAP (highlighted in green) right in the wireshark interface:
Now that you have followed these few simple steps, you can dig in the snoop data and graph statistics.
For example, you can very easily graph the ratio of LDAP operations to LDAP operation taking more than one second with the statistics graphing tool:
Of course, there are dozens of other invaluable tools in wireshark that are not only of the greatest quality but immensely useful as well to spot problems in your traffic that many higher level tools won't be able to help with, what comes to mind is stuff like:
- issues with lower-level than like IP problems (for e.g. CRC errors) or TCP retransmits
- hitting the maximum bandwith on the interface (use the statistics->IO Graphs tools and select YAxis to Bytes/Tick)
- LDAP level SLA not being met: you can check wehther a particular LDAP Time was met or not over your snoop file as shown above
- Check for particular LDAP return codes
- Check for particular LDAP filters
- Filter your results on anything contained in an LDAP packet
As I said earlier, this tool is truly fantastic.
Hope this ends up helping someone. Enjoy!