Monday Feb 15, 2010

Directory Service Performance Optimization Strategy: Data Priming

Directory servers usually run for long period of times and have stable performances as all caches are warmed by the traffic. But how to get optimum performances as fast as possible right after starting the server ? Brad Diggs has published Directory Data Priming Strategies, another blog post added to the series of articles on Sun (now Oracle) Directory Server Enterprise Edition 7, ZFS and Flash Technologies.

Technorati Tags: , , , ,

Monday Feb 08, 2010

The basics of Flash Memory

These days, everybody get excited with Solid State Disks, flash memory and the performance improvements they have over other mass storage solutions.

Sun F20 Flash accelerator boardWe've been running some benchmarks of Sun Oracle Directory Server 7.0 leveraging new Sun flash based hardware modules. Before we go in details about their benefits, my colleague Brad Diggs posted a very educational article on the basics of Flash Memory to set a common understanding of the technology.

Read on and get ready for more data points on how ZFS and Flash Memory can improve Directory Server performances and scalability.

Technorati Tags: , , , ,

Friday Feb 05, 2010

Oracle and Sun Directory Services...

Mark Wilcox, principal product manager for Oracle Virtual Directory has posted an initial update with regards to Oracle and Sun directory services.
Nothing really detailed so far, but it's good place to post your comments on the Oracle + Sun Identity Management Strategy and more specifically regarding directory services.

To me and my coworkers, the most important messages are :

We are going to continue to offer both Oracle Internet DirectoryAND Sun Directory Server Enterprise Edition


OpenDS will remain an open-source project

Details are still being discussed and ironed out, but I hope to be able to share them soon. Stay tuned !

Technorati Tags: , , , , ,

Friday Jan 22, 2010

Sun Directory compresses data for better performance !

Sun Directory Server Enterprise Edition 7.0 was released last November, and in the December timeframe Brad Diggs and Wajih Ahmed, both Principal Field Technologists and big experts in Directory Services, backed with engineers from the Directory engineering team and Mr Benchmark, put the product on the test bench to evaluate its performance and scalability with Sun new hardware and especially the new F-20 PCIe flash drives (see also what Mr Benchmark says about the F-20).

Brad's first article describes how much Directory Server 7 entry compression rocks, "extending search performance by more than 50% through increased caching potential". Brad provides details of his findings and gives the commands to run to get the benefits of DSEE 7 in your deployment.

The entry compression feature is also available in the technology that will power future versions of Sun Directory Enterprise Edition: the OpenDS project. In OpenDS, there are 2 options to reduce the size of entries stored in the database. The first one is called entry compaction, and it's enabled by default. The entry compaction feature removes all references to attribute names and replace them with small identifiers. The second option is actually entry compression which will use the popular ZLib algorithm. This option is not activated by default, but it's just a command away :

<OPENDS_HOME>/bin/dsconfig -X -p 4444 -h localhost -D cn=Directory\\ manager\\
 -w password -n set-backend-prop \\
 --backend-name userRoot --set entries-compressed:true

Below is the dsconfig usage for disabling entry compaction with OpenDS:

<OPENDS_HOME>/bin/dsconfig -X -p 4444 -h localhost -D cn=directory\\ manager\\
 -w password -n set-backend-prop \\
 --backend-name backend --set compact-encoding:false

Here's a table that compares the size of the databases of OpenDS 2.2.0 with no compat encoding, with it (default settings) and with compression enabled. The table compares the size of the entry record within the database as well as the overall size of the database which also includes indexes (default OpenDS settings).

Entry Count LDIF Entry Size Uncompacted Entry Size Compacted Entry Size Compressed Entry Size Uncompacted DB Size Compacted DB Size Compressed DB Size
100K 599 b 645 b 481 b 361 b 178.8 MB 163.20 MB 151.65 MB
-34% - 25% -9.6% - 7.1%
1M 603 b 649 b 485 b 364 b 1,515 MB 1,358 MB 1,243 MB
-34% - 25% -11.5% - 8.5%
10M 607 b 653 b 490 b 363 b 13,973 MB 12,416 MB 11,188 MB
-33% - 26% -12.5% - 9.9%

The percentages are computed from the reference value which is the default i.e. compacted. A negative value means an increased size, a positive one means a reduced size.

The second table compares the import times for the 3 different modes for storing entries, for the 3 sample data files.

Entry Count Uncompacted Compacted Compressed
100K 21 s 21 s 22 s
1.1% - -3.5%
1M 106 s 107 s 112 s
0.5% - -4.9%
10M 1006 s 1009 s 1101 s
0.2% - -8.9%

Note: in this table, negative numbers represent increase in time required to import compared to the default settings.

Enabling compression does result in a smaller disk use with that sample data (fully random values), but does come with a performance penalty at least at import time, less than 10% but the penalty increases with the amount of entries. If you've read Brad's article on DSEE entry compression, you understand that the smaller the entries in the database, the more can be potentially cached in the Database Cache and the better the overall performances are. So if your entries are quite large, contain values that are strings, you should consider enabling the entry compression with OpenDS.

Changing from the default mode (compacted) to uncompacted mode does not give any real advantage in performance, but does increase the disk space usage, so I do not see the value of changing these settings in OpenDS.

Anyway, the benefits of having compact entries in the database are available today with Sun Directory Server Enterprise Edition 7 and Sun OpenDS Standard Edition 2.2, and are helping customers to reduce the overall cost of ownership of the directory services.

Technorati Tags: , , , ,

Friday Oct 23, 2009

What's new in Sun Directory Server Enterprise Edition 7 ?

Did you attend the event I talked about last week ? Remember, it was a webinar about Sun DSEE 7 and Role Manager 5.
Well, if you could not attend the webinar, you can watch it now, or download the video. The slides are also available.

Technorati Tags: ,

Friday May 29, 2009

In Love with DSEE !

Some people seem to be in love with our Directory Server Enterprise Edition product and are showing it ! I wonder when we will start seeing OpenDS license plates ;-)

DSEE lover

Photo courtesy of Kent Spaulding, CTO at Skyworth TTG.

Technorati Tags: ,

Thursday Mar 15, 2007

Directory Server and advanced certificate management

Directory Server 6.0 introduced many changes in its administration tools: a new GUI, new CLIs such as dsconf and dsadm.

dsadm has a set of commands to do certificate management for directory server instances, such as requesting new certificates, listing certificates, adding certificates. This feature has been added in Directory Server 6, because certutil, the utility available with the NSS library is not officially supported.

The dsadm utility does the work in most of the cases but there are some known limitations such as no support for the subjectAltName extension. For those advance use cases, the workaround is to use certutil (at your own risks).

One big difference between dsadm and certutil is the certificate store password. By default, the password is unknown to the administrators, and managed through a file. Certutil does require the password to be known.

To change the default password and be able to use certutil, you need to launch the following command as root or the owner of the directory server instance:

>  /opt/SUNWdsee/ds6/bin/dsadm set-flags /local/demo/dstest cert-pwd-prompt=on
Choose the new certificate database password:
Confirm the new certificate database password:
Certificate database password successfully updated.

From them, you will be able to run "certutil -d /local/demo/dstest/alias -P slapd- ..." with the appropriate options.

When you're done, you can store the password again in a text file for use by dsadm or Directory Server at restart with the following command:

>  /opt/SUNWdsee/ds6/bin/dsadm set-flags /local/demo/dstest cert-pwd-prompt=off
Enter the certificate database password:
Certificate database password has been successfully stored.

Monday Mar 05, 2007

DSCC - customizing and troubleshooting

Directory Service Control Center (DSCC) is the new graphical user interface to manage a complete directory service deployment. Below is a screen-shot of the main panel when starting DSCC.

DSCC Screenshot 

DSCC is relying on the Solaris WebConsole, which is available by default on Solaris but has been ported to the other supported platforms (HP-UX, Linux, Windows).

If you want to get a better understanding of the Web Console, want to change its default configuration or need to troubleshoot it, please refer to this document:

Wednesday Dec 20, 2006

Directory Services in the Telco world

Last week I was invited to a meeting with one of our customers, a wireless telecom operator happily  user of Sun Directory Server 5.2 (patch3) with a few tens of million entries.

With the convergence of voice and data, the telcos are looking for ways to reduce the number of databases they have and consolidate the data in a single repository such as LDAP-based directory services.
The discussion went on the subject of the data models, the differences between the LDAP model and the relational model, drifting to which model would be the most appropriate in consideration with the Generic User Profile recommendation from the IMS specifications. Clearly the discussion was reaching the limits of my expertise (while I'm quite confident in the LDAP area, IMS is not something that I've followed), but it was very informative.

The one thing that I really found interesting in this discussion: at no time, the consideration of performances was mentioned. It seemed obvious for all parties that LDAP directory services (and probably more specifically our Directory Server) do have the capability of keeping with the high throughput and low response time requirements of the network equipments.

And in fact, they really do. We will have some evidence of this with Directory Server Enterprise Edition 6.0 very soon.


This is the blog of a senior software engineer, specialized in LDAP, Directory Server and OpenDS. Ludovic Poitou works in France at the Grenoble Engineering Center, in the Directory Services Engineering team. Outside work, I love skiing and taking photo


« April 2014