Tuesday Oct 20, 2009

Enumeration based attributes in LDAP

Yesterday I've explained how to restrict LDAP attribute values using Regular Expression based syntaxes, with the OpenDS directory server. There is another use case for restricting attribute values: when there is an enumerated list of possible values. It's possible to define finite list of values as a regular expression, but as we wanted to be able to provide additional values, we added in OpenDS the ability to define Enumeration based syntaxes, and we implemented it as a syntax definition extension as well.

Here's an example of use of an Enumeration syntax for the day of the week. Let's first define and load the syntax in the OpenDS directory server's schema :

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=schema
changetype: modify
add: ldapsyntaxes
ldapSyntaxes: ( 1.3.6.1.4.1.32473.4 DESC 'Day Of The Week'
X-ENUM ( 'monday' 'tuesday' 'wednesday' 'thursday'
'friday' 'saturday' 'sunday' ) )

Processing MODIFY request for cn=schema
MODIFY operation successful for DN cn=schema
\^D

Let's use the syntax in an attribute, itself used in an object classes:

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':

dn: cn=schema
changetype: modify
add: attributetypes
attributetypes: ( 1.3.6.1.4.1.32473.5 NAME 'test-attr-enum'
SYNTAX 1.3.6.1.4.1.32473.4 )
-
add: objectclasses
objectclasses: ( 1.3.6.1.4.1.32473.6 NAME 'testOCenum' SUP top
AUXILIARY MUST test-attr-enum)
-

Processing MODIFY request for cn=schema
MODIFY operation successful for DN cn=schema
\^D

Let's create a test entry :

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=TestEntry,dc=example,dc=com
changetype: add
sn: TestEntry
cn: TestEntry
objectclass: Person

Processing ADD request for cn=TestEntry,dc=example,dc=com
ADD operation successful for DN cn=TestEntry,dc=example,dc=com
\^D

And now, let's make use of the newly created schema objects with that test entry :

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=TestEntry,dc=example,dc=com
changetype: modify
add: objectclass
objectclass: testOCenum
-
add: test-attr-enum
test-attr-enum: monday

Processing MODIFY request for cn=TestEntry,dc=example,dc=com
MODIFY operation successful for DN cn=TestEntry,dc=example,dc=com
\^D

But if the value isn't part of the enumeration, it gets rejected :

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':

dn: cn=TestEntry,dc=example,dc=com
changetype: modify
replace: test-attr-enum
test-attr-enum: Lundi

Processing MODIFY request for cn=TestEntry,dc=example,dc=com
MODIFY operation failed
Result Code: 21 (Invalid Attribute Syntax)
Additional Information: When attempting to modify entry cn=TestEntry,dc=example,dc=com to replace the set of values for attribute test-attr-enum, value "Lundi" was found to be invalid according to the associated syntax: The provided value "Lundi" cannot be parsed because it is not allowed by enumeration syntax with OID "1.3.6.1.4.1.32473.4"
$

The enumeration syntaxes, like the regular expression one, matches like a DirectoryString, that is matches using CaseIgnoreMatch equality rule.

$ bin/ldapsearch -p 1389 -D cn=directory\\ manager -w secret12 \\
-b "dc=example,dc=com" '(test-attr-enum=Monday)'

dn: cn=TestEntry,dc=example,dc=com
objectClass: Person
objectClass: top
objectClass: testOCenum
test-attr-enum: monday
cn: TestEntry
sn: TestEntry

But the biggest advantage of the Enumeration syntax is the ability to use Ordering match, which is not based on strings, but on the order of the enumerated values in the syntax definition. So "Monday" is lower than "Tuesday" which is lower than "Wednesday"...

$ bin/ldapsearch -p 1389 -D cn=directory\\ manager -w secret12 \\
-b "dc=example,dc=com" '(test-attr-enum<=Thursday)'

dn: cn=TestEntry,dc=example,dc=com
objectClass: Person
objectClass: top
objectClass: testOCenum
test-attr-enum: monday
cn: TestEntry
sn: TestEntry

I hope you will find this useful and make use of these syntaxes. To do so, you need to download and install OpenDS 2.2 Release Candidate 1 (or higher).
And if you have additional requirements with syntaxes, I'd be happy to hear about them.

Technorati Tags: , , , ,

Monday Oct 19, 2009

Regular Expression based attributes in LDAP

One of the question that I get frequently asked when discussing with customers or coworkers about defining custom schema and attributes, is how to restrict the values that can be set to an attribute. From a pure LDAP standard point of view, you would need to define a new syntax and describe the valid values. Then you would need to check with the directory server's vendor or discuss with the open source developers to get the syntax implemented in the server, either in the core product, or as a plug-in extension. In the end, the easy choice goes to use a standard syntax (like DirectoryString) and let all client applications validate the values.

In OpenDS, we've choose another option. We have added support for regular expression based syntaxes, and implemented this as a syntax definition extension.

So in order to define, for example, an attribute whose values must be in the form of host:port, you simply need to define a new syntax for it with the regular expression pattern and load it in the server's schema:

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=schema
changetype: modify
add: ldapsyntaxes
ldapSyntaxes: ( 1.3.6.1.4.1.32473.1
DESC 'Host and Port in the format of HOST:PORT'
X-PATTERN '\^[a-zA-Z][.a-zA-Z0-9-]+:[0-9]+$' )

Processing MODIFY request for cn=schema
MODIFY operation successful for DN cn=schema
\^D

And then you can make use of the newly defined syntax in attributes.

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=schema
changetype: modify
add: attributetypes
attributetypes: ( 1.3.6.1.4.1.32473.2 NAME 'test-attr-regex' SYNTAX 1.3.6.1.4.1.32473.1 )
-
add: objectclasses
objectclasses: ( 1.3.6.1.4.1.32473.3 NAME 'testOCregex' SUP top AUXILIARY MUST test-attr-regex)
-

Processing MODIFY request for cn=schema
MODIFY operation successful for DN cn=schema
\^D

Let's create a test entry

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=TestEntry,dc=example,dc=com
changetype: add
sn: TestEntry
cn: TestEntry
objectclass: Person

Processing ADD request for cn=TestEntry,dc=example,dc=com
ADD operation successful for DN cn=TestEntry,dc=example,dc=com
\^D

And now make use of this new attribute and objectclass:

$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':

dn: cn=TestEntry,dc=example,dc=com
changetype: modify
add: objectclass
objectclass: testOCregex
-
add: test-attr-regex
test-attr-regex: localhost:1389
-

Processing MODIFY request for cn=TestEntry,dc=example,dc=com
MODIFY operation successful for DN cn=TestEntry,dc=example,dc=com
\^D
$ bin/ldapmodify -D cn=directory\\ manager -p 1389
Password for user 'cn=directory manager':
dn: cn=testentry,dc=example,dc=com
changetype: modify
replace: test-attr-regex
test-attr-regex: foobar.com

Processing MODIFY request for cn=testentry,dc=example,dc=com
MODIFY operation failed
Result Code: 21 (Invalid Attribute Syntax)
Additional Information: When attempting to modify entry cn=testentry,dc=example,dc=com to replace the set of values for attribute test-attr-regex, value "foobar.com" was found to be invalid according to the associated syntax: The provided value "foobar.com" cannot be parsed as a valid regex syntax because it does not match the pattern "\^[a-zA-Z][.a-zA-Z0-9-]+:[0-9]+$"

It's simple and efficient. But wait there's more to come, tomorrow.

Technorati Tags: , , , ,

Thursday Oct 15, 2009

Tip on OpenDS localization and error messages...

The OpenDS LDAP directory server is localized by default in many different languages, thanks to our community.
All (well we try) messages from the client tools, command lines or graphical, are translated in Chinese (Simplified and Traditional), German, French, Japanese, Korean and Spanish (and soon Polish). But the server error messages are also localized, and the OpenDS directory server picks up the current locale of the process owner to choose in which language to print them.
Not everyone wants to have the server error messages in their own language, especially in distributed or international teams. There is a way to make sure the server always uses English as the language for the message, regardless of who starts it, and it's very simple (thanks to Josu for reminding me how to do it ;) ):

Edit the java.properties file (from the config/ directory) and append the following to the start-ds.java-args line:

-Duser.language=en -Duser.country=US

Example:

start-ds.java-args=-server -Xms128m -Xmx256m -Duser.language=en -Duser.country=US

Now run the dsjavaproperties command and restart the server.

Et voila ! All in English.

Technorati Tags: , , , , , ,

Tuesday Oct 13, 2009

OpenDS 2.2.0 Release Candidate 1 is now available

Opends Logo TagThe OpenDS development team is very pleased to announce the immediate availability of OpenDS 2.2.0-RC1 which is the first release candidate for OpenDS 2.2.

OpenDS 2.2 offers the following new features from OpenDS 2.0 :

  • Scalable import and indexing
  • External changelog compliant with the Internet-Draft "Definition of an Object Class to Hold LDAP Change Records", draft-good-ldap-changelog-04.txt
  • Fractional replication
  • Extensible matching rules for time base attributes
  • Support for custom syntaxes based on substitution, regular expressions or enumeration
  • Remote server management in control panel
  • Recurrent tasks in control Panel
  • Default automatic Backup in the control panel
  • Separation of LDAP Servers and Replication Servers for replication
  • Ability to merge disjoint replication topologies
  • Dsconfig script friendly mode

We've also captured a first snapshot of the OpenDS 2.2 documentation and hosted it on it's own wiki: https://docs.opends.org/2.2/. The documentation is not complete yet, but will be almost at the same time we will do the final release of OpenDS 2.2.

The purpose of the Release Candidate is to solicit one last round of testing before the final release. So please test the OpenDS release with your client applications, in your environment or on your favorite platform.

If you do find a bug, please report it with Issue Tracker.

We welcome feedback. Please report you experience with OpenDS on our mailing lists, or on #opends IRC channel on Freenode.

OpenDS 2.2.0-RC1 is built from revision 5941 of our source tree.

The direct link to download the core server is: http://www.opends.org/promoted-builds/2.2.0-RC1/OpenDS-2.2.0-RC1.zip

The direct link to download the DSML gateway is: http://www.opends.org/promoted-builds/2.2.0-RC1/OpenDS-2.2.0-RC1-DSML.war

We have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://www.opends.org/promoted-builds/2.2.0-RC1/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://www.opends.org/promoted-builds/2.2.0-RC1, including the detailed change log

Major changes incorporated since OpenDS 2.1.0-build002 include:

  • Revisions 5870, 5888 (Issue #4181) - Resolves a Null pointer exception in DSML Gateway with specific substring search filters
  • Revision 5871 (Issue #4217) - Fixes an issue with ACI containing parenthesis in the description field
  • Revision 5874 - Improves the rebuild-index processing for performances
  • Revision 5880 (Issue #4252) - Fixes a replication issue between OpenDS 2.1/2.2 and OpenDS 2.0
  • Revision 5883 (Issue #4203) - Fixes an issue where restore -l (list the available backups) would exit with return code 1
  • Revision 5926 (Issue #4257) - Fixes an error raised when deleting recurrent tasks

Technorati Tags: , , , , , , ,

Monday Oct 05, 2009

Directory "Engineering"

Arnaud a co-worker from the Sun directory engineering team, has taken the term "Directory Engineering" to a new level. Arnaud has always been a doer, someone who starts playing with things, investigate, test, benchmark... Recently, he's been deploying OpenDS on Amazon cloud, configuring a Sun workstation running OpenSolaris with 4 displays in Xinerama mode and much more... 

But in the past few weeks, Arnaud started to play with hardware devices like USB Bit Whacker, a few lines of codes, his favorite server product and finally built this :

OpenDS Weather Station

The OpenDS Weather Station, provides a dashboard of the important metrics from an OpenDS server, showing instantaneously how loaded is the server.
Arnaud already has 3 Stations in order (I and other members of our team want one for demo purpose), but I'm not sure he's ready to accept orders from other people and turn this into another business :)
Anyway, this is a nice little engineering project !

Technorati Tags: , , , ,

Wednesday Sep 30, 2009

OpenDS 2.1.0-build002 is now available

Opends Logo TagWe have just uploaded OpenDS 2.1.0-build002, built from revision 5868 of our source tree, to our promoted builds folder.

The direct link to download the core server is: http://www.opends.org/promoted-builds/2.1.0-build002/OpenDS-2.1.0-build002.zip

The direct link to download the DSML gateway is: http://www.opends.org/promoted-builds/2.1.0-build002/OpenDS-2.1.0-build002-DSML.war

We have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://www.opends.org/promoted-builds/2.1.0-build002/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://www.opends.org/promoted-builds/2.1.0-build002, including the detailed change log

Major changes incorporated since OpenDS 2.1.0-build001 include:

  • Multiple fixes to the new Import code and new Public ChangeLog feature.
  • Revision 5783 (Issue #4171) - Fixes a hang in replica initialization when the replication servers are unreachable.
  • Revision 5804 - Performance and scalability improvements with monitoring.
  • Revision 5842 (Issue #4194) - Resolves an issue where objectclasses would disappear when modified.
  • Revision 5843 - Upgrade the underlying Berkeley DB JE to version 3.3.87.
  • Revision 5847 (Issue #4164) - Fixes a decoding problem .
  • Revision 5848 (Issue #4229) - Resolves an issue where the connection handler thread hangs and cause potential DoS attack.
  • Revision 5849 (Issue #4226) - Improves the PartialDateOrTime matching rule to match on time as well as date.
  • Revision 5854 (Issue #4240) - Resolves an issue in the Control-Panel when displaying attributes with a syntax that has no name.
  • Revision 5863 & 5867 (Issue #4117) - Resolves an issue with MODDN operation that could impact ability to export and reimport from LDIF.
  • Revision 5865 (Issue #4060) - Prevents a new server process to start while OpenDS server is shutting down. Also preserves the server.pid when in-core restart is performed.

Technorati Tags: , , , , ,

Monday Sep 14, 2009

Jack and Pat on OpenSSO and OpenDS...

Pat Patterson reminded me of a conversation he had at OSCON 2009 with Jack Adams about OpenSSO. Luckily, the discussion was captured in video.
During the conversation, they talk about OpenDS as well. Thanks for the plug, Pat !

 

 

Technorati Tags: , , , , , ,

Wednesday Sep 09, 2009

Another new feature in OpenDS Control Panel

Opends Logo TagOpenDS 2.0 has been out for a couple of months now but the development team kept on the pace of development.

Beside its ability to manage remote OpenDS servers, the Control Panel has been enhanced to support the Recurrent Tasks introduced in the OpenDS 2.0 server, and both Export LDIF and Backup can be scheduled to happen at a later time or on a regular basis.

OpenDS control panel Backup screen

Notice the "Change" button in the Backup Options.

OpenDS Control Panel, Choice for scheduling a backup

You can then choose the proper kind of scheduling and tune it very simply as illustrated below.

OpenDS Control Panel, scheduling a weekly backupOpenDS Control Panel, Scheduling a backup with Cron like notation

Technorati Tags: , , , ,

Tuesday Sep 08, 2009

Managing multiple OpenDS servers

Opends Logo Tag
Up until now, to manage an OpenDS server, one would need to log onto the machine and starts the Control Panel.

In the next release of OpenDS (OpenDS 2.2), the Control Panel can now connect to remote servers, allowing an administrator to remotely monitor and tune any running instance of OpenDS.

Let's see what has changed in the Control Panel for the remote access, and what are the limitations.

The first thing you will notice when starting the Control Panel is a new dialog which allows you to choose between the local server or a remote server.
OpenDS Control Panel, connection dialogOpenDS Control Panel new connection dialog

Once you've selected the server to administer, you will see the usual Control Panel window with its left action bar and information on the right.
OpenDS Control Panel remote server view

You can change server while the Control Panel is running. It's in the File menu, when you are on the Main window of the Control Panel.
OpenDS Control Panel, Changing Server to Administer
OpenDS Control Panel,  Changing Server to Administer

There is very little difference between managing a local server and managing a remote server.
One thing you will notice when administering a remote server is that you can't stop or restart it. Also, you cannot use the Control Panel to configure the Java properties of a remote server. That's it.

The Control Panel cannot be installed as a standalone tool, it's a part of the OpenDS server installation, and it can only manage one server at a time, local or remote. But the ability to manage remote servers will reduce the need to logon to each host and run the Control Panel on each instance either physically or using a remote display, simplifying the task of the directory administrators.

If you want to check this capability, you can download and install one of the recent OpenDS daily builds, or wait for next promoted build (2.1.0-build001).

Technorati Tags: , , , ,

Wednesday Sep 02, 2009

Everything has an end...

And so do vacations, and blog silence.

I've been back in the office for over a week now but I was trying to catch up with emails, irc, blogs and news, too busy to find the time to blog again.
There's a lot to say on the LDAP and OpenDS front.

While I was happily riding the Mont Ventoux and around with friends and family, the project kept on moving on the path to OpenDS 2.2 and several new features have been committed by the team in the code repository:

  • The Control Panel can now be used to manage remote server instances.
  • OpenDS now publishes all changes in a public ChangeLog accessible (subject to access control) under the cn=changelog naming context.
  • Replication now supports a Fractional mode allowing to exclude or include only specific attributes of all replicated entries.
  • dsreplication utility has been improved to allow separating the replication service from the replicated OpenDS instance.
  • The import feature has been rewritten and optimized, reducing the time and memory required to import very large set of data.
  • The server now supports 2 new MatchingRules to better deal with Time and Dates (GeneralizedTime syntax).
  • The server now supports the ability to declare a new syntax but default it's implementation to an existing one.
  • The server now supports the ability to declare new Regular Expression based syntaxes and attributes.
  • The server now supports the ability to declare new Enumeration based syntaxes and attributes.

Most of the new features are already documented as part of the User Documentation of the OpenDS documentation wiki. You can test these features in recent daily builds, or you can wait for the next promoted build (2.1.0-build001) that should come pretty soon.
I will be starting a series of articles to describe with illustrations and details those new features, in the coming days and weeks.

Also in a separated branch, Matt and Bo have been working on an LDAP Client API, which is getting in a good shape to be released for beta testing soon (probably along with OpenDS 2.2).

LDAPCon 2009
The 2nd. International conference on LDAP, LDAPCon 2009 will be held on September 20th and 21st at Waterfront Marriot Hotel , Portland OR, USA. If you haven't registered yet, please register now ! The registration fee includes access to the LinuxCon 2009 (Sep 21 - 23), and if you still need to be convinced that it's worth attending, you can check the agenda. I hope to see you there.

Also noticed in the blogosphere and the websphere :

Finally I know the title of this post may have alarmed some of you. I don't know what's going to happen in the coming days, but I just hope I won't have to write another post with the same title on the subject of OpenDS or myself.

Technorati Tags: , , , , , ,

Tuesday Jul 28, 2009

OpenDS turns 3 today...

OpenDS open source project is 3 years oldAnother year has passed and we already end of July. Today is the anniversary day for the OpenDS project which is turning 3 this year.

As usual, this is also time to look back in the mirror and consider what we've achieved.

A little more than 10 days ago, we announced the availability of OpenDS 2.0, the new and stable release of our LDAPv3 directory server. OpenDS 2.0 came just about one year after OpenDS 1.0 and 6 months after OpenDS 1.2.
You can read about OpenDS 2.0 features in the Release Notes, but also in the various articles that have relayed our own announcement such as:

Sun OpenDS Standard Edition 2.0 CD
Yesterday, Sun publicly announced the general availability of Sun OpenDS Standard Edition 2.0, a Sun supported version of the OpenDS project, as well as the release of OpenSSO Express Build 8 (due in a couple of weeks).

Sun OpenDS Standard Edition 2.0 has the same features as OpenDS 2.0. Differences are in the branding, the license, the documentation that is available from docs.sun.com in HTML and PDF and of course the support services offered by Sun.
Mark Craig has already posted an illustrated article describing how easy it was to install Sun OpenDS Standard Edition 2.0 on Windows XP.

OpenSSO Express builds are supported snapshots of OpenSSO development. As Pat Patterson, Community Manager for OpenSSO and covering all Identity Products at Sun, detailed on his blog, OpenSSO Express Build 8 includes a new Mobile One Time Password Feature, the Fedlet for .Net and a new task flow enabling single sign-on to Salesforce.com.

As OpenDS is getting mature, we're seeing public endorsement and use of it. In the last couple of weeks, we had 2 success stories including the use of OpenDS :

Finaly within a year, the OpenDS Community has more than doubled, in term of members in the community, but as well in the number of active contributors and participants in the #opends IRC channel, and in term of unique visitors on the www.OpenDS.org.

OpenDS.org Monthly visits

I'm proud of what we've accomplished in 3 years and even more of the past year. We still have a lot of ideas and customers requirements to build in the OpenDS project. Overall we know where we want to go and we hope our new executives will agree that it's a nice and viable path to follow...

Technorati Tags: , , , , , , , ,

Monday Jul 27, 2009

OpenDS 2.0 on Mac OS X with the latest JVM...

Opends2 PictoMacOSX 10.5.7There is an issue in the start and stop scripts that is preventing OpenDS 2.0 to be installed via Java Web Start on Mac OS X 10.5 with the latest version of the JVM (Update 4 a.k.a 1.6.0_13). I've discovered the problem at the same time we were releasing OpenDS 2.0.0 release candidate 4 which was planned to be the last release candidate. So the fix is not the release but has been committed to the trunk.

The issue is that the new JVM does use a larger default minimal heap size and reject any calls with -Xmx if the maximum heap size is smaller than its internal default (around 30MB).

Still OpenDS 2.0 can be installed on Mac OS X and used with the latest JVM, by downloading the Zip file, unzipping it and doing minor edition in the start-ds and stop-ds scripts.

$ unzip ~/Desktop/OpenDS-2.0.0.zip
Archive: /Users/ludo/Desktop/OpenDS-2.0.0.zip
creating: OpenDS-2.0.0/
...
inflating: OpenDS-2.0.0/upgrade
$ cd OpenDS-2.0.0/bin

In the start-ds and the stop-ds scripts, replace all occurences of the string "-Xms8M -Xmx8M" with "-client"

$ cp start-ds start-ds.orig
$ sed -e "s/-Xms8M -Xmx8M/-client/g" < start-ds.orig > start-ds
$ cp stop-ds stop-ds.orig
$ sed -e "s/-Xms8M -Xmx8M/-client/g" < stop-ds.orig > stop-ds

OpenDS QuickSetup App IconYou can now run the setup program (or launch the QuickSetup application) to install and configure the OpenDS directory server.

Technorati Tags: , , , , , ,

Thursday Jul 23, 2009

Assured Replication: A New Feature of OpenDS 2.0

OpenDS 2.0 has just been released and there are several new and exciting features in it.

To me, the biggest innovation in this release is "Assured Replication", an extension to the loose consistency multi-master replication feature that brings tighter consistency of data between replica. "Assured Replication" is not to be taken for a full synchronous and transactional replication mechanism. A change is not transactionally applied to a set of or all replicas of a topology. With "Assured Replication", the response to an LDAP modification is delayed until the change has been received or applied by other servers, in a best effort mode. It provides a greater assurance that a change is not lost even if the server receiving it crashes.

Opends Assured Replication with Safe Data level 2

Assured Replication can function in 2 modes :

  • Safe Data Mode: an update must be propagated to a defined number of Replication Servers before returning a response to the client. So if the server or the replication server is stopped, the data is still available to all other replicas.
  • Safe Read Mode: an update must be propagated to all directory servers in the domain before the client is returned a response for the update.

Of course, for both modes, it's possible to configure a timeout interval to prevent LDAP clients to be waiting indefinitely if some servers are not available.

Configuring Assured Replication is pretty straightforward but cannot be done when setting up replication itself. So the first step is to configure Multi-Master Replication for a domain with dsreplication.

$ bin/dsreplication enable --host1 localhost --port1 5444 --bindDN1 'cn=directory manager' --bindPassword1 secret12 --replicationPort1 8989 --host2 localhost --port2 6444 --bindDN2 'cn=directory manager' --bindPassword2 secret12 --replicationPort2 8990 --adminUID admin --adminPassword secret12 --baseDN "dc=example,dc=com" -X -n

Establishing connections ..... Done.
Checking Registration information ..... Done.
Configuring Replication port on server localhost:5444 ..... Done.
Configuring Replication port on server localhost:6444 ..... Done.
Updating replication configuration for baseDN dc=example,dc=com on server localhost:5444 ..... Done.
Updating replication configuration for baseDN dc=example,dc=com on server localhost:6444 ..... Done.
Updating Registration configuration on server localhost:5444 ..... Done.
Updating Registration configuration on server localhost:6444 ..... Done.
Updating replication configuration for baseDN cn=schema on server localhost:5444 ..... Done.
Updating replication configuration for baseDN cn=schema on server localhost:6444 ..... Done.
Initializing Registration information on server localhost:6444 with the contents of server localhost:5444 ..... Done.
Initializing schema on server localhost:6444 with the contents of server localhost:5444 ..... Done.

Replication has been successfully enabled. Note that for replication to work you must initialize the contents of the base DN's that are being replicated (use dsreplication initialize to do so).

$ bin/dsreplication initialize --baseDN "dc=example,dc=com" --adminUID admin --adminPassword secret12 --hostSource localhost --portSource 5444 --hostDestination localhost --portDestination 6444 -X -n

Initializing base DN dc=example,dc=com with the contents from localhost:5444:
23 entries processed (100 % complete).
Base DN initialized successfully.

See
/var/folders/SH/SHFsRjymHtqiZ4GxPNZERU++Fwk/-Tmp-/opends-replication-737929812662715818.log
for a detailed log of this operation.

$ bin/dsreplication status -h localhost -p 5444 --adminUID admin --adminPassword secret12 -X

dc=example,dc=com - Replication Enabled
=======================================
Server : Entries : M.C. (1) : A.O.M.C. (2) : Port (3) : Security (4)
---------------:---------:----------:--------------:----------:-------------
localhost:5444 : 23 : 0 : N/A : 8989 : Disabled
localhost:6444 : 23 : 0 : N/A : 8990 : Disabled

Now that replication is setup, we can enable the Assured Replication mode, using the dsconfig utility. For this, on each of the OpenDS direcotry servers, we first need to retrieve the full name of the replication domain.

$ bin/dsconfig -D cn=directory\\ manager -w secret12 -n -s list-replication-domains --provider-name "Multimaster Synchronization"

cn=admin data (domain 29167)
cn=schema (domain 9674)
dc=example,dc=com (domain 14741)

$ bin/dsconfig -D cn=directory\\ manager -w secret12 -n set-replication-domain-prop --provider-name "Multimaster Synchronization" --domain-name "dc=example,dc=com (domain 14741)" --advanced --set assured-type:safe-data --set assured-sd-level:2

Note that the Replication Domain has a different value on each server, so you have to repeat these 2 commands on each instance.
Setting the assured level for Safe Data to 2 means that the server will make sure the data has been received by at least 2 replication services before returning to the LDAP client the response to the update request.

From a client point of view, there should be no difference, except that the server might take a little longer to return the response to an update request. In our measures, we found that the response time increased by 25% for Safe Data Level 2, which seems a lot, but honestly, when the response time is in the order of 2ms, it's hard to notice !

You can find more information about Assured Replication on OpenDS 2.0 documentation wiki, both in the overview of OpenDS Replication Architecture and the Replication Administration Guide, and more specifically Assured Replication Administration Guide

Technorati Tags: , , , , ,

Monday Jul 20, 2009

LDAP Tip : Counting the number of entries in a branch...

This is a general LDAP tip and it applies to OpenDS the open source LDAP directory service in Java, as well as Sun Directory Server (all versions) and other LDAP servers:
How can I know the number of entries under a specific node of the Directory Information Tree ?
Well it's simple. Every entry contains an operational attribute that specify the number of immediate subordinates entry : numSubordinates.
So to retrieve the number of entries under a specific node of the DIT, for example in the ou=people,dc=example,dc=com, a simple read is required.

$ bin/ldapsearch -p 3389 -D "cn=directory manager" -w - -b "ou=people,dc=example,dc=com" -s base '(objectclass=\*)' numsubordinates

Password for user 'cn=directory manager':
dn: ou=People,dc=example,dc=com
numsubordinates: 21

This attribute is defined in an expired Internet-Draft but has been well implemented in many servers. There are often some limitations, like the value only counts entries on the same server, but overall it's a very useful attribute especially when browsing through the DIT.

OpenDS and Sun Directory Server also implements another attribute : hasSubordinates, defined in X.501. hasSubordinates is a boolean and returns "true" or "false" depending on whether the entry is a branch or a leaf in the Directory Information Tree.

Technorati Tags: , , , ,

New in OpenDS 2.0: Recurrent and Scheduled Tasks

Opends2 PictoOpenDS 2.0 has just been released and there are several new and exciting features on it.

Today we will focus on one simple feature that greatly reduce cost of administration: scheduled tasks.

Being a Directory Server administrator often implies that you have to perform some administrative tasks on a regular basis. One of those tasks for example that an administrator has to do is a backup of the database. With most Directory Servers, the administrator would write a script to be run on a specific time of the day (or rather the night) that would proceed with the backup.
With OpenDS and the Recurrent Tasks, we've simplified this to the extreme: Just instruct OpenDS to do a backup on a weekly or daily basis, and as long as the server is running, it will execute the backup procedure at the desired time.

Here's how to schedule an hourly, compressed backup for the main back-end :

$ bin/backup -p 5444 -D cn=directory\\ manager -w secret12 -n userRoot \\
-d ./backups -c --recurringTask '0 \* \* \* \*'
Recurring Backup task BackupTask-dc89d98e-4ade-410e-ad19-325279af8f67
scheduled successfully

Now, just wait for the hour to pass, and check if the backup has been taken ;-)

The string passed as a parameter following the --recurringTask option has the same format as for the crontab(5) time/date: a 5 integer pattern field, separated by blank spaces: Minute (0-59), Hour (0-23), Day Of Month (1-31) Month Of Year (1-12) Day Of The Week (0-6 with 0 being Sunday).

The recurrent tasks are not limited to backups. They can be applied to all tasks, although some may not be that useful to everyone. Although I do see some use of a daily import of an LDIF file from a well know location, as a way to synchronize with external sources.

And of course, you can list the scheduled and recurrent tasks with dsconfig and cancel them if needed.

In the next release of OpenDS, you will be able to configure the recurrent tasks with the Control Panel. If you can't wait, you can try with the latest daily build.

You can find more information on recurrent tasks on the OpenDS Documentation Wiki.

Technorati Tags: , , , , ,

About

This is the blog of a senior software engineer, specialized in LDAP, Directory Server and OpenDS. Ludovic Poitou works in France at the Grenoble Engineering Center, in the Directory Services Engineering team. Outside work, I love skiing and taking photo

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today