Friday Sep 14, 2007

OpenDS 1.0.0-build005 is now available

I have just uploaded OpenDS 1.0.0-build005, built from revision 3056 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/68459/OpenDS-1.0.0-build005.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/68461/OpenDS-1.0.0-build005-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

NOTE: -- Even though it is displayed as an option in the QuickSetup installer, we do not support upgrading from previous OpenDS builds to the 1.0.0-build005 release. There are some changes in this release that are not backward compatible with the configuration used by previous releases, and these changes may cause the upgrade process to fail.


Detailed information about this build is available at http://builds.opends.org/weekly-builds/1.0.0-build005/. Some of the changes that have been incorporated since OpenDS 1.0.0-build004 include:
  • Revision 2796 (Issue #2030) -- Update the filesystem entry cache to provide the ability to use a compact entry encoding.

  • Revision 2804 (Issues #2104, 2162) -- Update a number of the command-line utilities so that they operate in interactive mode rather than non-interactive mode by default.

  • Revision 2806 -- Update the account status notification handler API so that it is possible to provide additional properties along with the notification. This makes it possible to develop account status notification handlers that can act more intelligently and/or provide more useful information.

  • Revision 2811 (Issue #2097) -- Fix a problem in which total update initialization can fail to send a message to the replication cache.

  • Revision 2820 (Issues #43, 72) -- Implement support for the numSubordinates and hasSubordinates virtual attributes. Also, provide a new dbtest tool, which can be used to perform low-level debugging for the backends using the Berkeley DB Java Edition.

  • Revision 2824 (Issue #581) -- Provide an SMTP account status notification handler that can be used to send e-mail messages whenever an account status notification is generated. The notification message can be sent to the user that is the target of the notification and/or a specified set of administrators. The messages that will be sent are generated based on user-editable templates.

  • Revision 2843 (Issue #1831) -- Implement complete support for an interactive mode for dscfg. The tool now provides a menu-driven interface for examining and updating the server configuration.

  • Revision 2856 -- Update the CreateRCScript tool so that it provides the ability to specify the user that the server should run as, and also lets the user specify the JAVA_HOME and JAVA_ARGS settings that should be used. Also, update the start-ds and stop-ds commands to support a "--quiet" argument, which causes them to not generate any output. This mode will be used when starting and stopping the server through the generated RC script.

  • Revision 2877 -- Fix a memory leak that can occur when a backend based on the Berkeley DB Java Edition is disabled.
  • Revision 2879 (Issue #2180) -- Fix a problem in the JE backend in which contention on an index key might cause that key to contain an incomplete or incorrect value.

  • Revision 2882 (Issue #2205) -- Fix a problem that caused replication to behave incorrectly when a replicated change included an attribute type that was not defined in the schema of the target server.

  • Revision 2889 (Issue #2158) -- Add support for storing compressed schema representations in the JE backend and re-enabled the compact entry encoding by default.

  • Revision 2894 -- Add number of new configuration definitions for objects that were previously using "generic" definitions. This will help make it much easier for users to create new instances of these kinds of configuration objects.

  • Revision 2899 -- Add new directory environment properties that can be used to indicate whether the server should maintain a configuration archive, and if so the maximum number of archived configurations that should be maintained.

  • Revision 2900 (Issue #1945) -- Update the server so that it has the ability to save a copy of its current configuration into a ".startok" file whenever it starts successfully. Also, expose an option in the start-ds script and in the directory environment configuration that provide the ability to start the server using the "last known good" configuration rather than the current configuration.

  • Revision 2904 (Issues #1481, 2031) -- Add the ability to set any Berkeley DB JE property in the server configuration, for both backends based on the Berkeley DB Java Edition and the filesystem entry cache.

  • Revision 2913 (Issue #257) -- Implement support for a plugin that can be used to maintain referential integrity within the server. Whenever an entry is deleted or renamed, then any references to that entry in a specified set of attributes will be removed or renamed accordingly.

  • Revision 2921 -- Update the LDAP connection handler to explicitly close the selector when it is disabled or the server is shut down to prevent problems with being unable to re-bind to that port when the server is restarted.

  • Revision 2926 (Issue #139) -- Implement support for a maximum blocked write time limit in the LDAP connection handler. If an attempt to write data to the client is stalled for too long, then the client connection will be terminated.

  • Revision 2932 (Issue #261) -- Implement support for a 7-bit clean plugin, which can be used to ensure that the values of a specified set of attributes will only be allowed to contain ASCII characters.

  • Revision 2933 (Issue #2218) -- Update the LDIFPluginResult object to provide a mechanism that can be used to indicate why an entry should not be imported/exported.

  • Revision 2935 (Issue #1830) -- Implement support for secure communication in the dsconfig utility.

  • Revision 2950 (Issue #2216) -- Implement support for an LDIF connection handler, which may be configured to watch for new LDIF files to be created in a specified directory and have changes defined in those files automatically applied in the server through internal operations.

  • Revision 2955 (Issue #2181) -- Implement support for delete and modify operations in the task backend.

  • Revision 2961 -- Update a number of command-line tools that can be used to perform operations either directly against a backend or through the task backend so that if a port number and/or bind DN are provided, then the tool will default to using the tasks interface.

  • Revision 2966 -- Implement support for encryption and authentication when using replication.

  • Revision 2974 (Issue #2155) -- Update the server configuration so that a password storage scheme is referenced by its DN rather than the storage scheme name.

  • Revision 2986 -- Update the replication changelog database so that it implements the backend API. This provides the ability to backup and restore the changelog database, and provides a groundwork for future LDAP access to the changelog contents.

  • Revision 2998 (Issue #1594) -- Provide the ability to expose a monitor entry for the server entry cache.

  • Revision 2999 (Issue #2057) -- Update the server to provide a basic framework to control when plugins will be invoked. In particular, this adds the ability to indicate whether a plugin should be invoked for internal operations, and it also adds the ability to have plugins that are notified whenever changes are applied through synchronization. The unique attribute plugin has been updated so that it can detect uniqueness conflicts introduced through synchronization and generate an alert to notify administrators of the problem.

  • Revision 3006 -- Make a number of minor changes to improve server performance.

  • Revision 3008 -- Update the server configuration handler to fix a problem in which some change listeners may not be notified when the associated entry is updated.

  • Revision 3024 -- Make a number of additional changes to improve server performance.

  • Revision 3031 (Issues #1335, 1336, 1878, 2201, 2250) -- Provide new utilities that can be used to configure the server to participate in a replication environment.

  • Revision 3033 -- Upgrade the Berkeley DB Java Edition library to version 3.2.44.

  • Revision 3044 -- Make a couple of minor changes to improve server performance.

  • Revision 3048 -- Add a new tool that may be used to manage tasks defined in the server.

  • Revision 3051 (Issue #2059) -- Display SHA-1 and MD5 digests of a certificate fingerprint instead of the complete certificate when prompting the user about whether the certificate should be trusted in the status panel.

  • Revision 3054 -- Update the server so that it is possible to call EmbeddedUtils.startServer after having previously called EmbeddedUtils.stopServer. Previously, the server shutdown process did not leave the server in a sufficient state to allow it to be restarted in the same JVM.

Tuesday Sep 04, 2007

Configuring OpenDS with dsconfig -- part 2

Several weeks ago, I wrote about the dsconfig tool that can be used to help manage the OpenDS configuration. It provides a convenient way to view and edit the server configuration, and it can be very helpful in writing administrative scripts, much like the dscfg tool does in DSEE 6.

Honestly, though, I think that it's time to come clean about something: whenever I'm configuring DSEE 6, I rarely use the dscfg tool. I think the main reason for this is that I got so used to managing the server with ldapmodify over the years with the 3.x, 4.x, and 5.x versions that it's always been just as easy for me to use ldapmodify (since I know all of the configuration entry DNs and attribute names by heart) than to try to remember all of the command line arguments to make dscfg do what I want.

Until very recently, I had fallen into the same trap with the OpenDS dsconfig tool. However, last week Matt Swift updated the dsconfig tool to provide a new interactive mode that takes all the effort out of it. The interactive mode isn't suitable for use in writing scripts that automate config changes, but the non-interactive mode is still available for that. However, if you just want to make a configuration change to the server (or even if you just want to see what is available to be configured) then I think that you'll like the new interactive mode.

To start it up in interactive mode, simply invoke the dsconfig tool with no arguments. For example, on a UNIX-based system, you can just use:
bin/dsconfig
The tool will then prompt you for information about how to connect to the server, and then it will present you with a menu of options. You can use this interactive mode to view information about the current configuration, edit or remove existing configuration objects, or create new configuration objects (basically, all of the same things that you can do with the non-interactive mode, but without the need to remember any subcommand, argument, or property names).

As an example, here's the output from a session that I used to edit the default password policy in order to configure passwords to expire after 90 days. I've formatted the output so that the stuff I typed is bold and underlined and the output from the tool is in italics:
$ bin/dsconfig


>>>> Specify OpenDS LDAP connection parameters

Directory server hostname or IP address [localhost]: 127.0.0.1

Directory server port number [389]: 389

Administrator user bind DN [cn=directory manager]: cn=Directory Manager

Password for user 'cn=directory manager': password


>>>> OpenDS configuration console main menu

What do you want to configure?

    1)   Access Control Handler               20)  Matching Rule
    2)   Account Status Notification Handler  21)  Monitor Provider
    3)   Alert Handler                        22)  Multimaster Domain
    4)   Attribute Syntax                     23)  Password Generator
    5)   Backend                              24)  Password Policy
    6)   Certificate Mapper                   25)  Password Storage Scheme
    7)   Connection Handler                   26)  Password Validator
    8)   Crypto Manager                       27)  Plugin
    9)   Debug Target                         28)  Plugin Root
    10)  Entry Cache                          29)  Replication Server
    11)  Extended Operation Handler           30)  Root DN
    12)  Global Configuration                 31)  Root DSE Backend
    13)  Group Implementation                 32)  SASL Mechanism Handler
    14)  Identity Mapper                      33)  Synchronization Provider
    15)  JE Index                             34)  Trust Manager
    16)  Key Manager                          35)  Virtual Attribute
    17)  Log Publisher                        36)  VLV JE Index
    18)  Log Retention Policy                 37)  Work Queue
    19)  Log Rotation Policy

    q)   quit

Enter choice: 24


>>>> Password Policy management menu

What would you like to do?

    1)  List existing Password Policies
    2)  Create a new Password Policy
    3)  View and edit an existing Password Policy
    4)  Delete an existing Password Policy

    b)  back
    q)  quit

Enter choice [b]: 3


>>>> Select the Password Policy from the following list:

    1)  Default Password Policy
    2)  Root Password Policy

    c)  cancel
    q)  quit

Enter choice [c]: 1


>>>> Configure the properties of the Password Policy

         Property                                   Value(s)
    ---------------------------------------------------------------------------
    1)   account-status-notification-handler-dn     -
    2)   allow-expired-password-changes             false
    3)   allow-multiple-password-values             false
    4)   allow-pre-encoded-passwords                false
    5)   allow-user-password-changes                true
    6)   default-password-storage-scheme            SSHA
    7)   deprecated-password-storage-scheme         -
    8)   expire-passwords-without-warning           false
    9)   force-change-on-add                        false
    10)  force-change-on-reset                      false
    11)  grace-login-count                          0
    12)  idle-lockout-interval                      0 s
    13)  last-login-time-attribute                  -
    14)  last-login-time-format                     -
    15)  lockout-duration                           0 s
    16)  lockout-failure-count                      0
    17)  lockout-failure-expiration-interval        0 s
    18)  maximum-password-age                       0 s
    19)  maximum-password-reset-age                 0 s
    20)  minimum-password-age                       0 s
    21)  password-attribute                         userpassword
    22)  password-change-requires-current-password  false
    23)  password-expiration-warning-interval       5 d
    24)  password-generator-dn                      "cn=Random Password
                                                    Generator,cn=Password
                                                    Generators,cn=config"
    25)  password-history-count                     0
    26)  password-history-duration                  0 s
    27)  password-validator-dn                      -
    28)  previous-last-login-time-format            -
    29)  require-change-by-time                     -
    30)  require-secure-authentication              false
    31)  require-secure-password-changes            false
    32)  skip-validation-for-administrators         false
    33)  state-update-failure-policy                reactive

    ?)   help
    f)   finish - apply any changes to the Password Policy
    c)   cancel
    q)   quit

Enter choice [f]: 18


>>>> Configuring the "maximum-password-age" property

    Specifies the maximum length of time that a user may continue using the
    same password before it must be changed.

    Specifies the maximum length of time that a user may continue using the
    same password before it must be changed (i.e., the password expiration
    interval). The value of this attribute should be an integer followed by a
    unit of seconds, minutes, hours, days, or weeks. A value of 0 seconds will
    disable password expiration. Changes to this configuration attribute will
    take effect immediately.

    Syntax: DURATION (s)

Do you want to modify the "maximum-password-age" property?

    1)  Keep the default value: 0 s
    2)  Change the value

    ?)  help
    q)  quit

Enter choice [1]: 2


Enter a value for the "maximum-password-age" property [continue]: 90 days

Press RETURN to continue


>>>> Configure the properties of the Password Policy

         Property                                   Value(s)
    ---------------------------------------------------------------------------
    1)   account-status-notification-handler-dn     -
    2)   allow-expired-password-changes             false
    3)   allow-multiple-password-values             false
    4)   allow-pre-encoded-passwords                false
    5)   allow-user-password-changes                true
    6)   default-password-storage-scheme            SSHA
    7)   deprecated-password-storage-scheme         -
    8)   expire-passwords-without-warning           false
    9)   force-change-on-add                        false
    10)  force-change-on-reset                      false
    11)  grace-login-count                          0
    12)  idle-lockout-interval                      0 s
    13)  last-login-time-attribute                  -
    14)  last-login-time-format                     -
    15)  lockout-duration                           0 s
    16)  lockout-failure-count                      0
    17)  lockout-failure-expiration-interval        0 s
    18)  maximum-password-age                       12 w 6 d
    19)  maximum-password-reset-age                 0 s
    20)  minimum-password-age                       0 s
    21)  password-attribute                         userpassword
    22)  password-change-requires-current-password  false
    23)  password-expiration-warning-interval       5 d
    24)  password-generator-dn                      "cn=Random Password
                                                    Generator,cn=Password
                                                    Generators,cn=config"
    25)  password-history-count                     0
    26)  password-history-duration                  0 s
    27)  password-validator-dn                      -
    28)  previous-last-login-time-format            -
    29)  require-change-by-time                     -
    30)  require-secure-authentication              false
    31)  require-secure-password-changes            false
    32)  skip-validation-for-administrators         false
    33)  state-update-failure-policy                reactive

    ?)   help
    f)   finish - apply any changes to the Password Policy
    c)   cancel
    q)   quit

Enter choice [f]: f
The Password Policy was modified successfully

Press RETURN to continue


>>>> Password Policy management menu

What would you like to do?

    1)  List existing Password Policies
    2)  Create a new Password Policy
    3)  View and edit an existing Password Policy
    4)  Delete an existing Password Policy

    b)  back
    q)  quit

Enter choice [b]: q
$

As I mentioned above, this was just integrated last week, so it will be in our next build (build005, which will hopefully be available at the end of this week). If you want to try it out before then, then feel free to check out and build the server for yourself.

Sunday Aug 26, 2007

OpenDS 1.0.0-build004 is now available

I have just uploaded OpenDS 1.0.0-build004, built from revision 2794 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/65720/OpenDS-1.0.0-build004.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/65721/OpenDS-1.0.0-build004-DSML.war.

I have also updated the archive that may be used to install or upgrade OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://builds.opends.org/weekly-builds/1.0.0-build004/. Some of the changes that have been incorporated since OpenDS 1.0.0-build003 include:
  • Revision 2567 -- Update the global ACI definitions to ensure that anyone will be allowed to use the password policy request control by default.

  • Revision 2569 (Issue #2061) -- Update the "Who Am I?" extended operation handler so that it works properly in conjunction with the proxied authorization control and alternate SASL authorization identities.

  • Revision 2570 (Issue #2059) -- Update graphical panels to eliminate cases in which some panels used a different background color than the underlying window. Fix a problem in which it was necessary to click a button twice in order to accept a certificate. Make sure to display a more appropriate representation of the certificate fingerprint.

  • Revision 2584 (Issue #752) -- Implement support for subordinate modify DN plugins, which can be used to receive notification whenever an entry is renamed because of of its superiors was targeted by a modify DN operation.

  • Revision 2586 (Issue #1894) -- Provide the ability to clean up replication information in other servers when uninstalling an instance with replication configured. Also, eliminate the statuspanel.jar file to try to keep the total number of JAR files to a minimum (for easier embedded use), and minimize the number of classes that go in quicksetup.jar to reduce the initial download time when launching QuickSetup.

  • Revision 2595 -- Update the global ACI definitions to ensure that anyone will be allowed to use the authorization identity request control by default.

  • Revision 2601 (Issue #2087) -- Implement support for an identity mapper that can use regular expressions to transform the provided ID string before searching for the appropriate matching user.

  • Revision 2611 (Issue #423) -- Implement support for nested static groups. If the member/uniqueMember attribute of a static group references the DN of another group, then the members of that child group will be considered members of the parent group.

  • Revision 2612 -- Provide a new org.opends.server.util.EmbeddedUtils class that can be used to simplify the process of running the server as an embedded application. It includes methods for starting, stopping, and restarting an embedded server, as well as a means of determining whether the server is currently running.

  • Revision 2613 -- Provide a new @PublicAPI annotation type that can be used to tag code to indicate whether we consider it part of our public API, and if so what the stability level is for that code (i.e., how likely the interface is to change in an incompatible manner in the future) and the ways in which it may be accessed by third-party developers.

  • Revision 2614 (Issue #2098) -- Update the DIGEST-MD5 processing code to properly degrade to initial authentication whenever a client attempts to use subsequent authentication.

  • Revision 2619 (Issue #588) -- Implement support for the "list" tag in MakeLDIF. Note that the syntax for this implementation is not the same as the syntax used for the version of MakeLDIF provided with SLAMD, but it is more internally consistent with other tags that are part of MakeLDIF.

  • Revision 2641 -- Migrate to a new framework for message handling within the server. This will provide a much better framework for future I18n support, and moves the default English-language messages to properties files rather than having them embedded in the code.

  • Revision 2648 -- Correct a number of spelling errors identified in the English-language messages.

  • Revision 2650 -- Apply the @PublicAPI annotation to packages and classes that are part of the OpenDS codebase to indicate their anticipated role in our public API.

  • Revision 2660 -- Add a new convenience constructor for the InternalClientConnection class that makes it possible to create a connection authenticated as a given user by providing only that user's DN.

  • Revision 2664 -- Provide a new EmbeddedUtils.initializeForClientUse() method that can be used to initialize the proper internal structures so that OpenDS code can be more easily used for client-side use.

  • Revision 2687 (Issues #788, 791) -- Update the replication mechanism so that it has the ability to automatically repair inconsistencies that may be detected.

  • Revision 2706 -- Add a new internal LDAP socket implementation that provides a mechanism for leveraging third-party LDAP SDKs to perform internal operations in OpenDS. This makes it easier for applications that already support communication with external LDAPv3 directory servers to also interact with an embedded OpenDS instance. This capability has been tested with both JNDI and the Mozilla LDAP SDK for Java.

  • Revision 2707 -- Merge the installation and upgrade utilities into a single application. Now, when you launch QuickSetup via Java Web Start, you will be asked whether you want to install a new server or upgrade an existing installation.

  • Revision 2721 -- Implement support for an attribute uniqueness plugin, which can be used to ensure that the values for a specified set of attributes are all unique throughout the server (e.g., that no two users are allowed to have the same uid or e-mail address).

  • Revision 2736 (Issue #1602) -- Fix a problem in the replication code in which the removal of the entry at the root of a replication domain could cause a large number of changes to be replayed.

  • Revision 2737 (Issue #1804) -- Update the replication code so that the server can generate an administrative alert whenever a replication conflict is detected.

  • Revision 2743 -- Update the import-ldif and export-ldif tools so that they can be used to launch tasks to perform the import and export operations in addition to operating directly on the backend.

  • Revision 2748 (Issue #2134) -- Fix a problem in the configuration of VLV indexes that prevented them from being managed through the dsconfig tool.

  • Revision 2750 (Issue #2097) -- Fix a problem in the replication total update code that could cause initialization to fail if the replication code received any messages that were not related to the total update.

  • Revision 2755 (Issue #2103) -- Fix an issue with log rotation that could cause it to behave incorrectly if the time interval was changed and the new value was shorter than the previous value.

  • Revision 2757 -- Update the backup and restore tools so that they can be used to launch tasks in addition to operating directly on the backend.

  • Revision 2772 (Issue #2135) -- Add a configuration option that can be used to store entries using a more compact encoding.

  • Revision 2780 -- Update the default server configuration to ensure that the uniqueMember attribute type is indexed for equality.

  • Revision 2781 -- Add a tool that can be used to base64 encode or decode data provided as a string, contained in a file, or piped in via standard input.

  • Revision 2783 -- Update the lock manager configuration to provide a property that can be used to indicate whether fairness should be guaranteed for contended locks.

  • Revision 2784 (Issue #526) -- Add a mechanism for generating an RC script that can be installed on UNIX systems to configure the server to automatically start when the system boots. Also, update the stop-ds script so that if the server is to be stopped using a kill but no PID file is present, then the stop script will generate an error rather than trying to stop the server using a task (which is guaranteed to fail, since no credentials will have been provided).

Friday Aug 24, 2007

Where's the latest OpenDS build?

If you've been watching, you might have noticed that we didn't release a build last week. It was scheduled, and we generated the build and put it through its paces, but unfortunately a huge last-minute commit (which will help make the server easier to internationalize) caused a little more instability than we would have liked. Although most of the problems were found and fixed pretty quickly, we decided to delay the build for another week just to get in some more testing.

This week, we unfortunately encountered another round of problems in the server due to another last-minute commit (which made a change to the behavior the JVM used for contended read/write locks) just before the build was to be generated. This introduced some undesirable side effects in the replication subsystem. We think that we've got those problems worked out and have respun the build and will be kicking off the latest round of tests again. We'll hopefully have a new build available within the next couple of days.

When the build is released, I'll do a much more complete write-up on the changes that it includes, but some of the things that have been checked in since the last build include:
  • We've merged the QuickSetup install and upgrade utilities into a single tool
  • We've added support for nested static groups
  • We've added a basic attribute uniqueness plugin
  • We've added a new identity mapper that lets you use a regular expression to transform the provided identifier string.
  • We've annotated lots of our code to indicate what we intend to expose as part of our public API
  • We've added several utilities to help make it easier to use the server as an embedded application
  • We've added some basic capabilities to allow replication to automatically repair inconsistencies that may be detected
  • We've updated the replication code so that it's easier for administrators to determine if any conflicts have been detected
  • We've further compacted the way that we store information in the database so that entries are significantly smaller and faster to encode/decode
  • We've added command-line tools to help invoke import, export, backup, and restore operations through the tasks interface
  • We've added a command-line tool that can base64 encode and decode information
  • We've added a command-line tool that can generate an RC script to allow the server to automatically start at boot time on UNIX-based systems

Monday Aug 20, 2007

Internal operations in OpenDS

If you're interested in writing an extension to OpenDS (like a plugin, password validator, identity mapper, virtual attribute provider, etc.), then there's a decent chance that you'll want to be able to search for or make changes to content in the directory as part of your processing. And if you intend on using OpenDS in an embedded manner, then it's almost certainly going to be a requirement for you. One option could be to simply establish a connection to the LDAP listener and issue a request, but that's inefficient and can also have undesirable side effects (e.g., if your plugin creates a new LDAP operation, then the server will try to invoke plugins on that operation, which can create a kind of infinite recursion loop). A better alternative, however, is to perform internal operations within the server. Performing an internal operation is more efficient than creating a new external operation, and internal operations can do things that aren't allowed for external operations (e.g., update attributes marked as NO-USER-MODIFICATION) and they are also marked as internal operations so that it's possible to do things like skip plugin processing for them.


The Internal Client Connection Object

For quite a while now, OpenDS has provided the org.opends.server.protocols.internal.InternalClientConnection class, which offers a relatively simple way to perform internal operations in the server. To use it, you first need to obtain an internal client connection, which you can do in one of the following ways:
  • If you don't need to worry about enforcing access control for the internal operation, you can use the InternalClientConnection.getRootConnection() method. This will give you a connection established as an internal user with root privileges (bypass-acl, modify-acl, config-read, config-write, ldif-import, ldif-export, backend-backup, backend-restore, server-shutdown, server-restart, disconnect-client, cancel-request, password-reset, privilege-change, and unindexed-search) that can do just about anything in the server.

  • If you do want access control enforced for the internal operation (e.g., only do this operation if the authenticated user has the right to do it), then you will want to get an internal client connection as that user. To do that, you can create a new internal client connection with either the InternalClientConnection(DN) or InternalClientConnection(AuthenticationInfo) constructor.

Once you have a handle to the internal client connection, then you can use it to perform internal operations in the server. For example, to perform a simple search, you can use something like:
InternalSearchOperation searchOperation =
     internalClientConnection.processSearch("dc=example,dc=com", SearchScope.WHOLE_SUBTREE,
                                            "(uid=john.doe)");
for (SearchResultEntry matchingEntry : searchOperation.getSearchEntries())
{
  // Do something with the entry.
}

We've added lots of convenience methods in the InternalClientConnection class to help make performing internal operations easy, and we hope to have official documentation on this in the near future. For now, however, you can look at the Javadoc from our latest daily build to see what's available.


The Internal LDAP Socket

The internal operations API referenced above should be easy to use, and should allow you to do pretty much anything that you need. I would expect that someone who has ever done any programming involving an existing LDAP SDK (e.g., JDNI or the Mozilla LDAP SDK for Java) should find it especially easy to pick up. However, last week I was talking with some people who were interested in embedding OpenDS and had a bit of a dilemma because they wanted to be able to embed OpenDS and communicate efficiently with it, but they also wanted to have the ability for their application to communicate with external directory servers, and they wanted to avoid having to write all of their client code twice (once using the OpenDS internal API, and another time using the Mozilla LDAP SDK for Java which is their API of choice for external LDAP communication). I gave this a little thought and came up with a solution that I think is a rather elegant approach to the problem: I created a custom socket implementation that is able to convert LDAP requests into internal operations, process the operation, and then convert the response back into LDAP.

This code is implemented in the org.opends.server.protocols.internal.InternalLDAPSocket class, which extends the java.net.Socket superclass. Going along with this socket (and doing all the real work behind the scenes) are the InternalLDAPInputStream and InternaLDAPOutputStream classes. As long as the LDAP SDK that you are using supports the use of a custom socket factory, you can use this custom socket implementation to allow you to use that LDAP SDK to perform internal operations. Both JNDI and the Mozilla LDAP SDK for Java support this, and I've tested my implementation with both of them.

For JNDI, if you want to use this custom socket implementation, the only thing that you have to do out of the ordinary is to make sure that your environment properties have the "java.naming.ldap.factory.socket" property set to "org.opends.server.protocols.internal.InternalLDAPSocketFactory". For example:
Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY,
        "com.sun.jndi.ldap.LdapCtxFactory");
env.put("java.naming.ldap.factory.socket",
        "org.opends.server.protocols.internal.InternalLDAPSocketFactory");
env.put(Context.PROVIDER_URL, "ldap://doesntmatter:389/");
env.put(Context.SECURITY_AUTHENTICATION, "simple");
env.put(Context.SECURITY_PRINCIPAL, "cn=Directory Manager");
env.put(Context.SECURITY_CREDENTIALS, "password");

DirContext context = new InitialDirContext(env);
// Do whatever you want with the connection here

For the Mozilla LDAP SDK for Java, there's just a tiny bit more that you have to do, because it uses its own socket factory implementation (netscape.ldap.LDAPSocketFactory) instead of the one provided by the JDK (javax.net.SocketFactory, although to be fair that class didn't exist when the Mozilla SDK was written). That interface requires a single makeSocket method, which you can implement as follows:
public Socket makeSocket(String host, int port)
{
  return new InternalLDAPSocket();
}

When you're creating a connection with the Mozilla LDAP SDK for Java, then all you need to do is to use the LDAPConnection(LDAPSocketFactory) constructor to have it use that socket factory for its communication. As an example, I have written a simple EmbeddedMozillaLDAPSDKTest program that can be used to start an embedded OpenDS instance, perform some internal operations using the Mozilla LDAP SDK for Java, and shut down the server. In order to create an appropriate environment for this test, all I needed to do was:
  1. Create an empty directory to use as the server root.
  2. Copy the config directory structure from an OpenDS installation into the new server root.
  3. Create a lib directory below the new server root and copy all of the lib/\*.jar files from an OpenDS installation into it.
  4. Create empty db, logs, and locks directories below the new server root
Then, when I compiled and ran the program, it started the server, interacted with it using the Mozilla SDK, and shut it down again. Note that I could have trimmed things down even more if I wanted to put more effort into it (e.g., we don't need everything under the config directory, and if we disable loggers then we wouldn't have needed the logs directory, etc.), but I wanted to keep this example simple.


Limitations of the Internal LDAP Socket Implementation

In general, if you're using the internal LDAP socket implementation you can do pretty much anything that you could do if you were talking to OpenDS using actual LDAP network communication. However, there are a few important things to note about this implementation:
  • You can only use clear-text communication when interacting with the Directory Server. This implementation doesn't support the use of SSL or StartTLS to secure the communication, but that really shouldn't be a problem since there's no actual communication and the operations never leave the JVM.

  • This implementation does not support the use of SASL authentication, so only simple binds are allowed. Although technically it would have worked with some mechanisms (e.g., ANONYMOUS, CRAM-MD5, DIGEST-MD5, and PLAIN), others (in particular, EXTERNAL and GSSAPI) would not have worked. Again, given that all the communication is purely internal to the JVM, I really didn't see a need to use anything other than simple authentication, so that's all that is exposed.

  • Abandon operations don't do anything. OpenDS doesn't provide a mechanism for abandoning internal operations, so any abandon operation requested in this manner will simply be ignored. Although I haven't tested it, I would expect that attempts to use the LDAP cancel extended operation would probably be rejected with a "cannot cancel" result.

Friday Aug 17, 2007

Tips on using OpenDS in an embedded manner

Later today, I'm going to be talking to a group that is interested in using OpenDS with their application. One of the key things that interests them is the fact that it can be used as a fast, scalable, replicated data store that they can embed as part of their application. Unfortunately, there is not a lot of documentation available about how to do this, and until recently there weren't a lot of facilities in the code to help out with that either, which means that even though it was possible you had to get pretty familiar with the code in order to figure out how to use OpenDS in an embedded manner.

Fortunately, things are starting to change in this area. Some of the things that we've done recently around this include:
  • We've created an org.opends.server.util.EmbeddedUtils class that can be used to start, stop, and restart the server in an embedded environment, as well as to check whether it's currently running and also to initialize server data structures if you just want to use the libraries for some reason without actually running the server.

  • We've started to identify which classes are part of our public API. We have created an @PublicAPI annotation type, which shows up in the Javadoc documentation and is available via reflection. We hope to be able to use this to write tools that can examine third-party code to see if they're using any private or unstable interfaces.

  • We've added a few convenience methods in our internal operations API that can be used to help make it easier to invoke operations in the server using method calls rather than protocol-level communication.

The slides that I'm going to be using to discuss using OpenDS in an embedded manner are available in this PDF. It's just an overview of the facilities that we have in place and what's coming, and hopefully we'll have even more improvements in this area in the weeks to come.

Friday Aug 03, 2007

OpenDS 1.0.0-build003 is now available

I have just uploaded OpenDS 1.0.0-build003, built from revision 2550 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/63458/OpenDS-1.0.0-build003.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/63459/OpenDS-1.0.0-build003-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://builds.opends.org/weekly-builds/1.0.0-build003. Some of the changes that have been incorporated since OpenDS 1.0.0-build002 include:
  • Revision 2446 (Issue #1986) -- Update the graphical and command-line setup mechanisms to attempt to detect and prevent install paths into directories containing a percent sign, especially on Windows.

  • Revision 2447 (Issue #2006) -- Update QuickSetup to prevent users from choosing the same ports for different protocols (like LDAP and LDAPS).

  • Revision 2448 (Issue #221) -- Add a general framework that may be used to allow OpenDS to send e-mail messages. Also, add an SMTP alert handler that can be used to send e-mail messages in response to administrative alerts generated within the server.

  • Revision 2449 (Issue #452) -- Add a new "targetcontrol" facility to the access control framework that can be used to restrict which clients are allowed to use specified request controls.

  • Revision 2457 (Issue #1819) -- Provide the ability to define certain configuration options as "advanced". This will make it possible to hide these options in administrative interfaces like dsconfig unless the user explicitly requests to see advanced options.

  • Revision 2461 (Issue #1849) -- Fix a problem with the evaluation of the debugsearchindex operational attribute (used to provide information about index processing performed during a search) in which evaluation was not correct with double-negation (a "NOT" inside a "NOT").

  • Revision 2475 (Issue #1015) -- Update a number of the command-line tools provided with OpenDS so that they include improved debugging support. When used in "--verbose" mode, they can now trace the contents of incoming and outgoing LDAP messages and ASN.1 elements in many cases.

  • Revision 2479 (Issue #443) -- Add a new access control "extop" keyword that can be used to restrict which extended operations may be invoked for a given client.

  • Revision 2480 (Issue #1831) -- Provide initial support for an "--interactive" mode for the dsconfig tool. This provides a basic text-based, menu-driven interface for interacting with the server configuration, although there are a number of known issues that still need to be addressed with this capability.

  • Revision 2483 (Issue #1971) -- Allow partial non-append imports for a backend with multiple base DNs (i.e., allow content under one base DN to be imported without impacting content below other base DNs in the same backend).

  • Revision 2499 (Issue #38) -- Provide a mechanism for performing VLV indexing, which makes it possible to efficiently use virtual list view and server-side sorting for searches that might not otherwise be indexed and/or inefficient.

  • Revision 2500 -- Fix a problem in access control evaluation in which use of the targetattr keyword to match all attributes except a named list could incorrectly grant access to operational attributes.

  • Revision 2503 (Issue #429, 478, 2025) -- Add support for a new disconnect client task that can be used to allow an appropriately-privileged administrator (needing at least the "disconnect-client" privilege) to terminate a client connection if the need arises. Also, add a "Get Connection ID" extended operation that can be used to allow a client to get the connection ID associated with its connection.

  • Revision 2505 (Issue #2024) -- Implement support for restricting the set of tasks that can be invoked in the server. Only those tasks which are listed in the ds-cfg-allowed-task attribute in the cn=config entry may be invoked in the server.

  • Revision 2508 (Issue #1683) -- Provide a mechanism to disable privileges in the server if necessary. If a given privilege is disabled, then it will be assumed that all clients have that privilege.

  • Revision 2509 (Issue #1787) -- Provide a configuration option that makes it possible for an administrator to control whether responses to failed bind operations should be allowed to include an error message that explains the problem. By default, this message will not be included for security reasons, but administrators may configure the server to allow it to be sent to provide the client with information about the reason for the failure.

  • Revision 2512 (Issue #2027) -- Provide a way to configure each alert handler with an explicit set of alert types that will be allowed or ignored. This makes it possible to restrict the types of alerts that a given alert handler will be asked to process.

  • Revision 2514 (Issue #118) -- Update the server to provide support for an idle time limit configuration option for LDAP clients. That is, it is now possible to configure the server to automatically terminate a client connection if it has remained unused for too long. The idle time limit is a server-wide configuration option, but it can be overridden on a per-user basis.

  • Revision 2515 (Issue #2026) -- Ensure that processes launched by QuickSetup will use the same JVM as is used to run QuickSetup, even if an alternate JAVA_HOME is configured in the environment.

  • Revision 2521 -- Update the status panel utility so that it can communicate with the server over a secure communication channel. If the server certificate is not trusted, the user will be prompted about whether to accept it.

  • Revision 2529 (Issue #2033, 2034) -- Update the task backend to provide the ability to send an e-mail message whenever a task is complete. The set of recipients may be configured on a per-task basis, and it is possible to specify whether the message should always be sent or only if the task fails.

  • Revision 2533 (Issue #1991) -- Improve the "dsconfig list-properties" output and make its usage more consistent with other dsconfig subcommands.

  • Revision 2535 (Issue #2032) -- Update the password policy to ensure that the sum of the minimum password age and the password expiration warning interval should always be less than the maximum password age (if the corresponding options are configured). This will prevent undesirable configurations, like a minimum age that is greater than the maximum age, or sending expiration warning messages to the client during a time when the user is not allowed to change the password.

  • Revision 2539 -- Update the access control handler to provide the ability to control whether a smart referral (i.e., a named subordinate reference as per RFC 3296) should be returned to the client.

  • Revision 2542 -- Fix a problem with the way that the Netscape password expired control was being encoded to ensure that it always has an appropriate value.

OpenDS switching to bi-weekly builds

For the last year we have generally made pre-packaged OpenDS builds available once a week. There have been a few exceptions, but in general we've had a weekly build schedule. Starting with this week's build (or actually, with last week's lack of a build) we are changing to a bi-weekly (i.e., fortnightly -- once every two weeks, not twice a week) process. The main reason for this is that our QA team has started looking more closely at what goes into these builds and has continued to improve their test coverage. Since at least some of this isn't automated and involves manual testing, reducing the frequency of these QA-tested builds gives them more time to get other stuff done (like writing new automated test cases) in between verifying builds that we want to make public.

So what, if anything, does this change? Not a whole lot. Of course, since they have two weeks of effort instead of one, there will be a larger number of changes from one build to another. But at least for now, where you get the builds (https://opends.dev.java.net/servlets/ProjectDocumentList?folderID=5700&expandFolder=5700&folderID=0) won't change and there will still be places that refer to them as "weekly" builds, although we'll probably clean up these references at some point.

If the arrival of the latest OpenDS build is the highlight of your week and you're not sure how you'll be able to handle waiting twice as long between builds, you can always look at our daily builds (http://builds.opends.org/daily-builds/), or you can check out and build the code for yourself whenever you want. But for this week, you should be able to get your fix in a few minutes when we post OpenDS 1.0.0-build003.

Friday Jul 20, 2007

Configuring OpenDS with dsconfig

Like the Sun Java System Directory Server, the configuration in OpenDS is stored in an LDIF file rooted at "cn=config", and can be read and updated over LDAP. As of DSEE 6.0, the Directory Server also has a Web-based administration GUI, as well as a command-line utility named dsconf that can be used to manage the server configuration. For OpenDS, we don't yet have a widespread administration GUI, but starting with this week's build we have a new command-line tool named dsconfig that can be used to interact with the server configuration. On UNIX systems, you can find it at bin/dsconfig; on Windows, it's bat\\dsconfig.bat.

Matt Swift (who has done most of the development of the dsconfig tool, and the underlying administration framework) has written a document that describes the dsconfig utility and gives an overview of how to use it. You can find that on our documentation wiki at https://www.opends.org/wiki/page/ConfiguringOpenDSUsingTheDsconfigTool.

Note that the documentation page for this tool also includes a section at the bottom with some known issues and potential usability problems that we intend to fix in the near future. However, we would also appreciate any feedback that you might have (e.g., problems that you've encountered or suggestions for improvement). If you find anything, feel free to open a new issue in our issue tracker (https://opends.dev.java.net/servlets/ProjectIssues) or send an e-mail to dev@opends.dev.java.net.

OpenDS 1.0.0-build002 is now available

I have just uploaded OpenDS 1.0.0-build002, built from revision 2441 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/62416/OpenDS-1.0.0-build002.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/62417/OpenDS-1.0.0-build002-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://builds.opends.org/weekly-builds/1.0.0-build002. Some of the changes that have been incorporated since OpenDS 1.0.0-build001 include:
  • Revision 2382 (Issue #1897) -- Update the verify-index and rebuild-index utilities to include a "--countErrors" option that can be used to return a non-zero exit code if any errors are encountered.

  • Revision 2391 (Issue #1771) -- Update the entry cache implementations to ensure that they flush all entries from a backend whenever it is taken offline.

  • Revision 2405 (Issue #1988) -- Implement a monitor provider that can be used to publish information about the client connections that are currently established.

  • Revision 2422 (Issue #1953) -- Update the Berkeley DB JE backend so that if a problem occurs in the database that causes a RunRecoveryException to be thrown, the server will provide notification in the form of administrative alerts.

  • Revision 2424 (Issue #339) -- Implement support for password history functionality. The password history can be maintained either based on the number of previous passwords, or the length of time the previous passwords have been retained, or both.

  • Revision 2428 (Issue #1603) -- Make updates to the way that the server attempts to register the Windows service on Windows Vista.

  • Revision 2430 -- When running the QuickSetup installer, if there are no backends found to replicate then disable the "Replicate Suffix" option and automatically select the "Create New Suffix" option.

  • Revision 2437 (Issue #90) -- Update the server to provide more complete support for the password policy control as defined in draft-behera-ldap-password-policy.

  • Revision 2438 -- Update the graphical tools to use the term "Base DN" instead of "Suffix".

  • Revision 2439 -- Update the server so that the set of alert handlers are configurable rather than always using a hard-coded JMX alert handler.

  • Revision 2441 -- Expose the dsconfig tool in the Directory Server build.

Friday Jul 13, 2007

OpenDS 1.0.0-build001 is now available

After last week's release of OpenDS 0.9.0, we've started the push toward the 1.0.0 release, and the first build toward that milestone is now available. Since the 0.9.0 release, we have been primarily focused on fixing bugs as part of our Summer 2007 Bugfest, and this build includes the fix for over 80 bugs of varying severities.

This week's build, OpenDS 1.0.0-build001, is based on revision 2381. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/61938/OpenDS-1.0.0-build001.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/61939/OpenDS-1.0.0-build001-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://builds.opends.org/weekly-builds/1.0.0-build001. Some of the changes that have been incorporated since OpenDS 0.9.0 include:
  • Revision 2238 -- Perform a refactoring of much of the core server to provide the framework that will be needed to support, proxy, distribution, and virtual functionality.

  • Revision 2254 -- Display an "Open" dialog box rather than a "Save" dialog when prompting the user to select the desired directory in the QuickSetup tool.

  • Revision 2255 (Issue #1812) -- Fix a problem in which the server could become blocked if it was configured for replication but none of the replication servers were available.

  • Revision 2257 (Issues #1325, 1850, 1863) -- Provide the ability to create a basic replication environment at install time using the QuickSetup utility. Provide the ability to cancel an installation process midway through, even after potentially making changes to remote servers. Provide better support for merging the QuickSetup and QuickUpgrade functionality into a single tool.

  • Revision 2259 (Issue #577) -- Update the server so that it uses a non-success result for operations that include the LDAP no-op control. This control isn't completely standardized and there is no official result code reserved for it, but we have decided to use the same result code that OpenLDAP uses (result code 16654) until an official result code is assigned.

  • Revision 2260 (Issue #1476) -- Update the SASL ANONYMOUS mechanism handler so that any trace string provided in the client credentials will be included in the access log.

  • Revision 2262 (Issues #1565, 1848) -- Update the LDAP filter processing code so that decoding string filters will be more strict and more correct. This will catch filters that were invalid but previously not rejected.

  • Revision 2263 (Issue #1879) -- Fix potential null pointer exceptions in the replication code finalization on systems with poorly-configured name resolution.

  • Revision 2264 (Issue #670) -- Update the support for access control rules based on the client's IP address to handle IPv6 addresses.

  • Revision 2267 (Issue #1856) -- Use more correct and robust argument parsing for the status utility.

  • Revision 2268 (Issues #1874, 1875) -- Provide better error handling in the replication domain initialization process.

  • Revision 2269 (Issue #1473) -- Fix a problem with inconsistent format checking for the targattrfilters ACI keyword.

  • Revision 2272 (Issue #1876) -- Implement a change in replication that can help improve performance in the case where the flush to the replication server database is a bottleneck.

  • Revision 2274 (Issues #1423, 1839, 1871) -- Eliminate the ConfigurableComponent interface, as it has been replaced by the ConfigurationChangeListener interface and is no longer needed.

  • Revision 2276 (Issue #1836) -- Update the import-ldif utility so that it includes a new "--countRejects" option that can be used to count the number of rejected entries. That value will be used as the exit code, and can be used in scripts to determine if there were any failures during the import.

  • Revision 2277 (Issue #1883) -- Update all of the command-line utilities to ensure that the exit code should always be between 0 and 255. Any value that would have been outside that range will be changed to 255.

  • Revision 2278 (Issues #704, 705, 706, 918, 1901) -- Fix a number of issues related to search filter processing.

  • Revision 2279 (Issue #631) -- Update the password modify extended operation so that it properly sets password policy state attributes that were not previously updated, including last login time if the user provides the current password and it is correct, or the auth failure times if the user provides an incorrect password.

  • Revision 2280 (Issue #1898) -- Fix a potential problem when encountering replication conflicts for an entry that has been deleted.

  • Revision 2282 (Issue #1789) -- Use a more appropriate result code when interacting with the filesystem entry cache if there is a problem with its configuration.

  • Revision 2285 (Issue #1872) -- Use valid OIDs for the historical CSN ordering matching rule and the associated attribute syntax.

  • Revision 2287 (Issue #1632) -- Skip schema checking for replicated operations.

  • Revision 2289 (Issue #874) -- Fix a problem in which the ldifsearch utility always returned the object class attribute even if it should not have been included in the results.

  • Revision 2303 (Issue #1474) -- Use a more user-friendly message for the case in which an LDAP client loses a connection to the Directory Server.

  • Revision 2305 (Issue #1906) -- Update the server to reject requests from LDAPv2 clients if those requests contain controls. Previously, the server would only strip out response controls, but this had the potential to hide problems, and the new behavior is more consistent with that exhibited by DSEE.

  • Revision 2306 (Issue #1907) -- Update the way that the server handles search requests with the virtual list view control to be more forgiving with target ranges that are out of the bounds of the actual result set.

  • Revision 2307 (Issue #1908) -- Update the default access logger so that it will include an attrs="ALL" element in the case that the client did not request any specific attributes to return for a search operation.

  • Revision 2313 (Issue #1403) -- Update the file-based trust manager provider so that it will reject certificates that are expired or not yet valid.

  • Revision 2314 (Issue #1909) -- Update the LDIF reader so that it has the ability to perform syntax validation as well as schema checking. It will honor the ds-cfg-invalid-attribute-syntax-behavior configuration option.

  • Revision 2315 (Issue #1911) -- Update the MakeLDIF tool so that the "random:telephone" tag will generate telephone numbers that include the country code so that they are more compliant with the ITU-T E.123 specification.

  • Revision 2316 (Issue #1231, 1234) -- Update the LDAP and JMX connection handlers so that they attempt to bind a server socket to the configured port for all appropriate addresses during the initialization phase. This should provide a more reliable mechanism for determining whether the connection handler will be allowed to start.

  • Revision 2317 (Issue #1861) -- Update the configuration file handler so that it will report back to clients if a problem occurs while applying a configuration add, delete, or modify after the new configuration has been written to disk.

  • Revision 1423 (Issue #1428) -- Update the import-ldif utility to provide the ability to write skipped entries to a specified file.

  • Revision 2326 -- Update the server to provide a new ds-cfg-strip-syntax-minimum-upper-bound configuration attribute to strip the suggested minimum upper bound from the attribute type syntax OID. This may cause problems with certain APIs (e.g., JNDI).

  • Revision 2334 (Issue #1880) -- Provide better error handling for runtime exceptions encountered during the JMX connection handler.

  • Revision 2337 (Issues #1861, 1932, 1936, 1937) -- Fix a set of problems with the configuration interface in which insufficient validation was performed when applying configuration changes.

  • Revision 2338 (Issue #1368) -- Update the Berkeley DB Java Edition backend to ensure that the matched DN component of the request is set when appropriate.

  • Revision 2340 (Issue #1462) -- Fix a problem that could cause a deadlock in the server when performing a subtree delete operation.

  • Revision 2342 (Issue #1899) -- Update the configuration for the Berkeley DB Java Edition backend to ensure that it will require a valid value for the database logging level.

  • Revision 2345 (Issue #987) -- Update all of the tools provided with OpenDS to make them easier to invoke programmatically.

  • Revision 2346 (Issue #1957) -- Change the log level for many of the messages generated by the import-ldif utility to reduce the amount of output generated by default.

  • Revision 2348 (Issue #1238) -- Update the server's DIGEST-MD5 SASL mechanism handler so that it provides the ability to process the digest-uri element of the request if a value is provided for the ds-cfg-server-fqdn configuration attribute.

  • Revision 2351 (Issue #1321) -- Update the access and audit logging systems to provide a way to control whether synchronization messages should be logged. If they are (which is the default configuration), then those messages will be noted with a "type=synchronization" flag.

  • Revision 2354 (Issue #1916) -- Fix the manage-account tool to use the correct underlying Java class.

  • Revision 2355 -- Fix a problem with the way that the server encoded the "reverse order" flag in the server-side sort control.

  • Revision 2359 (Issue #1972) -- Update the bind processing code so that the "bind in progress" flag will not be incorrectly unset between stages of a multi-stage SASL bind.

  • Revision 2363 (Issue #1810) -- Update the password policy configuration to support a new ds-cfg-state-update-failure-policy that can make it possible to configure whether an otherwise successful bind operation should fail if a problem occurs while attempting to update the password policy state information for the user.

  • Revision 2364 (Issues #1588, 1589) -- Improve the locking code used for server entry cache implementations to simplify the code and eliminate the potential for returning a stale entry.

  • Revision 2365 (Issue #1974) -- Fix a problem that prevented attribute syntaxes and matching rules from being notified of configuration changes.

  • Revision 2368 (Issue #1217) -- Update the JMX connection handler to ensure that any client connecting to the server using JMX will be required to have the jmx-read privilege. The jmx-notify privilege has been deprecated, as Java does not provide a reliable mechanism for trapping client registration to receive notifications.

  • Revision 2373 (Issue #1895) -- Fix a problem in which total update initialization did not work properly with environments containing three servers that are also replication servers.

  • Revision 2377 (Issue #614) -- Update the LDAP connection handler to eliminate the possibility of a race condition if a client sent a second request with the same message ID as the first request while the server was still performing post-response processing for the first operation.

Monday Jul 02, 2007

OpenDS 0.9.0 is now available

I have just uploaded OpenDS 0.9.0, built from revision 2217 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/61128/OpenDS-0.9.0.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/61129/OpenDS-0.9.0-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://builds.opends.org/weekly-builds/0.9.0/. Some of the changes that have been incorporated since OpenDS 0.9.0-build004 include:
  • Revision 2116 (Issue #1596) -- Fix problems in the access control handler in which it was not possible to use the userattr bind target if it contained an LDAP URL.

  • Revision 2118 (Issue #1797) -- Fix problems leading to potential deadlocks in the replication subsystem.

  • Revision 2127 (Issue #1799) -- Fix a problem with the upgrade utility that could cause upgrades to fail if SSL was enabled.

  • Revision 2131 -- Improve the replication monitoring information that the server makes available, particularly in the case of a replication server that is not connected to any directory server instances.

  • Revision 2135 (Issues #1455, 1480, 1575) -- Make a number of changes to the Berkeley DB JE backend, including adding the ability to manage a number of settings with the server online (e.g., indexes, entry index limit, substring length, entry compression).

  • Revision 2138 (Issue #1578) -- Support the use of alternate root bind DNs in the userdn bind target.

  • Revision 2143 -- Update the upgrade process to ensure that any files present in the previous installation's config directory but not in the config directory for the new instance are properly restored.

  • Revision 2144 -- Make the QuickSetup and QuickUpgrade dialog boxes use the same height.

  • Revision 2149 (Issue #1801) -- Improve the error message that is generated from the start-ds script if there is a problem with the Java environment. It should now say "Java 5.0 or higher" instead of just "Java 5.0", and if there is a chance that the problem was with the JAVA_ARGS values, then the message will also indicate that could be the culprit.

  • Revision 2157 (Issue #253) -- Provide a way to control the order in which plugins are invoked.

  • Revision 2161 -- Update the set of plugins included with the server so that they will look for changes to the set of registered plugin types and reject the change if any of the new plugin types are not appropriate.

  • Revision 2166 (Issue #751) -- Update the pre-parse and pre-operation plugin result objects to add the ability to skip core processing without skipping post-operation plugins. Also, make sure that post-response plugins are always invoked for all operations that get far enough in their processing to have called the pre-parse plugins.

  • Revision 2176 -- Add a new configuration option that makes it possible to control how the server should handle changes if it is unable to connect to a replication server.

  • Revision 2186 (Issue #849) -- Update the way that the password policy import plugin encodes passwords during an LDIF import. If an entry specifies a specific password policy, then its default storage schemes will use. Otherwise, it is possible to specify the default schemes that should be used. If all else fails, then the server will fall back on hard-coded default schemes (SSHA for user password values, and SHA1 for auth password values).

  • Revision 2190 -- Fix a problem in the server's configuration handler in which configuration changes made by internal operations may not be persisted to disk due to a failure to close the output stream.

  • Revision 2201 (Issues #292, 579, 1782, 1845) -- Add a new extended operation that can be used to interact with password policy state information, including getting and setting various state variables. Also, add a new manage-account tool that can be used to interact with this extended operation. Finally, fix a bug in which it was not possible for users to authenticate if their account had the pwdReset flag and last login time tracking was enabled.

  • Revision 2202 (Issue #461) -- Update the way that the server handles access control rules that include criteria involving an IP address so that its behavior is consistent with that exhibited by DSEE.

  • Revision 2208 (Issue #1674) -- Provide the ability to cancel install and upgrade processing.

  • Revision 2209 (Issue #1815) -- Fix a problem in which the uninstaller has the potential to remove server configuration and/or log files even if the user selected to retain them.

Friday Jun 15, 2007

OpenDS 0.9.0-build004 is now available

I have just uploaded OpenDS 0.9.0-build004, built from revision 2108 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/59774/OpenDS-0.9.0-build004.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/59775/OpenDS-0.9.0-build004-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

This week, we are also making an upgrade utility available for use that can be used to upgrade an existing OpenDS installation to the latest version. This also works via Java Web Start, and you can launch it using the URL http://www.opends.org/install/QuickUpgrade.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickUpgradeTool for more information. Note that there is a known problem with the upgrader that prevents it from working if the existing instance has SSL/StartTLS enabled, and we hope to have that fixed in the next build.

Detailed information about this build is available at http://www.opends.org/weekly-builds/0.9.0-build004/. Some of the changes that have been incorporated since OpenDS 0.9.0-build003 include:
  • Revision 2056 (Issue #1778) -- Fix a problem in which setup appears to hang on Windows with no Windows service configured.

  • Revision 2057 -- Update the QuickSetup UI so that the user can choose whether to run OpenDS as a service when installing on Windows. This makes it possible to avoid updating the Windows service registry (which may be desirable during product evaluation, as well as under other conditions). It also makes it possible to use the server on Windows Vista, as the current service framework is not compatible with Vista.

  • Revision 2058 (Issue #1745) -- Fix a problem that may occur when using the get effective rights control with requested attributes of "\*" (all user attributes) and "+" (all operational attributes).

  • Revision 2059 (Issue #1780) -- Fix a problem that was preventing the server from writing messages to the error log. Also, improve performance when logging is disabled.

  • Revision 2062 (Issue #1781) -- Update the password policy so that it uses the correct calculation when determining the length of time that the account will remain locked due to authentication failures (only values reported to clients were inaccurate -- the server enforced the correct lockout duration).

  • Revision 2063 (Issue #1775) -- Fix a problem in which an incorrect database comparator could be used, which could cause entries to be returned in a problematic order (including subordinate entries returned before their parents).

  • Revision 2064 (Issues #1767, 1768) -- Update the filesystem entry cache implementation so that it will default to running in a disabled state if no valid database directory is specified, rather than defaulting to a hard-coded location. Also, eliminate a message about an invalid database checksum that could be displayed the first time the server was started.

  • Revision 2068 (Issue #604) -- Introduce a new operational attribute that can be used to mark entries for which replication conflicts could not be automatically resolved.

  • Revision 2086 -- Upgrade the version of the Berkeley DB Java Edition that we are using from 3.2.21 to 3.2.29. This update primarily includes bug fixes.

  • Revision 2094 (Issue #1779) -- Add support for "+" in the targetattrs element of access control rules. Also, make it possible to use "\*" in conjunction with other attributes, like "\*||entryUUID" or "\*||+".

  • Revision 2107 (Issue #1795) -- Update the server schema processing code so that it will detect the case in which schema configuration files contain more than one entry (or one entry plus other, unparseable data) and log a warning message at server startup.

Friday Jun 08, 2007

OpenDS 0.9.0-build003 is now available

I have just uploaded OpenDS 0.9.0-build003, built from revision 2052 of our source tree, to our weekly builds folder. The direct link to download the core server is https://opends.dev.java.net/files/documents/4926/59198/OpenDS-0.9.0-build003.zip. The direct link to download the DSML gateway is https://opends.dev.java.net/files/documents/4926/59199/OpenDS-0.9.0-build003-DSML.war.

I have also updated the archive that may be used to install OpenDS via Java Web Start. You may launch that using the URL http://builds.opends.org/install/QuickSetup.jnlp, or visit https://www.opends.org/wiki/page/OverviewOfTheQuickSetupTool for more information.

Detailed information about this build is available at http://builds.opends.org/weekly-builds/0.9.0-build003/. Some of the changes that have been incorporated since OpenDS 0.9.0-build002 include:
  • Revision 1997 (Issue #1749) -- Update the way that privileges are evaluated by the server. Previously, they were always evaluated based on the authentication identity. Now, all privileges except proxied-auth are evaluated based on the authorization identity.

  • Revision 2000 (Issue #1760) -- Fix a problem in the access control implementation that could prevent the use of operational attributes in the userattr bind rule.

  • Revision 2001 -- Rename the default error log file name from "error" to "errors" in order to be more consistent with other products.

  • Revision 2002 (Issue #1758) -- Update the server to provide a lockdown mode. This is a mode in which the server will only allow client connections over loopback interfaces, and will reject all requests from non-root users. A task has been added that can allow an administrator to manually enable or disable this mode, and an internal API is available to expose it to other server components.

  • Revision 2004 (issue #609) -- Update the replication mechanism to provide modify conflict resolution for single-valued attributes. This uses a different mechanism than for multi-valued attributes and can allow the server to maintain less historical information for the attribute.

  • Revision 2009 (Issue #1761) -- Fix a problem that could prevent the QuickSetup installer from running properly (especially on Windows systems) if JAVA_HOME is not set.

  • Revision 2010 -- Fix a problem in the error logger that prevented an override severity of "all" from being handled properly.

  • Revision 2011 (Issue #1753) -- Fix a problem on Windows systems where manually running the setup utility where arguments could be incorrectly interpreted.

  • Revision 2017 (Issue #1601) -- Update the QuickSetup and Status Panel tools to improve the way that they handle focus changes between components so that it is easier to interact with these tools using only the keyboard.

  • Revision 2021 (Issue #1616) -- Update the QuickSetup tool so that it will always provide a button that can be used to launch the status panel even if the installation fails.

  • Revision 2024 (Issue #1634) -- Update the GUI tools so that when a text field gets input focus, its text is automatically selected.

  • Revision 2025 (Issue #1764) -- Fix a problem in the replication initialization where it can enter an infinite loop if there is no replication server available.

  • Revision 2026 (Issue #1117) -- Provide an entry cache implementation that is backed by a Berkeley DB JE instance. The backing database can be placed on a tmpfs or other kind of memory-based filesystem to allow for a space-efficient caching mechanism.

  • Revision 2042 (Issue #1750) -- Update the access control handler so that if it encounters any access control rules that cannot be parsed when the server is starting up, they will be logged and the server will be placed in lockdown mode. This will help avoid problems in which an incorrectly-specified access control rule wouldn't be enforced as an administrator intended and inadvertently grant too much access to users.

  • Revision 2045 (Issue #1729) -- Make changes to the server to allow for better integration with the Penrose virtual directory product.

  • Revision 2046 (Issue #1633) -- Ensure that the JMX connection handler is disabled by default. Given that there is currently no way to configure it in the QuickSetup utility, it is better to have it disabled than running, potentially without the administrator knowing about it.

  • Revision 2048 -- Update the global ACI definitions so that they allow read access to the entryUUID operational attribute.

  • Revision 2049 (Issues 660, 1675, 1770) -- Provide a new mechanism for encoding entries. This provides a mechanism for excluding the DN from the encoded entry (which can be helpful for the filesystem entry cache), and also for compressing the object class sets and attribute descriptions to conserve space and improve encode/decode performance.

  • Revision 2050 (Issue #1775) -- Add a virtual attribute provider that can be used to assign entryUUID values for entries in private backends. The entryUUID values for these entries will be based on an MD5 digest of the normalized DN, but this should not present an instability problem because these entries aren't allowed to be renamed.

  • Revision 2051 (Issues #1765, 1776) -- Eliminate the search-unindexed privilege, since the unindexed-search privilege was added to do the same thing. Also, eliminate the index-rebuild privilege and fold all of its functionality into the ldif-import privilege, since having it as a separate privilege didn't add much value and created unnecessary administrative overhead.

  • Revision 2052 -- Update the entry cache initialization process so that a default entry cache is always instantiated before the backends are brought online. This helps avoid problems in backends that attempt to interact with the cache before the full entry cache initialization is complete.

Monday Jun 04, 2007

A Comparison of OpenDS and DSEE Functionality

I've been asked to give a brief presentation on OpenDS. The target audience is a group of people that are presumably already familiar with our existing DSEE product, so there's not much need to go into detail on what a directory is and why you might need one. I've also only been given about 10 minutes to talk about it, so I can't go into a lot of detail. With this in mind, I thought that one of the most pertinent topics to cover is a quick overview of where OpenDS is today in comparison to DSEE.

I have uploaded the slides for this presentation so that they are available at http://blogs.sun.com/DirectoryManager/resource/OpenDS-and-DSEE-Comparison-20070604.pdf. I apologize that it's relatively light on content, but with only ten minutes I don't have time for much more than a very high-level overview. More detailed information is available at https://www.opends.org/wiki/page/DSEE6VersusOpenDSFeatureComparison, and you can also check our issue tracker at https://opends.dev.java.net/servlets/ProjectIssues for even more detail.

About

cn_equals_directory_manager

Search

Top Tags
Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today