Tuesday Apr 08, 2014

OUD External change log and rootDSE search

Some LDAP client applications perform subtree searches with search base set to the rootDSE (empty DN).
Oracle Unified Directory (OUD) nicely routes the search to every top level suffix automatically.

When the replication is enabled, OUD automatically publicizes all changes that have occurred in a directory server database in the cn=changelog suffix. This is particularly useful for synchronizing the LDAP directory with other subsystems.  The cn=changelog suffix may contains millions of changes depending on the modification rate on the replication topology and the change retention policy (purge delay).

Subtree searches with search base set to the rootDSE are routed to the cn=changelog suffix as well as long as the replication is enabled. In general, this is not a problem in testing/stagging area, because the changelog is almost empty. However, in production, this may have big impact on performances as this suffix may contain many entries. Furthermore, custom  indexes corresponding to client access pattern do not exist on that suffix, so they can't be used to speed up entry processing.

In order to address that problem, you can disable the so-called external changelog, without disabling the underlying replication changelog used by the replication. To do so, run the following command on the OUD servers for each user suffixes:

dsconfig -h <hostname> -p <admin port>  -D "cn=directory manager" -w <admin password> -n \
  set-external-changelog-domain-prop \
  --provider-name "Multimaster Synchronization" --domain-name <your suffix>  \
  --set enabled:false

Note: some provisoning apps may require the external changelog to synchronize with external systems. If so, keep the external changelog enabled on a couple of OUD servers and reserve them for these apps.


Monday Aug 06, 2012

OUD Database Indexes & Performance Tuning

Oracle Unified Directory (OUD) database indexes enable (search) directory requests  to be processed efficiently. 

Indexes are files that contain lists of values, where each value is associated with a list of entry identifiers to suffixes in the directory server database. When the directory server processes a search request, it searches the database using the list of entry identifiers in the indexes, thus speeding up the search. If indexes did not exist, the directory server would have to look up each entry in the database, which dramatically degrades performance.

First of all, unindexed searches are rejected by default unless the (authenticated)  user has the privilege to perform them because unindexed searches  may negatively impact overall directory performances. In some circumstances, unindexed searches are a valid approach, so it is possible to explicitely  assign the unindexed-search privilege to a user or a group of user. A user attempting to perform an unindexed search gets the message "you do not have sufficient privileges to perform an unindexed search".

Note that unindexed searches always complete sucessfully when the amount of entries in the database does not exceed the value if index-entry-limit (4000 by default), which is unlikely to be seen in real deployments.

The verify-index command can be used to check the consistency between the index and entry data within the directory server database. This command also provides information about the number of index keys that have reached the index entry limit (See below).

The rebuild-index command can be used to rebuild an index. It is useful in the following cases:

  • When the index-entry-limit property of an index changes

  • When a new index is created

The index-entry-limit tuning parameter (known as AllIDs threshold in Oracle Directory Server Entreprise Edition) controls the maximum numbers of entries kept in an index record as scaning the whole database may become a more efficient option when the number of entries associated with a given index value becomes huge.

Tuning indexes upfront may not be an obvious task, that's the reason why OUD provides you with  both  static and  dynamic index analyser tools:

The static analyser is delivered as part of the dbtest tool.  The dbtest list-index-status -n <dbName> -b <dbSuffix> command let you figure out how many entries are associated with each index value, and tune index-entry-limit when needed..

The dynamic analyser can be used to monitor live search filters received by OUD and corresponding  index usage: Live index analysis must be first enabled at the database level. At that point, subsequent searches hitting OUD are analysed and the corresponding statistics are kept in memory.  Statistics can be retrieved from the in-memory suffix cn=monitor as part of the database environmenent entry.

To enable live index data collection, run  the following command:

$ dsconfig set-workflow-element-prop --element-name userRoot  --set index-filter-analyzer-enabled:true  --set max-entries:100 -h localhost -p 4444 -D cn=directory\ manager -w <password>

The set-max-entries parameter controls how many statistical records are kept in memory.

To retrieve the results, run the following search:

$ ldapsearch -p 1389 -D cn=directory\ manager -w <password>  -b "cn=<dbName> Database Environment,cn=monitor"  "(objectclass=*)"  filter-usefilter-use: (description=*) hits:2 maxmatches:-1 message:presence index type is disabled for the description attribute

Important note:  It is not recommanded to enable statistic collection in a production system for a long time as it consumes ressources.

It is also possible to use the operational attribute debugsearchindex to figure out which entries are targetted by each component of a LDAP search filter. This is a lightweight yet convenient way to debug index-related issues.

$ ./ldapsearch -p 11389 -b "o=example" -D "cn=directory manager" -w *** "(&(uid=user.9*)(objectclass=*))" debugsearchindex
dn: cn=debugsearch
debugsearchindex: filter=(&(objectClass=*)[NOT-INDEXED](uid=user.9*)[COUNT:111]) [COUNT:111] scope=wholeSubtree[LIMIT-EXCEEDED:4101] final=[COUNT:111] 


About


I am Sylvain Duloutre, I work as a Software Architect in the Oracle Directory Integration Team, the customer-facing part of Directory Services & Identity Management Product Development, working on Technical Field Enablement.

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
9
10
11
12
13
14
16
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today