Thursday Jan 13, 2011

DPS and slow DNS

Every time a new client connection is treated, one of the tasks done is to log it to the access and connection logs. To log it, the canonical host name is retrieved, probably from the DNS server. If the call to InetAddress.getCanonicalHostName() gets blocked (for instance, because of a slow DNS server or a network problem), the calling Connection Handler Thread will get blocked for some time, preventing the thread from processing the other new connections and reading from the other connections.

In controlled environments, in may be useful to tune java dns caching, by setting the security property networkaddress.cache.ttl to a bigger value, or to -1 (cache forever).

You can't set the value of networkaddress.cache.ttl directly on the command line but you can set the required value in the file located in %JRE%\\lib\\security



Thursday Nov 18, 2010

Unable to retrieve backend SEARCH connection

That error message is pretty common with Directory Proxy Server. Here are the reasons why:

Upon reception of a request, DPS tries to route the request to the target data views, based on the request (base) dn. When this step fails (misconfiguration), a "No such Object" error is returned with additional text "The entry foo=bar is not handled by the server.
Then DPS selects the "best" directory server associated with the target data view according to the load-balancing algorithm. If no valid directory server is found, the error "Unable to retrieve backend SEARCH connection" is returned.

There are 4 possible causes:
- the data source pool associated with the target data view is empty (configuration problem)
- the data sources in that data source pool are disabled or have search-weight/bind-weight set to 0 so they are not considered valid servers to process search/bind operations (configuration problem)
- every data source (directory server) in that data source pool are down
- the max number of connection to every data source is reached.

If you get these errors after a brain new deployment with the OOTB dps configuration, #1 is probably the cause: DPS default config comes with a default data view able to handle every suffix/request. This data view is associated with a data souce pool that is empty by default. You have to create at least one data source object, add it to that data source pool then set load-balancing weights.

More info are available at

Monday Sep 27, 2010

Dynamic provisioning of directory instances with DPS - Part 2

The goal here is still (see previous post ) to dynamically  add/remove a directory server instance from the mesh with no or limited impact on the client applications and w/o altering the HW load-balancers that are commonly deployed in front of the directories. In the rest of this post, we assume that client applications access the directory services via an access layer provided by the Sun/Oracle Directory Proxy Server (DPS).

The second method consists in changing the configuration of each Directory Proxy Server. Each data source (directory server instance) is put in disabled mode so that DPS immediatly stops forwarding traffic to them. The property is-enabled can be changed to false. To reactivate a data source, set that property to true again.

e.g. dpconf  set-ldap-data-source-prop  is-enabled:false

Wednesday Jun 02, 2010

Directory Proxy Server at high connection rate

In some DPS deployments, there is a high number of new connections being established. This high connection rate may lead to performances issues.

It may be useful to consider changing the tcp time wait interval: That parameter notifies TCP/IP on how long to keep the connection control blocks closed. After the applications drop the TCP/IP connection, the control blocks are kept for the specified time.  When high connection rates occur, a large backlog of the TCP/IP connections accumulate and can slow server performance. The server can stall during certain peak periods. If the server stalls, the netstat command shows that many of the sockets that are opened to the server are in TIME_WAIT state

On Solaris, you can change the tcp_timeout_interval using ndd.

Monday May 25, 2009

How to change DPS log location

By default, DPS access and error logs are stored in <instance_dir>/logs.  Default setting can be changed using the commands below:

$ dpconf get-access-log-prop log-file-name
log-file-name  :  logs/access

$ dpconf get-error-log-prop log-file-name
log-file-name  :  logs/error 

$ dpconf set-access-log-prop log-file-name:/a/path/to/a/new/location/access

$ dpconf set-error-log-prop log-file-name:/a/path/to/a/new/location/error

Tuesday May 05, 2009

etime granularity in access logs

Unlike DS, DPS 6.3 etimes in the access log are in millisecs.

For sake of performance, DPS uses a timeThread that gets the time regularly (every 500ms only in DPS 6.3)  instead of performing a system call each time the current time is needed so etimes shown in access logs may be biased to a large extend and show 0ms when the effective etime is < 500ms. Conversaly , etimes may show a value of 500ms when the effective time is smaller.

TimeThread -----T1-----T2-----T3-----T4----T5    with (Ti- T()i-1)) = 500ms
DPS time retrieval example 1 |   |   => real value = 5ms, computed value is T2-T2 = 0
DPS time retrieval example 2    |       |   => real value = 15ms, computed value is T3-T2 = 500ms

The time "granularity" can be changed to accommodate with your needs in DPS 7.0 (available soon) but you can get a fix earlier (RFE 6601029) by contacting the Sun support team.

Monday Jan 26, 2009

Forum about LDAP, Sun Directory, Proxy Server and Virtual Directory

If you have some questions about Sun Directory Server Edition, Directory Proxy and Virtual Directory or you want to share best practices, don't hesitate to use the Sun Developer Forum dedicated to these products.

See you there!

Friday Jan 23, 2009

Fail-over based on the directory server operational state

DPS 6.x load-balancing / fail-over can be configured to stop sending traffic to a specific directory server instance in maintenance, even when that instance is up and running. For instance, you may want to remove a directory server instance from the mesh while an import or re index is in progress.

DPS 6.x periodically checks each ds server for availability by issuing (amongst others) a search request. A directory server instance is considered unreachable when that search fails or does not return any entry. By default, the search request hits the rootDSE entry. It can be configured to hit an entry in cn=config or cn=monitor to take into account the directory server operation state.

For more info, look at dpconf properties monitoring-search-filter and monitoring-entry-dn in the ldap data source object.

Friday Dec 19, 2008

DIT changes with dn virtual transformations

Here is a summary of a common deployment scenario with Sun Directory Proxy Server:

LDAP entries are grouped by location in the DIT, e.g user entries are located under ou=north,ou=people,dc=company, dc=com or  ou=south,ou=people,dc=company, dc=com or ou=east,ou=people,dc=company, dc=com or ou=west,ou=people,dc=company, dc=com based on user physical location.

Later, for sake of simplicity, the DIT is flatten so that every user entry is stored immediatly under ou=people, dc=company, dc=com

New applications are aware of the DIT structure change but DPS is used so that legacy applications expecting the location container node can operate w/o problem.

The dn mapping needed can be achieved by using virtual data transformations as described  in

Let's assume that
- you have a data view DV1 with viewBase (suffix) set to dc=company,dc=com.
- entry location (north, east,...) is always available in each entry in attribute 'location'
- entry uid=\*,ou=(north|south|east|west),ou=people,dc=company,dc=com mapped to uid=\*,ou=people,dc=company,dc=com

You have to create a virtual data transformation on the 'dn' for data view DV1. For inbound traffic (requests), the proxy must get rid of the ou=(north|south|east|west) node. For outbound traffic (responses), the proxy gerenates a (fake) ou=(north|south|east|west)  from the content of the 'location' attribute of each entry.

Here is the dpconf command to do that:

dpconf add-virtual-transformation -h <host> -p <port> -d <proxy manager> DV1 mapping attr-value-mapping dn internal-value:uid=\\${uid},ou=people view-value:uid=\\${uid},ou=\\${location},ou=people

Note: you might have to escape some characters (e.g $) in the command below depending on the command interpreter you are using. In the example above, I used \\$ instead of plain $.
Note2: dn patterns used in virtual transformations must not contain the data view viewBase (dc=company,dc=com in this case) as it is implicit.

Friday Nov 21, 2008

About rootDSE again

For monitoring purpose, a customer needs to get access to the rootDSE entry of each directory server behind DPS.

By default, DPS manages the rootDSE itself so that (in a virtual context) it reflects all the naming contexts and capabilities exposed by the proxy.
Here is one way to have access to the rootDSEs in question:
- the rootDSE DS content is exposed through a data view dedicated to ds monitoring. This dataview view base is choosen to not conflict with any suffix used by client applications, for instance "o=ds status". This data view is associated with a pool containing every directory server. Dn renaming is used to map o=ds status onto rootDSE ("").

Pros: it is possible to expose this data view to monitoring info only via a connection handler; monitoring traffic is not impact by regular client traffic (unless apps are aware of the special suffix!), so it is easy to get the status from N servers with N searches.
Cons: monitoring app must be changed to search for "o=ds status" instead of rootDSE and proxy configuration need to be changed.

Note: please ask Sun support for fix 6767776 if you choose the option above.

My name is Sylvain Duloutre, I worked as a Software Architect in the Oracle Directory Integration Team, the customer-facing part of Directory Services & Identity Management Product Development, working on Technical Field Enablement and Solutions Architecture.

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

A mirror of this blog is available on Wordpress here.


« June 2016