By darcy on Feb 01, 2010
Java developers are familiar with dynamic linking. Class files are a kind of intermediate format with symbolic references. At runtime, a class loader will
load, link, and initialize new types as needed.
Typically the full classpath a class loader uses for searching will have several logically distinct sub components, including the boot classpath, endorsed standards, extension directories, and the user-specified classpath.
The manifest of a jar file can also contain
Together, these paths delineate the boundaries of "jar hell."
For many years, modern Unix systems have also supported dynamic linking for C programs. Instead of a classpath, there is a runpath of locations to look to for resolving symbolic references. Like the classpath, the full runpath has multiple components, including a default component for system facilities (analogous to boot classpath), a component stored in a shared object (analogous to jar file
Class-Path entries), as well as an end-user specified component (analogous to the
-classpath command line option or
CLASSPATH environment variable).
The details of linking on Solaris are well explained in Sun's
Linker and Libraries Guide.
Other contemporary Unix platforms like Linux and MacOS have similar facilities, although the details of the various commands differ.
One of the tasks the JDK's
launcher has handled is setting a suitable runpath for the JVM and platform libraries. Historically a runpath was needed to link in the desired JVM, such as client or server, and other system libraries. The client JVM and the server JVM are separate shared objects which support the same set of interfaces; by interpreting the command line flags the launcher selects which JVM to link in. Operationally, the linking is initiated by the Unix
dlopen library call.
So that the caller of the
java command did not need to set
LD_LIBRARY_PATH, after selecting the JVM to run the launcher would modify the
LD_LIBRARY_PATH environment variable by prepending the path to the JVM shared object (and paths to other directories with JDK native system libraries). However, the runtime linker only reads the value of
LD_LIBRARY_PATH when a process starts. Therefore, to have the new value take effect, the launcher would call an
exec-family system call to start the process anew.
execing to set
LD_LIBRARY_PATH is not recommended practice on Unix systems.
execing to set
LD_LIBRARY_PATH had a number of unpleasant consequences in the launcher code. There is only a narrow path to pass information between the
exec parent and the
exec child, such as by modifying environment variables, which is generally discouraged. To decide whether or not an exec was needed, the launcher checked whether the prefix of
LD_LIBRARY_PATH had the expected value; if it did, no
exec was done for that purpose and infinite
exec loops were avoided. Presetting
LD_LIBRARY_PATH to the right value before calling
java could thus be used to suppress the
exec. There were also complications with correctly supporting multiple
LD_LIBRARY_PATH variables on Solaris1 and handling suid
java executions on Linux.2
The proper way to accommodate such dependencies is not to set
LD_LIBRARY_PATH but rather to use the runtime linker facilities analogous to jar file
Class-Path entries; the facility is the
$ORIGIN dynamic string token for the runtime linker. As the name implies,
$ORIGIN is expanded to the path directory of the object file in question; thus relative paths to other directories can be specified. Therefore, as long as the directory structure of the JDK and JRE are known,
$ORIGIN can be used to record any necessary dependencies.
For some time, the JDK build has actually used
$ORIGIN in creating its native libraries. Therefore, it may have been the case that
LD_LIBRARY_PATH was not actually needed. However, verifying that
LD_LIBRARY_PATH was not actually needed would require building an
exec-free JDK on all supported Unix platforms and running tests that exercise the all libraries in the directories no longer added to
LD_LIBRARY_PATH. The engineering for
Kumar's purge of
LD_LIBRARY_PATH was generally straightforward: deleting the the
LD_LIBRARY_PATH-related code in the Unix
java_md.c file and doing builds on all platforms. Most of the effort of getting this fix back involved running tests to verify everything still worked. The testing revealed an
unneeded, troublesome symlink that was removed at the same time
LD_LIBRARY_PATH usage was purged.
While the launcher no longer
execs to set the
LD_LIBRARY_PATH, there are still cases where an
exec will occur for other reasons. If the
java command is requested to change data models using the
-d64 flag, that is, a 32-bit
java command is asked to run a 64-bit JVM or vice versa, an
exec is needed to effect the change. Also, multiple JRE support, where a different version is requested via the
-version:foo flag, will also cause an exec if a different Java version needs to be run.
However, before Kumar's fix the common case was that the launcher would
exec once; now the common case is that the launcher will exec zero times.3
I'm very happy this messy use of
LD_LIBRARY_PATH has finally been removed in JDK 7. The removal makes the launcher code both simpler and more maintainable. Unless your use of
java relies on the number of
execs that occur, the change should be largely transparent, other than startup being marginally faster.
One situation to be aware of is launching a
LD_LIBRARY_PATH-free JDK 7
java command from a JDK 6 or earlier
java process. If the
LD_LIBRARY_PATH variable of the older JDK is not cleared, it can affect the liking of the JDK 7 process.
1 Since Solaris 7, that OS line has supported three
LD_LIBRARY_PATH_32: if set, overrides
LD_LIBRARY_PATHfor 32-bit processes.
LD_LIBRARY_PATH_64: if set, overrides
LD_LIBRARY_PATHfor 64-bit processes.
LD_LIBRARY_PATH: used by both 32-bit and 64-bit processes is not overridden by a data model specific variable.
On Solaris, back in JDK 1.4.2 I fixed the launcher to properly take into account all three variables (4731671); on re-
execthe data model specific environment variable is unset and
LD_LIBRARY_PATHcontains the old data model specific value prepended with the JDK system paths. Tests to verify all this used to live in and around
test/tools/launcher/SolarisDataModel.sh, but they have thankfully been deleted as they are no longer relevant.
2 For suid or sgid binaries,
LD_LIBRARY_PATHis handled differently to avoid security problems. While the Solaris runtime linker applies more scrutiny to
LD_LIBRARY_PATHin this case, on Linux
LD_LIBRARY_PATHto the empty string. Since the empty string will not contain the expected JDK system directories, the prefix-checking logic detected this case to avoid an infinite
execloop (4745674). Running
javasuid or sgid isn't necessarily recommended, but it is possible. To actually resolve linking dependencies for such binaries, OS-specific configuration may be needed to add JDK directories to the set of trusted paths.
3 Before my batch of launcher fixes in JDK 1.4.2, the number of
execs was even more varied. Specifying a different data model would
exectwice, once to change the data model and again to set the
LD_LIBRARY_PATHfor that data model. From JDK 1.4.2 until the purge of
LD_LIBRARY_PATH, the launcher used a single
execto set the
LD_LIBRARY_PATHto the target data model (4492822).