Dynamic Libraries, RPATH, and Mac OS
By jdipol on Aug 15, 2008
One of my responsibilities on the Update Center 2.0 project is to perform builds of Python and wxPython for all of our supported platforms. One unique aspect of our environment is that we need these builds to be relocatable as far as the filesystem is concerned in order to support our multi-install requirement. That is: plop the build anywhere on your system and it must work.
A key to making relocatable software work is relative paths. All references to files within the relocatable install image must be relative (or at least start out relative and only made absolute dynamically at runtime). This includes any dependencies that binaries in the install image might have on dynamic libraries in the install image.
You can always use
the Mac) to force the runtime linker to locate the right
dynamic libraries -- but that's cheating, and should only be used as
a hack of last resort. Better to construct the binaries correctly
so that they can locate their dependencies without the need to
alter the user's environment.
I'm pretty familiar with dynamic libraries on Solaris and Linux.
On those platforms you can embed an
RPATH into a binary that is
searched by the runtime linker to locate libraries. Plus both Solaris
and Linux support the
$ORIGIN token so you can make these paths
relative to the install location of the binary.
On Solaris you can see these settings by using
$ dump -Lv wx/_core_.so wx/_core_.so: \*\*\*\* DYNAMIC SECTION INFORMATION \*\*\*\* .dynamic: [INDEX] Tag Value  NEEDED libCrun.so.1  NEEDED libwx_GTK2u_richtext-2.8.so.0.4.0  NEEDED libwx_GTK2u_aui-2.8.so.0.4.0  NEEDED libwx_GTK2u_xrc-2.8.so.0.4.0  NEEDED libwx_GTK2u_qa-2.8.so.0.4.0  NEEDED libwx_GTK2u_html-2.8.so.0.4.0  NEEDED libwx_GTK2u_adv-2.8.so.0.4.0  NEEDED libwx_GTK2u_core-2.8.so.0.4.0  NEEDED libwx_baseu_xml-2.8.so.0.4.0  NEEDED libwx_baseu_net-2.8.so.0.4.0  NEEDED libwx_baseu-2.8.so.0.4.0  INIT 0xc8ef4  FINI 0xc9000  RUNPATH $ORIGIN/../wxWidgets/lib  RPATH $ORIGIN/../wxWidgets/lib  HASH 0x94
_core_.so binary has dependencies on a number
of wx libraries.
RPATH is set to look for libraries relative to the
install location of
_core_.so, and therefore the linker can
find these libraries without needing to set
specify the value for
RPATH at link time using the
Linux is similar. In this case you use
objdump -p to inspect
$ objdump -p wx/_core_.so _core_.so: file format elf32-i386 . . . Dynamic Section: NEEDED libwx_gtk2u_richtext-2.8.so.0 NEEDED libwx_gtk2u_aui-2.8.so.0 NEEDED libwx_gtk2u_xrc-2.8.so.0 NEEDED libwx_gtk2u_qa-2.8.so.0 NEEDED libwx_gtk2u_html-2.8.so.0 NEEDED libwx_gtk2u_adv-2.8.so.0 NEEDED libwx_gtk2u_core-2.8.so.0 NEEDED libwx_baseu_xml-2.8.so.0 NEEDED libwx_baseu_net-2.8.so.0 NEEDED libwx_baseu-2.8.so.0 NEEDED libstdc++.so.6 NEEDED libm.so.6 NEEDED libgcc_s.so.1 NEEDED libpthread.so.0 NEEDED libc.so.6 RPATH $ORIGIN/../wxWidgets/lib INIT 0x1d384 . . .
But what about the Mac? That's new territory for me. Here is what I learned -- I welcome comments since I likely know just enough to be dangerous.
On the Mac a dynamic library (dylib) has an "install name". The install name is a path baked into the dynamic library that says where to find the library at runtime. When you link against the dylib this path is saved in your binary so that your binary can find the dylib at runtime. Seems a bit backwards to me -- but that's how it works.
You can see the install name of a dylib by using
otool. For example:
$ otool -D libwx_macu-126.96.36.199.0.dylib libwx_macu-188.8.131.52.0.dylib: /Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib
You can also use otool to inspect binaries and list their dependencies:
$ otool -L wx/_core_.so wx/_core_.so: /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0) /Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib (compatibility version 5.0.0, current version 5.0.0) . . .
wx/_core_.so depends on
which is fine. But notice that evil absolute path. That won't work in our
"install anywhere" world. So how do we fix this? What's the Mac OS equivalent
Well, there isn't any. But there is something we can do instead. After
the build is complete I use the
install_name_tool utility to
fix up the dylib install names and dependencies in our binaries. Fortunately
the Mac also supports a magic token:
@loader_path that can be used
in a fashion similar to
$ORIGIN on Solaris/Linux.
So when building wxPython on the Mac I first do a complete build. This results
in the absolute paths being used as mentioned above. Then as part of
my "make install" step I fix up these paths to be relative using the
For example, this command changes the install name of
to be relative to the location of the binary using it:
$ install_name_tool -id "@loader_path/../wxWidgets/lib/libwx_macu-184.108.40.206.0.dylib" libwx_macu-220.127.116.11.0.dylib
And this changes the dependency in a binary to use a relative path to locate the library (relative to the install location of the binary).
$ install_name_tool -change "/Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib" "@loader_path/../wxWidgets/lib/libwx_macu-2.8.0.dylib" wx/_core_.so
This mechanism is still not as flexible as the Solaris/Linux approach, since binaries can't specify a search path. It's also a bit more difficult to automate since you must determine the current install name in order to replace it with the new install name. But it does work.
Note that for your project you may be able to simplify this. For example, if you build your dynamic libraries with the correct install names first, then your binaries will pick up the correct install names at link time and you shouldn't need to change the dependencies post-build.