Note:This posting was originally written based on MacOS 10.4. See the Comments section for some updates since then.
One of my responsibilities on the Update Center 2.0 project is to perform builds of Python and wxPython for all of our supported platforms. One unique aspect of our environment is that we need these builds to be relocatable as far as the filesystem is concerned in order to support our multi-install requirement. That is: plop the build anywhere on your system and it must work.
A key to making relocatable software work is relative paths. All references to files within the relocatable install image must be relative (or at least start out relative and only made absolute dynamically at runtime). This includes any dependencies that binaries in the install image might have on dynamic libraries in the install image.
You can always use LD_LIBRARY_PATH (or DYLD_LIBRARY_PATH on the Mac) to force the runtime linker to locate the right dynamic libraries -- but that's cheating, and should only be used as a hack of last resort. Better to construct the binaries correctly so that they can locate their dependencies without the need to alter the user's environment.
I'm pretty familiar with dynamic libraries on Solaris and Linux. On those platforms you can embed an RPATH into a binary that is searched by the runtime linker to locate libraries. Plus both Solaris and Linux support the $ORIGIN token so you can make these paths relative to the install location of the binary.
On Solaris you can see these settings by using dump -Lv:
$ dump -Lv wx/_core_.so wx/_core_.so: \*\*\*\* DYNAMIC SECTION INFORMATION \*\*\*\* .dynamic: [INDEX] Tag Value  NEEDED libCrun.so.1  NEEDED libwx_GTK2u_richtext-2.8.so.0.4.0  NEEDED libwx_GTK2u_aui-2.8.so.0.4.0  NEEDED libwx_GTK2u_xrc-2.8.so.0.4.0  NEEDED libwx_GTK2u_qa-2.8.so.0.4.0  NEEDED libwx_GTK2u_html-2.8.so.0.4.0  NEEDED libwx_GTK2u_adv-2.8.so.0.4.0  NEEDED libwx_GTK2u_core-2.8.so.0.4.0  NEEDED libwx_baseu_xml-2.8.so.0.4.0  NEEDED libwx_baseu_net-2.8.so.0.4.0  NEEDED libwx_baseu-2.8.so.0.4.0  INIT 0xc8ef4  FINI 0xc9000  RUNPATH $ORIGIN/../wxWidgets/lib  RPATH $ORIGIN/../wxWidgets/lib  HASH 0x94
Here the _core_.so binary has dependencies on a number of wx libraries. RPATH is set to look for libraries relative to the install location of _core_.so, and therefore the linker can find these libraries without needing to set LD_LIBRARY_PATH. You specify the value for RPATH at link time using the -R option.
Linux is similar. In this case you use objdump -p to inspect the binaries:
$ objdump -p wx/_core_.so _core_.so: file format elf32-i386 . . . Dynamic Section: NEEDED libwx_gtk2u_richtext-2.8.so.0 NEEDED libwx_gtk2u_aui-2.8.so.0 NEEDED libwx_gtk2u_xrc-2.8.so.0 NEEDED libwx_gtk2u_qa-2.8.so.0 NEEDED libwx_gtk2u_html-2.8.so.0 NEEDED libwx_gtk2u_adv-2.8.so.0 NEEDED libwx_gtk2u_core-2.8.so.0 NEEDED libwx_baseu_xml-2.8.so.0 NEEDED libwx_baseu_net-2.8.so.0 NEEDED libwx_baseu-2.8.so.0 NEEDED libstdc++.so.6 NEEDED libm.so.6 NEEDED libgcc_s.so.1 NEEDED libpthread.so.0 NEEDED libc.so.6 RPATH $ORIGIN/../wxWidgets/lib INIT 0x1d384 . . .
But what about the Mac? That's new territory for me. Here is what I learned -- I welcome comments since I likely know just enough to be dangerous.
On the Mac a dynamic library (dylib) has an "install name". The install name is a path baked into the dynamic library that says where to find the library at runtime. When you link against the dylib this path is saved in your binary so that your binary can find the dylib at runtime. Seems a bit backwards to me -- but that's how it works.
You can see the install name of a dylib by using otool. For example:
$ otool -D libwx_macu-184.108.40.206.0.dylib libwx_macu-220.127.116.11.0.dylib: /Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib
You can also use otool to inspect binaries and list their dependencies:
$ otool -L wx/_core_.so wx/_core_.so: /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0) /Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib (compatibility version 5.0.0, current version 5.0.0) . . .
So wx/_core_.so depends on libwx_macu-2.8.0.dylib, which is fine. But notice that evil absolute path. That won't work in our "install anywhere" world. So how do we fix this? What's the Mac OS equivalent of $ORIGIN and RPATH?
Well, there isn't any. But there is something we can do instead. After the build is complete I use the install_name_tool utility to fix up the dylib install names and dependencies in our binaries. Fortunately the Mac also supports a magic token: @loader_path that can be used in a fashion similar to $ORIGIN on Solaris/Linux.
So when building wxPython on the Mac I first do a complete build. This results in the absolute paths being used as mentioned above. Then as part of my "make install" step I fix up these paths to be relative using the install_name_tool utility.
For example, this command changes the install name of libwx_macu-18.104.22.168.0.dylib to be relative to the location of the binary using it:
$ install_name_tool -id "@loader_path/../wxWidgets/lib/libwx_macu-22.214.171.124.0.dylib" libwx_macu-126.96.36.199.0.dylib
And this changes the dependency in a binary to use a relative path to locate the library (relative to the install location of the binary).
$ install_name_tool -change "/Users/dipol/wxPython/dist-darwin/wx-2.8/wxWidgets/lib/libwx_macu-2.8.0.dylib" "@loader_path/../wxWidgets/lib/libwx_macu-2.8.0.dylib" wx/_core_.so
This mechanism is still not as flexible as the Solaris/Linux approach, since binaries can't specify a search path. It's also a bit more difficult to automate since you must determine the current install name in order to replace it with the new install name. But it does work.
Note that for your project you may be able to simplify this. For example, if you build your dynamic libraries with the correct install names first, then your binaries will pick up the correct install names at link time and you shouldn't need to change the dependencies post-build.