dyld: The OS X Dynamic Loader

A breakdown of how the OS X Dynamic Loader works. This is a presentation that I gave internally at Continuum. Sadly Speaker Deck doesn't support gifs in the presentation so you'll just have to use your imagination.

Transcript

1. The OS X Dynamic Loader Aaron Meurer
2. Dynamic linking • Libraries can be linked statically or dynamically • Statically means the library code is copied in to your binary when it is compiled • Dynamically means the dynamic loader (dyld) loads it when your binary runs
3. Question • How does dyld find the dynamic libraries to load?
4. Answer (on OS X) install names
5. Every* dynamic library on OS X has an install name • Can be inspected with otool -D /path/to/ library.dylib $ otool -D /usr/lib/libSystem.B.dylib /usr/lib/libSystem.B.dylib: /usr/lib/libSystem.B.dylib • Also the first line of otool -L /path/to/ library.dylib • Typically the install name of a library is the full path to the library.
6. • When your binary is linked to a dependent library, its install name is copied into the binary. • Can see all the dependent install names with otool -L /path/to/library.dylib $ otool -L ~/anaconda/lib/libpython3.4m.dylib /Users/aaronmeurer/anaconda/lib/libpython3.4m.dylib: libpython3.4m.dylib (compatibility version 3.4.0, current version 3.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.0.0) /System/Library/Frameworks/CoreFoundation.framework/ Versions/A/CoreFoundation (compatibility version 150.0.0, current version 476.0.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
7. • The install name of the library and of all the dependent libraries can be changed with install_name_tool $ install_name_tool -id fishy libpython3.4m.dylib $ install_name_tool -change /usr/lib/libSystem.B.dylib nope libpython3.4m.dylib $ otool -L libpython3.4m.dylib libpython3.4m.dylib: fishy (compatibility version 3.4.0, current version 3.4.0) nope (compatibility version 1.0.0, current version 111.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/ CoreFoundation (compatibility version 150.0.0, current version 476.0.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) • Can also be set at compile time with some compiler flags
8. Magic • Install names can include some magic entries • @executable_path: The directory path of the loading executable. • @loader_path: The directory path of the loading library.
9. @loader_path • (we don’t really care about @executalbe_path) • Example: libzmq.3.dylib depends on libsodium.4.dylib (both will be in $PREFIX/lib) $otool -L ~/anaconda/lib/libzmq.dylib /Users/aaronmeurer/anaconda/lib/libzmq.3.dylib: libzmq.3.dylib (compatibility version 5.0.0, current version 5.0.0) @loader_path/./libsodium.4.dylib (compatibility version 10.0.0, current version 10.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 60.0.0) • No matter where libzmq.3.dylib is on the filesystem, it always loads the libsodium.4.dylib that sits right next to it.
10. This is how conda packages are relocatable on OS X • conda build calls install_name_tool automatically at the end of the build to convert absolute install names into relative install names using @loader_path
11. One More Magic • @rpath: @rpath is replaced with paths from LC_RPATH from each library in the dependency chain until a suitable replacement is found. • Each library can have zero or more rpaths in its LC_RPATH. • Rpaths can use @loader_path • Rpaths can be added when compiling with - Wl,-rpath,path or using install_name_tool
12. Environment Variables • The whole process can be steamrolled by setting the DYLD_LIBRARY_PATH environment variable. • Or DYLD_FALLBACK_LIBRARY_PATH, which is only used if everything else fails.