From Code to Binary

Transcript

1. Mach-O Mach-O 03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 1 From code to binary
2. Wait… but why? • Apps are Mach-O binaries • Binary frameworks • Carthage/Cocoapods errors • Get out of problems when things go south 03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 2 Undefined symbols for architecture x86_64: "_thisWillTotallyBeThere", referenced from: _main in trust-me-c9e7ba.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) dyld: Library not loaded: @rpath/Alamofire.framework/Alamofire Referenced from: /private/var/mobile/Containers/Bundle/Application/... Reason: image not found
3. What is Mach-O used for? 03.6.19 • Executables (/bin/ls) • Relocatable object files ( .o) • Static libraries (.a) • Dynamic Libraries (.dylib) • Desymbolication files (.dSYM) • Bundles (.bundles) • Core dumps (stack traces) Tom m aso Piazza - @ tm pz https://github.com /blender 3
4. 03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 4
5. What about .framework(s)? 03.6.19 • Directory trees with special structure • Pack additional resources • Info.plist • Asset bundles • Fonts • …. • Can be static or dynamic Tom m aso Piazza - @ tm pz https://github.com /blender 5
6. How are Mach-O files produced? 03.6.19 Tom m aso Piazza - @ tm pz https://github.com /blender 14
7. Compiling 03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 7 Mach-O Image: The Big Nerd Ranch
8. Compiling (2) Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 8 • The compiler checks your code against .h files • Name, arguments, return type • References to external symbols (functions, variables, constant from libraries) left undefined • Mismatch?
9. Linking 03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 9 Image: The Big Nerd Ranch
10. Static Linking Tom m aso Piazza - @ tm pz https://github.com /blender 10 man ld • The binary fromthe linked library is copied into the final product • No binary share • Relatively simple process You may knowld for messages like... Undefined symbols for architecture x86_64: "_thisWillTotallyBeThere", referenced from: _main in trust-me-c9e7ba.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) 03.6.19
11. Dynamic Linking Tom m aso Piazza - @ tm pz https://github.com /blender 11 man dyld • The binary from the linked library is not copied into the final product • Binary share • Address of symbols resolved at load time • Runtime penalty, DYLD_PRINT_STATISTICS_DETAILS=1 You may know dyld for messages like... dyld: Library not loaded: @rpath/Alamofire.framework/Alamofire Referenced from: /private/var/mobile/Containers/Bundle/Application/... Reason: image not found 03.6.19
12. Demo 03.6.19 Tom m aso Piazza - @ tm pz https://github.com /blender 12
13. From Code to Binary Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 3
14. Compile & Link Recap Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 14 swiftc -c Greeter.swift Colorizer.swift -module-name "Greet" swiftc -emit-module Greeter.swift Colorizer.swift -module-name "Greet" libtool -static Greeter.o Colorizer.o -o libGreeter.a swiftc -c main.swift -L`pwd` -I`pwd` -l`pwd` ld main.o -lGreeter -L`pwd` -lswiftCore \ -L/usr/lib/swift \ -o hello
15. Mach-O Format 03.6.19 Tom m aso Piazza - @ tm pz https://github.com /blender 15 Single Architecture Multiple Architecture
16. Mach-O Header • file:///Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach-o/loader.h • How to identify a Mach-O file? Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 16
17. Mach-O Header • Mach Magic number • MH_MAGIC, MH_CIGAM, MH_MAGIC_64, MH_CIGAM_64 • 0xfeedface , 0xcefaedfe, 0xfeedfacf , 0xcffaedfe • Indicator of Endianness and 32 of 64 bit Arch • CPU Type file:///Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/machine.h • FileType • MH_EXECUTE, MH_DYLIB, MH_DSYM, MH_OBJECT … • Read with otool –h (or objdump -macho -private-header) Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 17
18. Load Commands Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 Contains • Information about the Data part of the file • Segments LC_SEGMENT_64 • Sections in each segment • Read with • otool -l Answers • Where is the symbol table? LC_SYMTAB, • What is the minimum version of the OS? LC_VERSION_MIN_IPHONEOS • Where is main? LC_MAIN • What libraries should be loaded? LC_LOAD_DYLIB • Where is the code signature? LC_CODE_SIGNATURE 18
19. Symbols • LC_SYMTAB (own symbol table) • Exported symbols • Read with: dsymutil –symtab <file> if you have dsymor nm –Am <file> if not stripped • LC_DYSYMTAB (dynamic symbol table, AKA symbols from other files) • Can include debug symbol • strip –SxXNT <file> Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 19
20. FAT header 03.6.19 • Multiple architectures packed in one file • x86_64 • armv6s • armv7 • otool –f <file> • objdump -macho -universal-headers Tom m aso Piazza - @ tm pz https://github.com /blender file:///Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach-o/fat.h See yourself at: 20
21. Slimming down FAT files • lipo -thin <arch_to_keep> <file_to_slim_down> -output <output> Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 21
22. Party Trick nm –a <file> | grep –w “SO” Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 222
23. Thanks! 03.6.19 Tom m aso Piazza - @ tm pz https://github.com /blender 23 @tmpz blender Carthage Rome Speakerdeck
24. References 03.6.19 • https://www.bignerdranch.com/blog/manual-swift-understanding-the-swift-objective-c-build- pipeline/ • https://www.bignerdranch.com/blog/it-looks-like-you-are-trying-to-use-a-framework/ • https://www.iecc.com/linker/ • https://pewpewthespells.com/blog/static_and_dynamic_libraries.html • https://en.wikipedia.org/wiki/Object_file • https://en.wikipedia.org/wiki/Data_segment • https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLib raries/100-Articles/OverviewOfDynamicLibraries.html • https://developer.apple.com/library/archive/qa/qa1118/_index.html • http://nickdesaulniers.github.io/blog/2016/11/20/static-and-dynamic-libraries/ • https://nickdesaulniers.github.io/blog/2016/08/13/object-files-and-symbols • https://www.darlinghq.org/developer-zone/mach-o-dynamic-loader/ • https://yurylapitsky.com/exploring_mac-o_binaries_nm • http://web.cse.ohio-state.edu/~reeves.92/CSE2421au12/SlidesDay52.pdf • https://blog.timac.org/2016/1018-analysis-of-the-facebook-app-for-ios/ • https://www.geeksforgeeks.org/memory-layout-of-c-program/ • https://developer.apple.com/library/archive/technotes/tn2151/_index.html • https://lldb.llvm.org/symbols.html • https://en.wikipedia.org/wiki/Dynamic_linker • https://opensource.apple.com/source/dyld/dyld-635.2/ Tom m aso Piazza - @ tm pz https://github.com /blender 24
25. References (2) • https://www.catswhocode.com/blog/how-to-create-a-pure-swift-module • http://iokit.racing/machotricks.pdf • https://en.wikipedia.org/wiki/Mach-O • https://lowlevelbits.org/parsing-mach-o-files/ • http://www.m4b.io/reverse/engineering/mach/binaries/2015/03/29/mach-binaries.html • https://www.objc.io/issues/6-build-tools/mach-o-executables/ • https://en.wikipedia.org/wiki/Fat_binary • https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/MachOTopics/1- Articles/building_files.html • http://timetobleed.com/dynamic-symbol-table-duel-elf-vs-mach-o-round-2/ • http://www.idea2ic.com/File_Formats/MachORuntime.pdf • https://www.first.org/resources/papers/conf2016/FIRST-2016-130.pdf • http://bdunagan.com/2010/05/15/symbolification-shipping-symbols/ • https://stackoverflow.com/questions/27669766/how-to-read-mach-o-header-from-object-file • https://www.apriorit.com/dev-blog/225-dynamic-linking-mach-o • https://blog.smartdec.net/reading-ios-app-binary-files-2c9e63a381ad?gi=a704d31da280 • http://www.newosxbook.com/articles/DYLD.html • https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html • https://stackoverflow.com/questions/435352/limiting-visibility-of-symbols-when-linking-shared- libraries/452955#452955 • https://stackoverflow.com/questions/22102470/link-a-static-library-to-a-shared-library-and-hide-exported- symbols • https://clang.llvm.org/docs/Modules.html#module-maps • http://timetobleed.com/tag/mach-o/ • https://reverseengineering.stackexchange.com/questions/17697/macho-remove-a-load-command-from-ios- binary • https://samhuri.net/posts/2010/01/basics-of-the-mach-o-file-format • http://www.blackhat.com/presentations/bh-dc-09/Iozzo/BlackHat-DC-09-Iozzo-Macho-on-the-fly.pdf • https://github.com/JDevlieghere/LibEBC • http://www.cilinder.be/docs/next/NeXTStep/3.3/nd/DevTools/14_MachO/MachO.htmld/index.html • http://nicolascormier.com/documentation/security/Infecting_Mach-O_Files.pdf • https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/0 00-Introduction/Introduction.html • https://jameshfisher.com/2017/08/22/inspecting-mach-o-files.html Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 25