From Code to Binary

Transcript

1. Mach-O Mach-O
   03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 1
   From code to binary
   

   View Slide

2. Wait… but why?
   • Apps are Mach-O binaries
   • Binary frameworks
   • Carthage/Cocoapods errors
   • Get out of problems when things go south
   03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 2
   Undefined symbols for architecture x86_64:
   "_thisWillTotallyBeThere", referenced from:
   _main in trust-me-c9e7ba.o
   ld: symbol(s) not found for architecture x86_64
   clang: error: linker command failed with exit code 1 (use -v to see
   invocation)
   dyld: Library not loaded:
   @rpath/Alamofire.framework/Alamofire
   Referenced from:
   /private/var/mobile/Containers/Bundle/Application/...
   Reason: image not found
   

   View Slide

3. What is Mach-O
   used for?
   03.6.19
   • Executables (/bin/ls)
   • Relocatable object files ( .o)
   • Static libraries (.a)
   • Dynamic Libraries (.dylib)
   • Desymbolication files (.dSYM)
   • Bundles (.bundles)
   • Core dumps (stack traces)
   Tom m aso Piazza - @ tm pz https://github.com /blender 3
   

   View Slide

4. 03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 4
   

   View Slide

5. What about
   .framework(s)?
   03.6.19
   • Directory trees with special structure
   • Pack additional resources
   • Info.plist
   • Asset bundles
   • Fonts
   • ….
   • Can be static or dynamic
   Tom m aso Piazza - @ tm pz https://github.com /blender 5
   

   View Slide

6. How are Mach-O
   files produced?
   03.6.19 Tom m aso Piazza - @ tm pz https://github.com /blender
   14
   

   View Slide

7. Compiling
   03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 7
   Mach-O
   Image: The Big Nerd Ranch
   

   View Slide

8. Compiling (2)
   Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 8
   • The compiler checks your code
   against .h files
   • Name, arguments, return type
   • References to external
   symbols (functions, variables,
   constant from libraries) left
   undefined
   • Mismatch?
   

   View Slide

9. Linking
   03.6.19 Tommaso Piazza - @tmpz https://github.com/blender 9
   Image: The Big Nerd Ranch
   

   View Slide

10. Static Linking
    Tom m aso Piazza - @ tm pz https://github.com /blender 10
    man ld
    • The binary fromthe linked library is copied into the final product
    • No binary share
    • Relatively simple process
    You may knowld for messages like...
    Undefined symbols for architecture x86_64:
    "_thisWillTotallyBeThere", referenced from:
    _main in trust-me-c9e7ba.o
    ld: symbol(s) not found for architecture x86_64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    03.6.19
    

    View Slide

11. Dynamic Linking
    Tom m aso Piazza - @ tm pz https://github.com /blender 11
    man dyld
    • The binary from the linked library is not copied into the final product
    • Binary share
    • Address of symbols resolved at load time
    • Runtime penalty, DYLD_PRINT_STATISTICS_DETAILS=1
    You may know dyld for messages like...
    dyld: Library not loaded: @rpath/Alamofire.framework/Alamofire
    Referenced from: /private/var/mobile/Containers/Bundle/Application/...
    Reason: image not found
    03.6.19
    

    View Slide

12. Demo
    03.6.19
    Tom m aso Piazza - @ tm pz https://github.com /blender 12
    

    View Slide

13. From Code to Binary
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 3
    

    View Slide

14. Compile & Link Recap
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 14
    swiftc -c Greeter.swift Colorizer.swift -module-name "Greet"
    swiftc -emit-module Greeter.swift Colorizer.swift -module-name "Greet"
    libtool -static Greeter.o Colorizer.o -o libGreeter.a
    swiftc -c main.swift -L`pwd` -I`pwd` -l`pwd`
    ld main.o -lGreeter -L`pwd` -lswiftCore \
    -L/usr/lib/swift \
    -o hello
    

    View Slide

15. Mach-O Format
    03.6.19
    Tom m aso Piazza - @ tm pz https://github.com /blender 15
    Single Architecture
    Multiple Architecture
    

    View Slide

16. Mach-O Header
    • file:///Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach-o/loader.h
    • How to identify a Mach-O file?
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19
    16
    

    View Slide

17. Mach-O Header
    • Mach Magic number
    • MH_MAGIC, MH_CIGAM, MH_MAGIC_64, MH_CIGAM_64
    • 0xfeedface , 0xcefaedfe, 0xfeedfacf , 0xcffaedfe
    • Indicator of Endianness and 32 of 64 bit Arch
    • CPU Type
    file:///Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/machine.h
    • FileType
    • MH_EXECUTE, MH_DYLIB, MH_DSYM, MH_OBJECT …
    • Read with otool –h (or objdump -macho -private-header)
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19
    17
    

    View Slide

18. Load Commands
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19
    Contains
    • Information about the Data part of the file
    • Segments LC_SEGMENT_64
    • Sections in each segment
    • Read with
    • otool -l
    Answers
    • Where is the symbol table? LC_SYMTAB,
    • What is the minimum version of the OS?
    LC_VERSION_MIN_IPHONEOS
    • Where is main? LC_MAIN
    • What libraries should be loaded?
    LC_LOAD_DYLIB
    • Where is the code signature?
    LC_CODE_SIGNATURE
    18
    

    View Slide

19. Symbols
    • LC_SYMTAB (own symbol table)
    • Exported symbols
    • Read with: dsymutil –symtab if you have dsymor nm –Am if not stripped
    • LC_DYSYMTAB (dynamic symbol table, AKA symbols from other files)
    • Can include debug symbol
    • strip –SxXNT
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19
    19
    

    View Slide

20. FAT header
    03.6.19
    • Multiple architectures packed in one
    file
    • x86_64
    • armv6s
    • armv7
    • otool –f
    • objdump -macho -universal-headers
    Tom m aso Piazza - @ tm pz https://github.com /blender
    file:///Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach-o/fat.h
    See yourself at:
    20
    

    View Slide

21. Slimming down FAT files
    • lipo -thin -output
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19
    21
    

    View Slide

22. Party Trick
    nm –a | grep –w “SO”
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 222
    

    View Slide

23. Thanks!
    03.6.19
    Tom m aso Piazza - @ tm pz https://github.com /blender 23
    @tmpz
    blender
    Carthage
    Rome
    Speakerdeck
    

    View Slide

24. References
    03.6.19
    • https://www.bignerdranch.com/blog/manual-swift-understanding-the-swift-objective-c-build-
    pipeline/
    • https://www.bignerdranch.com/blog/it-looks-like-you-are-trying-to-use-a-framework/
    • https://www.iecc.com/linker/
    • https://pewpewthespells.com/blog/static_and_dynamic_libraries.html
    • https://en.wikipedia.org/wiki/Object_file
    • https://en.wikipedia.org/wiki/Data_segment
    • https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLib
    raries/100-Articles/OverviewOfDynamicLibraries.html
    • https://developer.apple.com/library/archive/qa/qa1118/_index.html
    • http://nickdesaulniers.github.io/blog/2016/11/20/static-and-dynamic-libraries/
    • https://nickdesaulniers.github.io/blog/2016/08/13/object-files-and-symbols
    • https://www.darlinghq.org/developer-zone/mach-o-dynamic-loader/
    • https://yurylapitsky.com/exploring_mac-o_binaries_nm
    • http://web.cse.ohio-state.edu/~reeves.92/CSE2421au12/SlidesDay52.pdf
    • https://blog.timac.org/2016/1018-analysis-of-the-facebook-app-for-ios/
    • https://www.geeksforgeeks.org/memory-layout-of-c-program/
    • https://developer.apple.com/library/archive/technotes/tn2151/_index.html
    • https://lldb.llvm.org/symbols.html
    • https://en.wikipedia.org/wiki/Dynamic_linker
    • https://opensource.apple.com/source/dyld/dyld-635.2/
    Tom m aso Piazza - @ tm pz https://github.com /blender 24
    

    View Slide

25. References (2)
    • https://www.catswhocode.com/blog/how-to-create-a-pure-swift-module
    • http://iokit.racing/machotricks.pdf
    • https://en.wikipedia.org/wiki/Mach-O
    • https://lowlevelbits.org/parsing-mach-o-files/
    • http://www.m4b.io/reverse/engineering/mach/binaries/2015/03/29/mach-binaries.html
    • https://www.objc.io/issues/6-build-tools/mach-o-executables/
    • https://en.wikipedia.org/wiki/Fat_binary
    • https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/MachOTopics/1-
    Articles/building_files.html
    • http://timetobleed.com/dynamic-symbol-table-duel-elf-vs-mach-o-round-2/
    • http://www.idea2ic.com/File_Formats/MachORuntime.pdf
    • https://www.first.org/resources/papers/conf2016/FIRST-2016-130.pdf
    • http://bdunagan.com/2010/05/15/symbolification-shipping-symbols/
    • https://stackoverflow.com/questions/27669766/how-to-read-mach-o-header-from-object-file
    • https://www.apriorit.com/dev-blog/225-dynamic-linking-mach-o
    • https://blog.smartdec.net/reading-ios-app-binary-files-2c9e63a381ad?gi=a704d31da280
    • http://www.newosxbook.com/articles/DYLD.html
    • https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html
    • https://stackoverflow.com/questions/435352/limiting-visibility-of-symbols-when-linking-shared-
    libraries/452955#452955
    • https://stackoverflow.com/questions/22102470/link-a-static-library-to-a-shared-library-and-hide-exported-
    symbols
    • https://clang.llvm.org/docs/Modules.html#module-maps
    • http://timetobleed.com/tag/mach-o/
    • https://reverseengineering.stackexchange.com/questions/17697/macho-remove-a-load-command-from-ios-
    binary
    • https://samhuri.net/posts/2010/01/basics-of-the-mach-o-file-format
    • http://www.blackhat.com/presentations/bh-dc-09/Iozzo/BlackHat-DC-09-Iozzo-Macho-on-the-fly.pdf
    • https://github.com/JDevlieghere/LibEBC
    • http://www.cilinder.be/docs/next/NeXTStep/3.3/nd/DevTools/14_MachO/MachO.htmld/index.html
    • http://nicolascormier.com/documentation/security/Infecting_Mach-O_Files.pdf
    • https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/0
    00-Introduction/Introduction.html
    • https://jameshfisher.com/2017/08/22/inspecting-mach-o-files.html
    Tom m aso Piazza - @ tm pz https://github.com /blender 03.6.19 25
    

    View Slide