VMM'17: An Analysis of Inline Assembly in C Projects

389c8e3d83119ec458c5c57e8d92da2a?s=47 Manuel Rigger
September 29, 2017

VMM'17: An Analysis of Inline Assembly in C Projects

Talk at Virtual Machine Meetup 2017 (http://vmmeetup.github.io/2017/)

389c8e3d83119ec458c5c57e8d92da2a?s=128

Manuel Rigger

September 29, 2017
Tweet

Transcript

  1. An Analysis of Inline Assembly in C Projects Manuel Rigger,

    Stefan Marr, Hanspeter Mössenböck VMM, September 29, 2017 Johannes Kepler University Linz
  2. Inline Assembly in C Projects 2 uint64_t clock_cycles() { unsigned

    int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; }
  3. Inline Assembly in C Projects 3 uint64_t clock_cycles() { unsigned

    int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Instructions
  4. Inline Assembly in C Projects 4 uint64_t clock_cycles() { unsigned

    int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Output operands
  5. Inline Assembly in C Projects 5 uint64_t clock_cycles() { unsigned

    int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Output operand constraints
  6. Inline Assembly in C Projects 6 uint64_t clock_cycles() { unsigned

    int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Input operands, side effects ,…
  7. uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh

    << 32)|tickl; } Inline Assembly in C Projects 7 clock_cycles(): rdtsc shl rdx, 32 mov eax, eax or rax, rdx ret
  8. uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh

    << 32)|tickl; } Inline Assembly in C Projects 7 What about C tools that cannot use an assembler to defer the work? clock_cycles(): rdtsc shl rdx, 32 mov eax, eax or rax, rdx ret
  9. Sulong 8

  10. Sulong 8 Ad-hoc approach to adding unsupported instructions

  11. Sulong 9 public abstract static class LLVMAMD64RdtscReadNode extends LLVMExpressionNode {

    public long executeRdtsc() { return System.currentTimeMillis(); } }
  12. Sulong 9 public abstract static class LLVMAMD64RdtscReadNode extends LLVMExpressionNode {

    public long executeRdtsc() { return System.currentTimeMillis(); } } Emulate the behavior of assembly
  13. Splint 10 Splint 3.1.2 --- 03 May 2009 test.c: (in

    function rdtsc) test.c:5:3: Unrecognized identifier: asm Identifier used in code has not been declared. (Use –unrecog to inhibit warning) test.c:5:15: Parse Error. (For help on parse errors, see splint -help parseerrors.) *** Cannot continue.
  14. Splint 10 Splint 3.1.2 --- 03 May 2009 test.c: (in

    function rdtsc) test.c:5:3: Unrecognized identifier: asm Identifier used in code has not been declared. (Use –unrecog to inhibit warning) test.c:5:15: Parse Error. (For help on parse errors, see splint -help parseerrors.) *** Cannot continue. Many analysis tools ignore inline assembly
  15. c2go 11 c2go transpile test.c panic: unknown node type: 'GCCAsmStmt

    0x3a991f8 <line:5:3, col:38>'goroutine 1 [running]:github_com_elliotchance_c2go_ast.Parse go/src/github.com/elliotchance/c2go/ast/ast.go:211main.convertLinesToNodes go/src/github.com/elliotchance/c2go/main.go:81main.Start go/src/github.com/elliotchance/c2go/main.go:219main.runCommand go/src/github.com/elliotchance/c2go/main.go:350main.main go/src/github.com/elliotchance/c2go/main.go:277goroutine 6 [finalizer wait]:
  16. c2go 11 Many source-to-source translators ignore inline assembly c2go transpile

    test.c panic: unknown node type: 'GCCAsmStmt 0x3a991f8 <line:5:3, col:38>'goroutine 1 [running]:github_com_elliotchance_c2go_ast.Parse go/src/github.com/elliotchance/c2go/ast/ast.go:211main.convertLinesToNodes go/src/github.com/elliotchance/c2go/main.go:81main.Start go/src/github.com/elliotchance/c2go/main.go:219main.runCommand go/src/github.com/elliotchance/c2go/main.go:350main.main go/src/github.com/elliotchance/c2go/main.go:277goroutine 6 [finalizer wait]:
  17. Current assumptions 12 “Inline assembly is rare in most programs”

    (Johnson 2014) “[…] programmers could provide C implementations of inline assembly blocks” (Johnson 2014)
  18. Current assumptions 13 “…” [Other papers]

  19. Open questions • How frequent is inline assembly? • How

    “complex” is the usage of inline assembly? • Which domains use inline assembly? • How diverse is the usage of inline assembly? 14
  20. Open questions • How frequent is inline assembly? • How

    “complex” is the usage of inline assembly? • Which domains use inline assembly? • How diverse is the usage of inline assembly? 14 Survey of inline assembly in C projects
  21. Approach • grep-based analysis of Github projects (excluding OS-level projects)

    • Most popular* 327 C projects • 937 C projects by keywords • Quantitative analysis  database for AMD64 instructions • Qualitative analysis 15 *C projects >= 850 Github stars
  22. How frequent is inline assembly? • ∼28% of the most

    popular C projects • ∼11% of the other C projects 16
  23. How frequent is inline assembly? • ∼28% of the most

    popular C projects • ∼11% of the other C projects 16 Sulong and other tools that process C cannot ignore inline assembly!
  24. How “complex” is its usage? 17 36 projects (18%) with

    inline assembly used complicated macro-metaprogramming  ignored
  25. How “complex” is its usage? 18 The majority of inline

    assembly fragments contain one or two instructions
  26. How “complex” is its usage? 18 The majority of inline

    assembly fragments contain one or two instructions 1 instruction
  27. How “complex” is its usage? 18 The majority of inline

    assembly fragments contain one or two instructions 2 instructions
  28. How “complex” is its usage? 19 The majority of projects

    with inline assembly contain only a few fragments
  29. How “complex” is its usage? 19 The majority of projects

    with inline assembly contain only a few fragments 10 fragments
  30. How “complex” is its usage? 20 rep nop

  31. How “complex” is its usage? 20 rep nop 0xF3 0x90

    pause
  32. How “complex” is its usage? 20 rep nop 0xF3 0x90

    pause Programmers sometimes have to work around old assemblers
  33. How “complex” is its usage? 21 .byte 0x66; clflush %0

  34. How “complex” is its usage? 21 .byte 0x66; clflush %0

    0x0FAE clflushopt
  35. Which domains use inline assembly? Domain # projects % projects

    Crypto 32 11.7% Networking 20 10.2% Media 17 8.6% Database 16 8.1% Language implementation 15 7.6% Misc 14 6.6% Concurrency 9 4.6% SSL 8 4.1% Text processing 8 4.1% Math library 7 3.6% Web server 7 3.6% 22 The domains of inline assembly are diverse
  36. How diverse is the usage of inline assembly? • ∼190

    unique inline assembly fragments • ∼170 unique instructions • Implement 46 instructions  support 76% of projects 23
  37. How diverse is the usage of inline assembly? • ∼190

    unique inline assembly fragments • ∼170 unique instructions • Implement 46 instructions  support 76% of projects 23 Only a small subset of the ∼1000 AMD64 instructions (Heule 2016) is needed
  38. How diverse is the usage of inline assembly? Instructions Contained

    in % projects (with inline assembly) rdtsc 26.9% cpuid 24.9% mov 23.9% <compiler barrier> 21.3% lock xchg 14.2% … … 24
  39. How diverse is the usage of inline assembly? • Instruction

    ordering and SMP programming 25 Compiler barriers Memory barriers Atomics …
  40. How diverse is the usage of inline assembly? • Instruction

    ordering and SMP programming • Performance optimizations 26 SIMD Endianness conversions Bitscans …
  41. How diverse is the usage of inline assembly? • Instruction

    ordering and SMP programming • Performance optimizations • Functionality unavailable in C 27 Elapsed clock cycles CPU features Data prefetching …
  42. How diverse is the usage of inline assembly? • Instruction

    ordering and SMP programming • Performance optimizations • Functionality unavailable in C • “Supporting” instructions (moves etc.) 28
  43. Selected Threat to Validity 29 #ifdef SOME_CONDITION // use inline

    assembly implementations #else // use other implementation #endif Instructions are not necessarily included in the binary
  44. Conclusions • We cannot ignore inline assembly • Majority of

    projects use few and short inline assembly fragments • <200 different instructions • But: • Projects that provide different SIMD implementations • Non-mnemonic instructions 30
  45. Thanks for listening! 31 https://github.com/graalvm/sulong/ @RiggerManuel

  46. Bibliography • Stefan Heule, Eric Schkufza, Rahul Sharma, and Alex

    Aiken. 2016. Stratified synthesis: automatically learning the x86-64 instruction set. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). • Rob Johnson and David Wagner. 2004. Finding user/kernel pointer bugs with type inference. In Proceedings of the 13th conference on USENIX Security Symposium - Volume 13 (SSYM'04), Vol. 13. USENIX Association, Berkeley, CA, USA, 9-9. 32
  47. Images • Page 13: Faras Saint Anne, Public Domain, https://en.wikipedia.org/wiki/File:Faras_Saint_Anne_(detail).jpg

    • Page 15: Cessna 172 Airplane: Nick Dean, Frze, Creative Commons Attribution-Share Alike 3.0 Unported license, https://commons.wikimedia.org/wiki/File:WRM_Airplane_-_Flugzeug_Cessna_172_m.jpg • Page 15: Under construction, Public Domain, https://commons.wikimedia.org/wiki/Category:Under_construction_icons#/media/File:UnderCon_icon_b lack.svg • Page 18: Stein der fünften Sonne, sog. Aztekenkalender: Anagoria, GNU Free Documentation License, https://commons.wikimedia.org/wiki/File:1479_Stein_der_f%C3%BCnften_Sonne,_sog._Aztekenkalend er,_Ollin_Tonatiuh_anagoria.JPG 33