VMM'17: An Analysis of Inline Assembly in C Projects

Talk at Virtual Machine Meetup 2017 (http://vmmeetup.github.io/2017/)

Transcript

1. An Analysis of Inline Assembly in C Projects Manuel Rigger, Stefan Marr, Hanspeter Mössenböck VMM, September 29, 2017 Johannes Kepler University Linz
2. Inline Assembly in C Projects 2 uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; }
3. Inline Assembly in C Projects 3 uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Instructions
4. Inline Assembly in C Projects 4 uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Output operands
5. Inline Assembly in C Projects 5 uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Output operand constraints
6. Inline Assembly in C Projects 6 uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Input operands, side effects ,…
7. uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Inline Assembly in C Projects 7 clock_cycles(): rdtsc shl rdx, 32 mov eax, eax or rax, rdx ret
8. uint64_t clock_cycles() { unsigned int tickl, tickh; asm("rdtsc":"=a"(tickl),"=d"(tickh)); return ((uint64_t)tickh << 32)|tickl; } Inline Assembly in C Projects 7 What about C tools that cannot use an assembler to defer the work? clock_cycles(): rdtsc shl rdx, 32 mov eax, eax or rax, rdx ret
9. Sulong 8
10. Sulong 8 Ad-hoc approach to adding unsupported instructions
11. Sulong 9 public abstract static class LLVMAMD64RdtscReadNode extends LLVMExpressionNode { public long executeRdtsc() { return System.currentTimeMillis(); } }
12. Sulong 9 public abstract static class LLVMAMD64RdtscReadNode extends LLVMExpressionNode { public long executeRdtsc() { return System.currentTimeMillis(); } } Emulate the behavior of assembly
13. Splint 10 Splint 3.1.2 --- 03 May 2009 test.c: (in function rdtsc) test.c:5:3: Unrecognized identifier: asm Identifier used in code has not been declared. (Use –unrecog to inhibit warning) test.c:5:15: Parse Error. (For help on parse errors, see splint -help parseerrors.) *** Cannot continue.
14. Splint 10 Splint 3.1.2 --- 03 May 2009 test.c: (in function rdtsc) test.c:5:3: Unrecognized identifier: asm Identifier used in code has not been declared. (Use –unrecog to inhibit warning) test.c:5:15: Parse Error. (For help on parse errors, see splint -help parseerrors.) *** Cannot continue. Many analysis tools ignore inline assembly
15. c2go 11 c2go transpile test.c panic: unknown node type: 'GCCAsmStmt 0x3a991f8 <line:5:3, col:38>'goroutine 1 [running]:github_com_elliotchance_c2go_ast.Parse go/src/github.com/elliotchance/c2go/ast/ast.go:211main.convertLinesToNodes go/src/github.com/elliotchance/c2go/main.go:81main.Start go/src/github.com/elliotchance/c2go/main.go:219main.runCommand go/src/github.com/elliotchance/c2go/main.go:350main.main go/src/github.com/elliotchance/c2go/main.go:277goroutine 6 [finalizer wait]:
16. c2go 11 Many source-to-source translators ignore inline assembly c2go transpile test.c panic: unknown node type: 'GCCAsmStmt 0x3a991f8 <line:5:3, col:38>'goroutine 1 [running]:github_com_elliotchance_c2go_ast.Parse go/src/github.com/elliotchance/c2go/ast/ast.go:211main.convertLinesToNodes go/src/github.com/elliotchance/c2go/main.go:81main.Start go/src/github.com/elliotchance/c2go/main.go:219main.runCommand go/src/github.com/elliotchance/c2go/main.go:350main.main go/src/github.com/elliotchance/c2go/main.go:277goroutine 6 [finalizer wait]:
17. Current assumptions 12 “Inline assembly is rare in most programs” (Johnson 2014) “[…] programmers could provide C implementations of inline assembly blocks” (Johnson 2014)
18. Current assumptions 13 “…” [Other papers]
19. Open questions • How frequent is inline assembly? • How “complex” is the usage of inline assembly? • Which domains use inline assembly? • How diverse is the usage of inline assembly? 14
20. Open questions • How frequent is inline assembly? • How “complex” is the usage of inline assembly? • Which domains use inline assembly? • How diverse is the usage of inline assembly? 14 Survey of inline assembly in C projects
21. Approach • grep-based analysis of Github projects (excluding OS-level projects) • Most popular* 327 C projects • 937 C projects by keywords • Quantitative analysis  database for AMD64 instructions • Qualitative analysis 15 *C projects >= 850 Github stars
22. How frequent is inline assembly? • ∼28% of the most popular C projects • ∼11% of the other C projects 16
23. How frequent is inline assembly? • ∼28% of the most popular C projects • ∼11% of the other C projects 16 Sulong and other tools that process C cannot ignore inline assembly!
24. How “complex” is its usage? 17 36 projects (18%) with inline assembly used complicated macro-metaprogramming  ignored
25. How “complex” is its usage? 18 The majority of inline assembly fragments contain one or two instructions
26. How “complex” is its usage? 18 The majority of inline assembly fragments contain one or two instructions 1 instruction
27. How “complex” is its usage? 18 The majority of inline assembly fragments contain one or two instructions 2 instructions
28. How “complex” is its usage? 19 The majority of projects with inline assembly contain only a few fragments
29. How “complex” is its usage? 19 The majority of projects with inline assembly contain only a few fragments 10 fragments
30. How “complex” is its usage? 20 rep nop
31. How “complex” is its usage? 20 rep nop 0xF3 0x90 pause
32. How “complex” is its usage? 20 rep nop 0xF3 0x90 pause Programmers sometimes have to work around old assemblers
33. How “complex” is its usage? 21 .byte 0x66; clflush %0
34. How “complex” is its usage? 21 .byte 0x66; clflush %0 0x0FAE clflushopt
35. Which domains use inline assembly? Domain # projects % projects Crypto 32 11.7% Networking 20 10.2% Media 17 8.6% Database 16 8.1% Language implementation 15 7.6% Misc 14 6.6% Concurrency 9 4.6% SSL 8 4.1% Text processing 8 4.1% Math library 7 3.6% Web server 7 3.6% 22 The domains of inline assembly are diverse
36. How diverse is the usage of inline assembly? • ∼190 unique inline assembly fragments • ∼170 unique instructions • Implement 46 instructions  support 76% of projects 23
37. How diverse is the usage of inline assembly? • ∼190 unique inline assembly fragments • ∼170 unique instructions • Implement 46 instructions  support 76% of projects 23 Only a small subset of the ∼1000 AMD64 instructions (Heule 2016) is needed
38. How diverse is the usage of inline assembly? Instructions Contained in % projects (with inline assembly) rdtsc 26.9% cpuid 24.9% mov 23.9% <compiler barrier> 21.3% lock xchg 14.2% … … 24
39. How diverse is the usage of inline assembly? • Instruction ordering and SMP programming 25 Compiler barriers Memory barriers Atomics …
40. How diverse is the usage of inline assembly? • Instruction ordering and SMP programming • Performance optimizations 26 SIMD Endianness conversions Bitscans …
41. How diverse is the usage of inline assembly? • Instruction ordering and SMP programming • Performance optimizations • Functionality unavailable in C 27 Elapsed clock cycles CPU features Data prefetching …
42. How diverse is the usage of inline assembly? • Instruction ordering and SMP programming • Performance optimizations • Functionality unavailable in C • “Supporting” instructions (moves etc.) 28
43. Selected Threat to Validity 29 #ifdef SOME_CONDITION // use inline assembly implementations #else // use other implementation #endif Instructions are not necessarily included in the binary
44. Conclusions • We cannot ignore inline assembly • Majority of projects use few and short inline assembly fragments • <200 different instructions • But: • Projects that provide different SIMD implementations • Non-mnemonic instructions 30
45. Thanks for listening! 31 https://github.com/graalvm/sulong/ @RiggerManuel
46. Bibliography • Stefan Heule, Eric Schkufza, Rahul Sharma, and Alex Aiken. 2016. Stratified synthesis: automatically learning the x86-64 instruction set. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). • Rob Johnson and David Wagner. 2004. Finding user/kernel pointer bugs with type inference. In Proceedings of the 13th conference on USENIX Security Symposium - Volume 13 (SSYM'04), Vol. 13. USENIX Association, Berkeley, CA, USA, 9-9. 32
47. Images • Page 13: Faras Saint Anne, Public Domain, https://en.wikipedia.org/wiki/File:Faras_Saint_Anne_(detail).jpg • Page 15: Cessna 172 Airplane: Nick Dean, Frze, Creative Commons Attribution-Share Alike 3.0 Unported license, https://commons.wikimedia.org/wiki/File:WRM_Airplane_-_Flugzeug_Cessna_172_m.jpg • Page 15: Under construction, Public Domain, https://commons.wikimedia.org/wiki/Category:Under_construction_icons#/media/File:UnderCon_icon_b lack.svg • Page 18: Stein der fünften Sonne, sog. Aztekenkalender: Anagoria, GNU Free Documentation License, https://commons.wikimedia.org/wiki/File:1479_Stein_der_f%C3%BCnften_Sonne,_sog._Aztekenkalend er,_Ollin_Tonatiuh_anagoria.JPG 33