Share
Embed
Understanding GCC Builtins to Develop Better Tools
Talk at ESEC/FSE 2019
Manuel Rigger
August 28, 2019
Transcript
90 slides
-
Understanding GCC Builtins to Develop Better Tools Manuel Rigger Stefan
Marr Bram Adams Hanspeter Mössenböck @RiggerManuel 28. August 2019 Session Empirical Studies I @ ESEC/FSE 2019 -
Motivating Example Task: Print the number of leading zero bits
in a 64-bit integer (implementing a C program) 3 -
Motivating Example 7 (decimal) Task: Print the number of leading
zero bits in a 64-bit integer (implementing a C program) 3 -
Motivating Example 7 (decimal) 000…000111 (binary) Task: Print the number
of leading zero bits in a 64-bit integer (implementing a C program) 3 -
Motivating Example 7 (decimal) 000…000111 (binary) 61 leading zero bits
Task: Print the number of leading zero bits in a 64-bit integer (implementing a C program) 3 -
Approach 1: Implement in Plain C int count_leading_zeroes(unsigned long x)
{ int count = 0; unsigned long cur = x; while (cur > 0) { cur = cur >> 1; count++; } return 64 - count; } int main() { unsigned long value = 0b111; int bits = count_leading_zeroes(value); printf("%d", bits); } 4 -
int main() { unsigned long value = 0b111; int bits
= __builtin_clzl(value); printf("%d", bits); } Approach 2: Reuse Existing Functionality 6 -
int main() { unsigned long value = 0b111; int bits
= __builtin_clzl(value); printf("%d", bits); } Approach 2: Reuse Existing Functionality GCC builtins are provided directly by the (GCC) compiler 6 -
int main() { unsigned long value = 0b111; int bits
= __builtin_clzl(value); printf("%d", bits); } Approach 2: Reuse Existing Functionality bsr rsi, rsi xor rsi, 63 Compiles to 7 -
int main() { unsigned long value = 0b111; int bits
= __builtin_clzl(value); printf("%d", bits); } Approach 2: Reuse Existing Functionality bsr rsi, rsi xor rsi, 63 Compiles to GCC builtins typically result in efficient machine code 7 -
Tools While GCC builtins provide convenience to users, adding support
in other tools is challenging 8 -
int main() { unsigned long value = 0b111; int bits
= __builtin_clzl(value); printf("%d", bits); } Why Should Tool Developers Care? Finding: builtins are used by 37% of projects that we analyzed 10 -
Many Tools Exist that Support Developing C Programs Clang (LLVM)
GCC ICC Mature compilers 11 -
Many Tools Exist that Support Developing C Programs Clang (LLVM)
GCC ICC Preliminary experimentation suggested that these compilers support common GCC builtins 12 -
Many Tools Exist that Support Developing C Programs Clang (LLVM)
GCC ICC CompCert Tiny C Compiler Special-purpose compilers 13 -
Many Tools Exist that Support Developing C Programs Clang (LLVM)
GCC ICC CompCert Tiny C Compiler KCC Frama-C CIL Static analyzers 14 -
Many Tools Exist that Support Developing C Programs Clang (LLVM)
GCC ICC CompCert Tiny C Compiler KCC Frama-C CIL KLEE Various other tools DragonEgg Sulong 15 -
Goal of our work Investigate the usage of builtins to
inform tool developers about adding support for builtins in their tools 16 -
Goal of our work Investigate the usage of builtins to
inform tool developers about adding support for builtins in their tools 16 -
Broader Implications of our Study First study on compiler builtins
17 • Compiler builtins are an important feature, but not widely understood -
18 • Compiler builtins are an important feature, but not
widely understood Emerging languages like Rust also provide builtins Broader Implications of our Study -
Broader Implications of our Study • Compiler builtins are an
important feature, but not widely understood • Language design Demonstrates what features programming languages like C lack 19 -
Broader Implications of our Study • Compiler builtins are an
important feature, but not widely understood • Language design • Developer feedback Informs developers on how builtin usage affects how well their code can be analyzed 20 -
Broader Implications of our Study • Compiler builtins are an
important feature, but not widely understood • Language design • Developer feedback • Implementation and maintenance of compilers Informs compilers developers about which builtins are often used 21 -
1. How frequently are builtins used? 2. How well do
tools that process C code support builtins? Research Questions 22 -
1. How frequently are builtins used? 2. How well do
tools that process C code support builtins? 3. How many builtins must be implemented to support most projects? Research Questions 22 -
1. How frequently are builtins used? 2. How well do
tools that process C code support builtins? 3. How many builtins must be implemented to support most projects? 4. (How does builtin usage vary over a project’s lifetime?) Research Questions 22 -
1. How frequently are builtins used? 2. How well do
tools that process C code support builtins? 3. How many builtins must be implemented to support most projects? 4. (How does builtin usage vary over a project’s lifetime?) 5. (For what purposes are builtins used?) Research Questions 22 -
Obtain C Projects Filter C Projects Methodology > 100 LOC?
23 ~4,900 projects -
Obtain C Projects Filter C Projects Extracting Builtin Uses grep
__builtin_clzl SQLite3 Database Methodology 23 -
Obtain C Projects Filter C Projects Extracting Builtin Uses Filter
Builtin Name Records // __builtin_clzl /* __builtin_clzl */ Methodology 23 -
Obtain C Projects Filter C Projects Extracting Builtin Uses Filter
Builtin Name Records Analyze the results Methodology 23 -
Obtain C Projects Filter C Projects Extracting Builtin Uses Filter
Builtin Name Records Analyze the results Methodology All steps are replicable, see https://github.com/jku-ssw/gcc-builtin-study 23 -
Obtain C Projects Filter C Projects Extracting Builtin Uses Filter
Builtin Name Records Analyze the results Methodology All steps are replicable, see https://github.com/jku-ssw/gcc-builtin-study 23 Repeating the study on newly- added builtins requires little effort -
Overview 37% 0 500 1000 1500 2000 Number of projects
Used builtins Machine-independent GCC builtins are used by many projects 25 -
Overview 37% 0 500 1000 1500 2000 Number of projects
Used builtins Machine-independent GCC builtins are used by many projects 25 ~3,000 different builtins were used -
Overview 37% 0 500 1000 1500 2000 Number of projects
Used builtins Machine-independent GCC builtins are used by many projects 25 ~3,000 different builtins were used Builtins are used infrequently within a project: 1 builtin every ~6K LOC -
Overview 37% 36% 0 500 1000 1500 2000 Number of
projects Used builtins Machine-independent Machine-specific Many projects rely on architecture- independent builtins 26 -
Architecture-independent Builtins __builtin_clzl() Compiles to Definition: Architecture-independent builtins are typically
supported on all common architectures 27 -
Architecture-independent Builtins Mainly builtins from the “other” and “sync” categories
are frequently used 28 -
Architecture-independent Builtins Builtin Category Projects __builtin_expect other (compiler interaction) 890
/ 48.3% __builtin_clz other (bitwise operation) 536 / 29.1% __builtin_bswap32 other (bitwise operation) 483 / 26.2% __builtin_constant_p other (compiler interaction) 430 / 23.3% __builtin_alloca other (stack allocation) 373 / 20.2% __sync_synchronize sync 356 / 19.3% __builtin_bswap64 other (bitwise operation) 347 / 18.8% __sync_fetch_and_add sync 332 / 18.0% __builtin_ctz other (bitwise operation) 324 / 17.6% __builtin_bswap16 other (bitwise operation) 304 / 16.5% 29 -
Architecture-independent Builtins Builtin Category Projects __builtin_expect other (compiler interaction) 890
/ 48.3% __builtin_clz other (bitwise operation) 536 / 29.1% __builtin_bswap32 other (bitwise operation) 483 / 26.2% __builtin_constant_p other (compiler interaction) 430 / 23.3% __builtin_alloca other (stack allocation) 373 / 20.2% __sync_synchronize sync 356 / 19.3% __builtin_bswap64 other (bitwise operation) 347 / 18.8% __sync_fetch_and_add sync 332 / 18.0% __builtin_ctz other (bitwise operation) 324 / 17.6% __builtin_bswap16 other (bitwise operation) 304 / 16.5% if (__builtin_expect(x, 0)) foo(); 29 -
Architecture-independent Builtins Builtin Category Projects __builtin_expect other (compiler interaction) 890
/ 48.3% __builtin_clz other (bitwise operation) 536 / 29.1% __builtin_bswap32 other (bitwise operation) 483 / 26.2% __builtin_constant_p other (compiler interaction) 430 / 23.3% __builtin_alloca other (stack allocation) 373 / 20.2% __sync_synchronize sync 356 / 19.3% __builtin_bswap64 other (bitwise operation) 347 / 18.8% __sync_fetch_and_add sync 332 / 18.0% __builtin_ctz other (bitwise operation) 324 / 17.6% __builtin_bswap16 other (bitwise operation) 304 / 16.5% 29 -
Architecture-independent Builtins Builtin Category Projects __builtin_expect other (compiler interaction) 890
/ 48.3% __builtin_clz other (bitwise operation) 536 / 29.1% __builtin_bswap32 other (bitwise operation) 483 / 26.2% __builtin_constant_p other (compiler interaction) 430 / 23.3% __builtin_alloca other (stack allocation) 373 / 20.2% __sync_synchronize sync 356 / 19.3% __builtin_bswap64 other (bitwise operation) 347 / 18.8% __sync_fetch_and_add sync 332 / 18.0% __builtin_ctz other (bitwise operation) 324 / 17.6% __builtin_bswap16 other (bitwise operation) 304 / 16.5% 29 -
Architecture-specific Builtins 37% 36% 8% 0 500 1000 1500 2000
Number of projects Used builtins Machine-independent Machine-specific Architecture-specific builtins are less frequently used 30 -
Architecture-specific Builtins 37% 36% 8% 0 500 1000 1500 2000
Number of projects Used builtins Machine-independent Machine-specific A project can rely on both architecture-specific and architecture-independent builtins 30 -
Architecture-specific Builtins vec_perm() Compiles to Definition: Architecture-specific builtins are typically
supported only on a specific architecture 31 -
Unique Uses Per Project that Uses Builtins 17 3 4
A project that uses machine-specific builtins uses them in a larger number 33 -
Test Suite for 100 most-frequently used builtins #include <assert.h> int
main() { volatile unsigned long value = -1; assert(__builtin_clzl(value) == 0); value = (unsigned long)-1 >> 3; assert(__builtin_clzl(value) == 3); value = (long)((unsigned long)-1 >> 5) - 4; assert(__builtin_clzl(value) == 5); return 0; } 35 Goal was to test common and corner case -
Test Suite for 100 most-frequently used builtins #include <assert.h> int
main() { volatile unsigned long value = -1; assert(__builtin_clzl(value) == 0); value = (unsigned long)-1 >> 3; assert(__builtin_clzl(value) == 3); value = (long)((unsigned long)-1 >> 5) - 4; assert(__builtin_clzl(value) == 5); return 0; } Compile Analyze Parse Warning? Error? 35 Goal was to test common and corner case -
Tool Support 36 Tools that concretely or abstractly execute code
support builtins also well -
Tool Support 36 Static analysis tools and special- purpose compilers
have limited/incorrect support -
Example: Bugs in CompCert https://github.com/AbsInt/CompCert/issues/243 __builtin_clz for long and long
long incorrectly assumed a 32-bit integer #include <assert.h> int main() { volatile unsigned long value = -1; assert(__builtin_clzl(value) == 0); value = (unsigned long)-1 >> 3; assert(__builtin_clzl(value) == 3); value = (long)((unsigned long)-1 >> 5) - 4; assert(__builtin_clzl(value) == 5); return 0; } a.out: test.c:7: int main(): Assertion `__builtin_clzl(value) == 3' failed. Aborted 37 -
Greedy Implementation Strategy # Builtin Supported Projects … … …
24 __sync_fetch_and_sub 43.16% 25 __sync_lock_release 44.47% 26 __builtin_clzl 45.45% 27 __builtin_choose_expr 46.23% 28 __builtin_frame_address 47.14% 29 __builtin_clzll 47.85% 30 __builtin_ctzll 50.79% … … … 41 -
Greedy Implementation Strategy # Builtin Supported Projects … … …
24 __sync_fetch_and_sub 43.16% 25 __sync_lock_release 44.47% 26 __builtin_clzl 45.45% 27 __builtin_choose_expr 46.23% 28 __builtin_frame_address 47.14% 29 __builtin_clzll 47.85% 30 __builtin_ctzll 50.79% … … … Machine-independent builtins are the “low-hanging fruits” to implement 41 -
Greedy Implementation Strategy # Builtin Supported Projects … … …
609 vreinterpret_u32_f32 89.37% 610 vld1_dup_s32 89.59% 611 vst1_s32 89.65% 612 vshrn_n_u32 89.65% 613 vshrq_n_u64 89.65% … … … 43 -
Greedy Implementation Strategy # Builtin Supported Projects … … …
609 vreinterpret_u32_f32 89.37% 610 vld1_dup_s32 89.59% 611 vst1_s32 89.65% 612 vshrn_n_u32 89.65% 613 vshrq_n_u64 89.65% … … … Machine-specific builtins represent the “long tail of the distribution” 43 -
Summary and Discussion GCC builtins are a challenge for tool
developers 37% of projects use GCC builtins (mostly machine-independent ones) Many tools lack support for GCC builtins Exponential number of builtins to support a specific number of projects @RiggerManuel https://github.com/jku-ssw/gcc-builtin-study -
Summary and Discussion GCC builtins are a challenge for tool
developers 37% of projects use GCC builtins (mostly machine-independent ones) Many tools lack support for GCC builtins Exponential number of builtins to support a specific number of projects @RiggerManuel https://github.com/jku-ssw/gcc-builtin-study -
Summary and Discussion GCC builtins are a challenge for tool
developers 37% of projects use GCC builtins (mostly machine-independent ones) Many tools lack support for GCC builtins Exponential number of builtins to support a specific number of projects @RiggerManuel https://github.com/jku-ssw/gcc-builtin-study -
Summary and Discussion GCC builtins are a challenge for tool
developers 37% of projects use GCC builtins (mostly machine-independent ones) Many tools lack support for GCC builtins Exponential number of builtins to support a specific number of projects @RiggerManuel https://github.com/jku-ssw/gcc-builtin-study -
Summary and Discussion GCC builtins are a challenge for tool
developers 37% of projects use GCC builtins (mostly machine-independent ones) Many tools lack support for GCC builtins Exponential number of builtins to support a specific number of projects @RiggerManuel https://github.com/jku-ssw/gcc-builtin-study -
Summary and Discussion GCC builtins are a challenge for tool
developers 37% of projects use GCC builtins (mostly machine-independent ones) Many tools lack support for GCC builtins Exponential number of builtins to support a specific number of projects @RiggerManuel https://github.com/jku-ssw/gcc-builtin-study