Programming '18: Introspection for C and its Applications to Library Robustness

See https://2018.programming-conference.org/event/programming-2018-papers-introspection-for-c-and-its-applications-to-library-robustness.

1. Introspection for C and its Applications to Library Robustness Manuel Rigger1, René Mayrhofer1, Roland Schatz2, Matthias Grimmer2, Hanspeter Mössenböck1 1 Johannes Kepler University Linz, Austria 2 Oracle Labs Linz, Austria <Programming> 2018, 12 April 2018, Nice, France.
2. Problem: Errors in C 2 • Problem 1: C lacks automatic checks • Errors can be exploited by attackers or cause hard-to-find-bugs • Problem 2: C objects lack type information • No defensive programming
3. Problem: Errors in C • No bounds checks • No bounds information 3 int *arr = malloc(3 * sizeof(int)); arr[5] = … Buffer Overflows
4. Problem: Errors in C 4 free(stackobject); Invalid free error free(heapobject); free(heapobject); Double-free error free(heapobject); heapobject[0] = val; Use-after-free error free(heapobject); Memory leak Memory Management Errors • No automatic memory management • No location/liveness information
5. Problem: Errors in C 5 Type Confusion void apply(long* arr, size_t n, long f(long arg1)); • No type safety • No type information
6. Problem: Errors in C 5 Type Confusion void apply(long* arr, size_t n, long f(long arg1)); int f(int arg1) • No type safety • No type information
7. Problem: Errors in C 6 Format String Vulnerabilities printf("%d %d %s" , 3, 5); Missing argument printf("%s", 3); Type Confusion • No checks for accessing variadic arguments • No information for number and types of variadic arguments
8. Problem: Errors in C • Problem 1: C lacks automatic checks • Errors can be exploited by attackers or cause hard-to-find-bugs • Problem 2: C objects lack type information • No defensive programming 7 Addressed by state-of-the-art tools
9. State-of-the-art Run-time Approaches 8 int *arr = malloc(3 * sizeof(int)); arr[5] = … Detected buffer overflow  exiting program
10. State-of-the-art Run-time Approaches • LLVM’s sanitizers • Valgrind, Dr. Memory • Libcrunch • SoftBound+CETS • Safe Sulong 9
11. State-of-the-art Run-time Approaches • LLVM’s sanitizers • Valgrind, Dr. Memory • Libcrunch • SoftBound+CETS • Safe Sulong 9 These systems track object metadata to implement checks
12. Problem: Errors in C • Problem 1: C lacks automatic checks • Errors can be exploited by attackers or cause hard-to-find-bugs • Problem 2: C objects lack type information • No defensive programming 10 Unaddressed
13. Goal 11 Expose metadata through an introspection interface so that programmers can enhance the robustness of their libraries
14. Motivation 12 Goals Log error and continue Return error code Add assertions
15. Motivation 12 Goals Log error and continue Return error code Add assertions Programmers could “override” the default behavior of aborting the program
16. Structure 13 Ongoing work Implementation in tools Case study on libc Introspection interface
17. Introspection interface 14
18. Introspection Functions 15 Intro- spection Object bounds Memory location Types Variadic arguments
19. Bounds 16 long size_right(void *); long size_left(void *);
20. Bounds 17 int *arr = malloc(sizeof (int) * 10) ; int *ptr = &(arr[4]); printf ("%ld\n", size_left(ptr)); // prints 16 _size_left() sizeof(int) * 10
21. Bounds 18 int *arr = malloc(sizeof (int) * 10) ; int *ptr = &(arr[4]); printf ("%ld\n", size_left(ptr)); // prints 16 printf ("%ld\n", size_right(ptr)); // prints 24 _size_right() sizeof(int) * 10
22. Memory Location 19 enum Location location(void *); enum Location { INVALID, AUTOMATIC, DYNAMIC, STATIC };
23. Memory Location 20 int a; void func() { static int b; int c; int *d = malloc(sizeof(int) * 10); free(d); }
24. Memory Location 20 int a; void func() { static int b; int c; int *d = malloc(sizeof(int) * 10); free(d); } location(&a)  STATIC
25. Memory Location 20 int a; void func() { static int b; int c; int *d = malloc(sizeof(int) * 10); free(d); } location(&b)  STATIC
26. Memory Location 20 int a; void func() { static int b; int c; int *d = malloc(sizeof(int) * 10); free(d); } location(&c)  AUTOMATIC
27. Memory Location 20 int a; void func() { static int b; int c; int *d = malloc(sizeof(int) * 10); free(d); } location(d)  DYNAMIC
28. Memory Location 20 int a; void func() { static int b; int c; int *d = malloc(sizeof(int) * 10); free(d); } location(d)  INVALID
29. Type Information 21 void* try_cast(void *, struct Type*);
30. Type Information 21 void* try_cast(void *, struct Type*); Return the pointer or NULL similar to dynamic_cast in C++
31. Type Information 22 void* try_cast(void *, struct Type*);
32. Type Information 22 void* try_cast(void *, struct Type*); Check if an object is "compatible" with a type
33. Type Information 23 int arr[10]; int *ptr = &(arr[9]); int val; val = try_cast(&ptr, type(val));
34. Variadic Arguments 24 int count_varargs(); void* get_vararg(int i, struct Type* type);
35. Variadic Arguments 25 int printf(const char * format, ... ) { if (count_varargs() != 3) { abort(); } int int1 = *get_vararg(0, type(int1)); int int2 = *get_vararg(1, type(&int2)); char* str = *get_vararg(2, type(&str)); } "%d %d %s"
36. Case Study on Libc 26
37. Introspection Goals 27 Goals Improve availability of the system Fix incomplete APIs Improve bug-finding capabilities
38. Introspection to Increase Availability • Continue execution after mitigating an error • E.g., make libc robust against not NUL-terminated strings 28 size_t strlen(const char *str); char *strcpy(char *dest, const char *src); …
39. Example: strlen() 29 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; }
40. Example: strlen() 29 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; } P r o g r a m m i n g \0 ... ...
41. Example: strlen() 29 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; } P r o g r a m m i n g \0 ... ...
42. Example: strlen() 29 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; } 11 P r o g r a m m i n g \0 ... ...
43. Example: strlen() 30 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; } P r o g r a m m i n g ... ...
44. Example: strlen() 30 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; } P r o g r a m m i n g ... ...
45. Example: strlen() 30 size_t strlen(const char *str) { size_t len = 0; while (*str != '\0') { len++; str++; } return len; } 23415 P r o g r a m m i n g ... ...
46. size_t strlen(const char *str) { size_t len = 0; while ( size_right(str) > 0 && *str != '\0') { len++; str++; } return len; } Example: strlen() 31 P r o g r a m m i n g ... ...
47. size_t strlen(const char *str) { size_t len = 0; while ( size_right(str) > 0 && *str != '\0') { len++; str++; } return len; } Example: strlen() 31 P r o g r a m m i n g ... ...
48. size_t strlen(const char *str) { size_t len = 0; while ( size_right(str) > 0 && *str != '\0') { len++; str++; } return len; } Example: strlen() 31 11 P r o g r a m m i n g ... ...
49. Introspection to Improve Bug-Finding • Check invariants • Abort when detecting inconsistencies • E.g., allow libc’s “safe” functions to detect incorrect buffer sizes 32 size_t strnlen_s(const char *str, size_t maxlen); errno_t strcpy_s(char *dest, rsize_t maxdestsz, const char *src);
50. Improve bug-finding capabilities 33 size_t strnlen_s(char *str, rsize_t maxsize) { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; }
51. Improve bug-finding capabilities 33 size_t strnlen_s(char *str, rsize_t maxsize) { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; } P r o g r a m m i n g \0 ... ... 100
52. Improve bug-finding capabilities 33 size_t strnlen_s(char *str, rsize_t maxsize) { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; } P r o g r a m m i n g \0 ... ... 100 11
53. Improve bug-finding capabilities 33 size_t strnlen_s(char *str, rsize_t maxsize) { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; } P r o g r a m m i n g \0 ... ... 100 11 Correct result but did not detect incorrect buffer size
54. Improve bug-finding capabilities 34 size_t strnlen_s(char *str, rsize_t maxsize) { if ( size_right(str) < maxsize) { abort(); } else { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; } }
55. Improve bug-finding capabilities 34 size_t strnlen_s(char *str, rsize_t maxsize) { if ( size_right(str) < maxsize) { abort(); } else { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; } } P r o g r a m m i n g \0 ... ... 100
56. Improve bug-finding capabilities 34 size_t strnlen_s(char *str, rsize_t maxsize) { if ( size_right(str) < maxsize) { abort(); } else { size_t i = 0; for (; i < maxsize && s[i] != '\0'; i++); return i; } } P r o g r a m m i n g \0 ... ... 100 Abort
57. Introspection to Fix Errors in API Design 35 char *gets(char *str);
58. Introspection to Fix Errors in API Design 35 char *gets(char *str); ... ...
59. Introspection to Fix Errors in API Design 35 char *gets(char *str); ... ...
60. Introspection to Fix Errors in API Design 36 char* gets(char *str) { int size = size_right(str); return gets_s(str, size); }
61. Introspection to Fix Errors in API Design 36 Make gets() robust against input that would overflow the buffer char* gets(char *str) { int size = size_right(str); return gets_s(str, size); }
62. Other examples 37 void qsort(void *base, size_t nitems, size_t size, int (*compar)(const void *, const void*)); void *bsearch(const void *key, const void *base, size_t nel, size_t width, int (*compar)(const void *, const void *));
63. Other examples 37 void qsort(void *base, size_t nitems, size_t size, int (*compar)(const void *, const void*)); void *bsearch(const void *key, const void *base, size_t nel, size_t width, int (*compar)(const void *, const void *)); Verify the function pointer
64. Other examples 38 int printf(const char *format, ...); int sprintf(char *str, const char *format, ...); int fprintf(FILE *stream, const char *format, ...);
65. Other examples 38 int printf(const char *format, ...); int sprintf(char *str, const char *format, ...); int fprintf(FILE *stream, const char *format, ...); Verify the number and types of variadic arguments
66. Other examples 39 void *realloc(void *ptr, size_t new_size); void free(void *ptr);
67. Other examples 39 void *realloc(void *ptr, size_t new_size); void free(void *ptr); Verify that the object was dynamically allocated
68. Implementation in Tools 40
69. Implementation in Safe Sulong 41 JVM C C++ Fortran ... Execute on
70. Implementation in Safe Sulong 41 JVM C C++ Fortran ... Execute on Safe Sulong tracks all necessary metadata
71. Implementation in Safe Sulong 42 int *arr = malloc(3 * sizeof(int)); int *ptr = &arr[1];
72. Implementation in Safe Sulong 42 int *arr = malloc(3 * sizeof(int)); int *ptr = &arr[1]; ptr: Address offset=4 data I32Array contents {0, 0, 0} location=AUTOMATIC location=DYNAMIC
73. Implementation in Safe Sulong 42 int *arr = malloc(3 * sizeof(int)); int *ptr = &arr[1]; ptr: Address offset=4 data I32Array contents {0, 0, 0} location=AUTOMATIC location=DYNAMIC size_right(ptr)  data.contents.length * sizeof(int) – offset  3 * 4 – 4 = 8
74. Implementation in Safe Sulong 43 int *arr = malloc(3 * sizeof(int)); int *ptr = &arr[1]; ptr: Address offset=4 data I32Array contents {0, 0, 0} location=AUTOMATIC location=DYNAMIC location(ptr)  DYNAMIC
75. Implementation in Safe Sulong 44 int *arr = malloc(3 * sizeof(int)); int *ptr = &arr[1]; ptr: Address offset=4 data I32Array contents {0, 0, 0} location=AUTOMATIC location=DYNAMIC try_cast(ptr, type(*ptr))  ptr
76. Ongoing Work 45 • Implementation in other tools • Case Study on real-world bugs
77. Ongoing work: LLVM’s AddressSanitizer 46 ... ... • LLVM’s AddressSanitizer: memory error detector based on shadow memory size_right()
78. Ongoing work: LLVM’s AddressSanitizer 46 ... ... • LLVM’s AddressSanitizer: memory error detector based on shadow memory size_right()
79. Ongoing work: GCC’s Intel MPX Bounds Checks Instrumentation • Intel MPX: bounds registers and instructions • GCC uses them to detect memory errors 47
80. Ongoing work: GCC’s Intel MPX Bounds Checks Instrumentation • Intel MPX: bounds registers and instructions • GCC uses them to detect memory errors 47 ssize_t size_right(void* p){ ssize_t upper_bounds = (ssize_t)__builtin___bnd_get_ptr_ubound(p); size_t size = (size_t) (upper_bounds + 1) - (size_t) p; return (ssize_t) size; }
81. Ongoing work: SoftBound • SoftBound: bounds checker • Tracks upper bounds for each pointer 48 ssize_t _size_right(const char* p) { const char* bound = __softboundcets_load_bound_shadow_stack(1); return bound - p; }
82. Ongoing work: Real-world bugs • Case study on bugs contained in the CVE database • Libc goal: Availability 49 https://cve.mitre.org/
83. CVE-2017-14493 • Dnsmasq: DHCP server • Incorrect size argument to memcpy() caused buffer overflow 50 state->mac_len = opt6_len(opt) - 2; memcpy(&state->mac[0], opt6_ptr(opt, 2), state->mac_len);
84. CVE-2017-14493 • Dnsmasq: DHCP server • Incorrect size argument to memcpy() caused buffer overflow 50 state->mac_len = opt6_len(opt) - 2; memcpy(&state->mac[0], opt6_ptr(opt, 2), state->mac_len); The server stayed fully functional after mitigation
85. CVE-2017-9047 • Libxml2: XML parsing library • String concatenating caused a buffer overflow 51 if (content->name != NULL) strcat(buf, (char *) content->name);
86. CVE-2017-9047 • Libxml2: XML parsing library • String concatenating caused a buffer overflow 51 The parser printed a truncated error message if (content->name != NULL) strcat(buf, (char *) content->name);
87. CVE-2017-16352 • GraphicsMagick: image processing library • Incorrect size argument to strncpy() 52 for (p=image->directory; *p != ’\0’; p++) { q=p; while ((*q != ’\n’) && (*q != ’\0’)) q++; (void) strncpy(image_info->filename,p,q-p); image_info->filename[q-p]=’\0’; }
88. CVE-2017-16352 • GraphicsMagick: image processing library • Incorrect size argument to strncpy() 52 The error was mitigated… for (p=image->directory; *p != ’\0’; p++) { q=p; while ((*q != ’\n’) && (*q != ’\0’)) q++; (void) strncpy(image_info->filename,p,q-p); image_info->filename[q-p]=’\0’; }
89. 53 for (p=image->directory; *p != ’\0’; p++) { q=p; while ((*q != ’\n’) && (*q != ’\0’)) q++; (void) strncpy(image_info- >filename,p,q-p); image_info->filename[q-p]=’\0’; }
90. CVE-2017-16352 • GraphicsMagick: image processing library • Incorrect size argument to strncpy() 54 for (p=image->directory; *p != ’\0’; p++) { q=p; while ((*q != ’\n’) && (*q != ’\0’)) q++; (void) strncpy(image_info->filename,p,q-p); image_info->filename[q-p]=’\0’; }
91. CVE-2017-16352 • GraphicsMagick: image processing library • Incorrect size argument to strncpy() 54 … but the application code had a subsequent out-of-bounds access for (p=image->directory; *p != ’\0’; p++) { q=p; while ((*q != ’\n’) && (*q != ’\0’)) q++; (void) strncpy(image_info->filename,p,q-p); image_info->filename[q-p]=’\0’; }
92. Discussion and Conclusion 55
93. Discussion • Introspection as a complement to automatic checks 56 P r o g r a m m i n g ... ...
94. Discussion • Introspection as a complement to automatic checks 56 size_t strlen(const char *str) P r o g r a m m i n g ... ... 11
95. Discussion • Introspection as a complement to automatic checks 56 size_t strlen(const char *str) P r o g r a m m i n g ... ... size_t my_strlen(const char *str) 11 abort
96. Discussion • Introspection as a complement to automatic checks 56 size_t strlen(const char *str) P r o g r a m m i n g ... ... size_t my_strlen(const char *str) 11 abort Automatic checks still abort if the error is not mitigated
97. Discussion • Introspection as a complement to automatic checks • What about partial/no support of introspection? 57
98. Discussion • Introspection as a complement to automatic checks • What about partial/no support of introspection? 57 size_right(ptr); LONG_MAX
99. Discussion • Introspection as a complement to automatic checks • What about partial/no support of introspection? 57 size_right(ptr); LONG_MAX Sensible default values and conservative checks?
100. Discussion • Introspection as a complement to automatic checks • What about partial/no support of introspection? • Safer languages 58 Legacy software
101. Discussion • Introspection as a complement to automatic checks • What about partial/no support of introspection? • Safer languages • Programming effort 59 Introspection checks only useful for frequently used libraries
102. Conclusion 60 @RiggerManuel