ICOOOLPS'16: Sulong - Execution of LLVM-Based Languages on the JVM

Talk at ICOOOLPS 2016

1. Sulong - Execution of LLVM-Based Languages on the JVM Manuel Rigger Matthias Grimmer Hanspeter Mössenböck Johannes Kepler University Linz ICOOOLPS 18. July, 2016
2. Java Platform JVM [1] Rose. "Bytecodes meet combinators: invokedynamic on the JVM." [2] Würthinger et al. "One VM to rule them all." 2
3. Java Platform: Native Languages? JVM Java Native Interface Native side 3
4. Disadvantages • Slow • Transitions between Java and native code • Conversions/marshaling • Language boundaries are compilation boundaries • Difficult to use • boiler plate code • Breaks Java‘s safety guarantees Java Native Interface Native side JVM 4
5. Goals Native Interface Static + Dynamic Optimizations Memory Safety 5
6. System Overview LLVM IR Interpreter Truffle LLVM IR Clang C C++ GCC Fortran Other LLVM frontend ... JVM + Graal tooling 6
7. System Overview LLVM IR Interpreter Truffle LLVM IR Clang C C++ GCC Fortran Other LLVM frontend ... JVM + Graal tooling Use LLVM front ends and LLVM IR 6
8. System Overview LLVM IR Interpreter • Only component that has to be implemented LLVM IR Interpreter Truffle LLVM IR Clang C C++ GCC Fortran Other LLVM frontend ... JVM + Graal tooling 6
9. System Overview Use Truffle and Graal • Language Interoperability • Dynamic compilation LLVM IR Interpreter Truffle LLVM IR Clang C C++ GCC Fortran Other LLVM frontend ... JVM + Graal tooling 6
11. Static + Dynamic Optimizations Can we combine static with dynamic optimizations to top peak performance of static compilers? 7
12. Static + Dynamic Optimizations LLVM: advanced static optimizations Truffle and Graal: speculative optimizations based on profiling Compile- time Link- time Run- time 8
13. LLVM IR Interpreter Truffle LLVM IR JVM+Graal opt Optimization Pipeline 9
14. LLVM IR Interpreter Truffle LLVM IR JVM+Graal opt Optimization Pipeline Static LLVM optimizations 9
15. LLVM IR Interpreter Truffle LLVM IR JVM+Graal opt Optimization Pipeline Profiling-based optimizations • Polymorphic inline caches for function pointer calls • Function inlining Static LLVM optimizations 9
16. LLVM IR Interpreter Truffle LLVM IR JVM+Graal opt Optimization Pipeline Profiling-based optimizations • Polymorphic inline caches for function pointer calls • Function inlining Compilation of the AST Static LLVM optimizations 9
17. Dynamic Optimization: Example 10 void bubble_sort(int *numbers, int count, (*compare)(int a, int b)) { for (int i = 0; i < count; i++) { for (int j = 0; j < count - 1; j++) { if (compare(numbers[j], numbers[j+1]) > 0) { swap(&numbers[j], &numbers[j+1]); } } } } int ascending(int a, int b){ return a - b; } int descending(int a, int b){ return b - a; }
18. Dynamic Optimization: Example 11 bubble_sort bubble_sort ascending ascending ~1.3x speedup Polymorphic inline cache + Function inlining
19. Performance: C Benchmarks Lower is better, run-time ratio relative to Clang O3 12
20. 13 Static + Dynamic Optimizations
21. Native Interface We can use Sulong as an alternative to JNI and to implement other language‘s native interfaces 13 Static + Dynamic Optimizations
22. Sulong as an Alternative to JNI Use Sulong as a Java library! Java C/C++/ Fortran Sulong 14
23. Case Studies: Sulong to Implement NFIs JRuby+Truffle FastR Graal.JS C/C++/ Fortran 15
24. Case Studies: Sulong to Implement NFIs JRuby+Truffle FastR Graal.JS C/C++/ Fortran 15
25. Truffle as a Multi Language Runtime [4] Grimmer, et al. "High-performance cross-language interoperability in a multi-language runtime." JVM Truffle Language Interoperability 16
26. Truffle as a Multi Language Runtime #include<stdio.h> struct complex { double r; double i; }; int main() { struct complex *a = …; struct complex *b = …; add(a, b); } function add(a, b) { var result = {r:0, i:0}; result.r = a.r + b.r; result.i = a.i + b.i; return result; } main.c complex.js 17
27. Truffle as a Multi Language Runtime #include<stdio.h> struct complex { double r; double i; }; int main() { struct complex *a = …; struct complex *b = …; add(a, b); } function add(a, b) { var result = {r:0, i:0}; result.r = a.r + b.r; result.i = a.i + b.i; return result; } add(a, b) main.c complex.js 17
28. Truffle as a Multi Language Runtime #include<stdio.h> struct complex { double r; double i; }; int main() { struct complex *a = …; struct complex *b = …; add(a, b); } function add(a, b) { var result = {r:0, i:0}; result.r = a.r + b.r; result.i = a.i + b.i; return result; } add(a, b) a->r b->r a->i b->i main.c complex.js 17
29. Case Studies: Sulong to Implement NFIs 18 program.c vm.h Program JRuby+Truffle Written against Interop call Interop call vm.c [5] Grimmer, et al. "Dynamically composing languages in a modular way: supporting C extensions for dynamic languages."
30. Native Interface Static + Dynamic Optimizations 19
31. Native Interface Static + Dynamic Optimizations Memory Safety Can we provide memory safety for C/C++ and Fortran? 19
32. Memory Errors in C int *arr = malloc(4 * sizeof(int)) 20 [6] Szekeres, et al. "Sok: Eternal war in memory."
33. Memory Errors in C int *arr = malloc(4 * sizeof(int)) … = arr[5] arr[5] = … 20 [6] Szekeres, et al. "Sok: Eternal war in memory."
34. Memory Errors in C int *arr = malloc(4 * sizeof(int)) … = arr[5] arr[5] = … Spatial memory safety error 20 [6] Szekeres, et al. "Sok: Eternal war in memory."
35. Memory Errors in C int *arr = malloc(4 * sizeof(int)) … = arr[5] arr[5] = … free(arr); … = arr[0] arr[0] = … Spatial memory safety error 20 [6] Szekeres, et al. "Sok: Eternal war in memory."
36. Memory Errors in C int *arr = malloc(4 * sizeof(int)) … = arr[5] arr[5] = … free(arr); … = arr[0] arr[0] = … Spatial memory safety error Temporal memory safety error 20 [6] Szekeres, et al. "Sok: Eternal war in memory."
37. Memory Errors in C int *arr = malloc(4 * sizeof(int)) … = arr[5] arr[5] = … free(arr); … = arr[0] arr[0] = … Spatial memory safety error Temporal memory safety error 20 • Segmentation faults • (Silent) data corruption • Reading of sensitive data • Code injection [6] Szekeres, et al. "Sok: Eternal war in memory."
38. Current Sulong implementation • Relies on unmanaged memory (sun.misc.Unsafe) • https://github.com/graalvm/sulong • Goal: use managed Java objects 21
39. Prevent Spatial Errors … = arr[5] arr[5] = … 22 (arr[5] ≈ arr + sizeof(int) * 5) ManagedAddress offset=20 data I32Array elementSize=4 contents {1, 2, 3, 4}
40. Prevent Spatial Errors … = arr[5] arr[5] = … contents[20 / 4]  ArrayOutOfBoundsException 22 (arr[5] ≈ arr + sizeof(int) * 5) ManagedAddress offset=20 data I32Array elementSize=4 contents {1, 2, 3, 4}
41. Prevent Temporal Errors free(arr); … = arr[0] arr[0] = … 23 ManagedAddress offset=0 data I32Array elementSize=4 contents=null
42. Prevent Temporal Errors free(arr); … = arr[0] arr[0] = … contents[0]  NullPointerException 23 ManagedAddress offset=0 data I32Array elementSize=4 contents=null
43. Limitations • Warm-up and start-up • Current limitations: multithreading, 80 bit floats, inline assembler support • Support of all library functions • Conundrum between memory safety and programmer freedom 24
44. Summary 25 Native Interface Static + Dynamic Optimizations Memory Safety JRuby+Truffle FastR Graal.JS C/C++/ Fortran … = arr[5] arr[5] = … contents[20 / 4]  ArrayOutOfBoundsException ManagedAddress offset=20 data I32Array elementSize=4 contents {1, 2, 3, 4}
45. References [1] Rose, John R. "Bytecodes meet combinators: invokedynamic on the JVM." Proceedings of the Third Workshop on Virtual Machines and Intermediate Languages. ACM, 2009. [2] Würthinger, Thomas, et al. "One VM to rule them all." Proceedings of the 2013 ACM international symposium on New ideas, new paradigms, and reflections on programming & software. ACM, 2013. [3] Lattner, Chris, and Vikram Adve. "LLVM: A compilation framework for lifelong program analysis & transformation." Code Generation and Optimization, 2004. CGO 2004. International Symposium on. IEEE, 2004. [4] Grimmer, Matthias, et al. "High-performance cross-language interoperability in a multi-language runtime." Proceedings of the 11th Symposium on Dynamic Languages. ACM, 2015. [5] Grimmer, Matthias, et al. "Dynamically composing languages in a modular way: supporting C extensions for dynamic languages." Proceedings of the 14th International Conference on Modularity. ACM, 2015. [6] Szekeres, Laszlo, et al. "Sok: Eternal war in memory." Security and Privacy (SP), 2013 IEEE Symposium on. IEEE, 2013. 26