Sulong, and Thanks for All the Fish Manuel Rigger, Jacob Kreindl, Hanspeter Mössenböck Roland Schatz, Christian Häubl MoreVMs 2018, April 9, 2018 

Project Sulong 2 Execute LLVM-based languages on the GraalVM 

3 https://twitter.com/LucasWerkmeistr/status/780829630890672128 So Long, and Thanks for All the Fish: fourth book of the Hitchhiker's Guide to the Galaxy by Douglas Adams 

Sulong as Part of GraalVM 4 Java Virtual Machine Graal Compiler Truffle Framework https://github.com/graalvm/ TruffleRuby Graal.js Graal.python FastR (Würthinger et al. 2016) 

Sulong as Part of GraalVM 4 Java Virtual Machine Graal Compiler Truffle Framework https://github.com/graalvm/ TruffleRuby Graal.js Graal.python FastR Optimization Boundary (Würthinger et al. 2016) 

Sulong as Part of GraalVM 5 Java Virtual Machine Graal Compiler Truffle Framework https://github.com/graalvm/ TruffleRuby Graal.js Graal.python FastR Optimization Boundary Java Native Interface (Würthinger et al. 2016) 

Sulong as Part of GraalVM 6 Java Virtual Machine Graal Compiler Truffle Framework https://github.com/graalvm/ TruffleRuby Graal.js Graal.python FastR Optimization Boundary LLVM IR Interpreter LLVM IR Clang Flang (Würthinger et al. 2016) 

Sulong as Part of GraalVM 7 Java Virtual Machine Graal Compiler Truffle Framework https://github.com/graalvm/ LLVM IR Interpreter LLVM IR Clang Flang 

Structure • LLVM IR interpreter • Data representation • Discussion: Undefined Behavior in C 8 

LLVM IR Interpreter 9 

Example Program 10 void processRequests () { int i = 0; do { processPacket (); i ++; } while (i < 10000) ; } C 

Example Program 10 void processRequests () { int i = 0; do { processPacket (); i ++; } while (i < 10000) ; } define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Clang C 

11 define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Implementation of Operations 

11 define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Executable Abstract Syntax Tree Implementation of Operations write %2 add read %i 1 

write %2 add read %i 1 Implementation of Operations 12 Executable Abstract Syntax Tree class LLVMI32AddNode extends LLVMExpressionNode { protected int executeI32(int left, int right) { return left + right; } } 

13 define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Implementation of Basic Blocks 

13 define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Executable Abstract Syntax Tree Implementation of Basic Blocks Block1 

Implementation of Control Flow Support 14 define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Unstructured control-flow is not representable as an AST interpreter 

Implementation of Control Flow Support 15 define void @processRequests () #0 { ; ( basic block 0) br label %1 ; :1 ( basic block 1) %i = phi i32 [ 0, %0 ], [ %2 , %1 ] call void @processPacket () %2 = add nsw i32 %i, 1 %3 = icmp slt i32 %2 , 10000 br i1 %3 , label %1 , label %4 ; :4 ( basic block 2) ret void } LLVM IR Block0 Block1 Block2 Basic Block Dispatch Node 1 2 -1 1 Bytecode-like interpreter 

Compilation • For frequently executed functions • Partial evaluation: inline through the interpreter (recursively) 16 Block0 Block1 Block2 Basic Block Dispatch Node 1 2 -1 1 

Block0 Block1 Block2 Basic Block Dispatch Node 1 2 -1 1 Compiler 17 int blockIndex = 0; block0: blockIndex = 1 %i.0 = 0 block1: while (true): processPacket() %2 = %i.0 + 1 %3 = %2 < 10000 if %3: blockIndex = 1 %i.0 = %2 continue; else: blockIndex = 2 block2: blockIndex = -1 return Partially evaluated interpreter 

Block0 Block1 Block2 Basic Block Dispatch Node 1 2 -1 1 Compiler 17 int blockIndex = 0; block0: blockIndex = 1 %i.0 = 0 block1: while (true): processPacket() %2 = %i.0 + 1 %3 = %2 < 10000 if %3: blockIndex = 1 %i.0 = %2 continue; else: blockIndex = 2 block2: blockIndex = -1 return Partially evaluated interpreter Graal further optimizes the partially evaluated interpreter 

Data Representations 18 

How to represent allocations 19 int *arr = malloc(sizeof(int) * 4); %1 = call i8* @malloc(i64 16) %2 = bitcast i8* %1 to i32* 

Tradeoffs Memory Safety Native Interoperability 20 int *arr = malloc(4 * sizeof(int)); arr[5] = … process(object) program.c lib.so Shared libraries without source code 

How to represent allocations 21 LLVM IR Interpreter Native Strategy LLVM IR Interpreter Safe Strategy Native Strategy 

How to represent allocations 22 + Native interoperability - No memory safety LLVM IR Interpreter Safe Strategy Native Strategy (Rigger et al. 2016) 

Native Sulong 23 Native Strategy %1 = call i8* @malloc(i64 16) %2 = bitcast i8* %1 to i32* unsafe.allocateMemory(16); sun.misc.Unsafe https://github.com/graalvm/sulong 0x8892042312 (Rigger et al. 2016) 

process(object) program.c lib.so Native Sulong: Native Interoperability 24 0x8892042312 Native Strategy (Rigger et al. 2016) 

Native Sulong: Memory Safety 25 int *arr = malloc(4 * sizeof(int)) arr[5] = … Native Strategy (Rigger et al. 2018) 

Native Sulong: Memory Safety 25 int *arr = malloc(4 * sizeof(int)) arr[5] = … Native Strategy (Rigger et al. 2018) 

How to represent allocations 26 + Memory safety - No interoperability LLVM IR Interpreter Safe Strategy Native Strategy (Rigger et al. 2018) 

Safe Sulong 27 Safe Strategy %1 = call i8* @malloc(i64 16) %2 = bitcast i8* %1 to i32* Address offset = 0 data UntypedAllocation size=16 (Rigger et al. 2018) 

Safe Sulong 28 %1 = call i8* @malloc(i64 16) %2 = bitcast i8* %1 to i32* Address offset = 0 data I32Array contents {0, 0, 0, 0} Safe Strategy Address offset = 0 data UntypedAllocation size=16 (Rigger et al. 2018) 

Prevent Out-Of-Bounds Accesses 29 int *arr = malloc(4 * sizeof(int)) arr[5] = … Safe Strategy Address offset = 5 data I32Array contents {0, 0, 0, 0} (Rigger et al. 2018) 

Prevent Out-Of-Bounds Accesses contents[5]  ArrayIndexOutOfBoundsException 29 int *arr = malloc(4 * sizeof(int)) arr[5] = … Safe Strategy Address offset = 5 data I32Array contents {0, 0, 0, 0} (Rigger et al. 2018) 

Address offset = 0 data I32Array contents {0, 0, 0, 0} process(object) program.c lib.so Native Interoperability 30 Safe Strategy (Rigger et al. 2018) 

Address offset = 0 data I32Array contents {0, 0, 0, 0} process(object) program.c lib.so Native Interoperability 30 Safe Strategy (Rigger et al. 2018) 

Native Interoperability 31 Ongoing work: safe execution of machine code on Safe Sulong Safe Strategy 

Native Interoperability 32 LLVM IR Interpreter LLVM IR lib.so Truffle Graal JVM x86 Interpreter x86 interpreter Safe Strategy (Lockhart et al. 2018) 

Native Interoperability 33 LLVM IR Interpreter LLVM IR LLVM IR Lifter lib.so Truffle Graal JVM Lifting Machine code to LLVM IR Safe Strategy 

How to represent allocations 34 Combine advantages LLVM IR Interpreter Safe Strategy Native Strategy Managed Strategy 

Semantic Holes in the C Standard 35 

write %2 add read %i 1 Simple Operations 36 Executable Abstract Syntax Tree The result of the binary operator is the sum of the operands. C11 standard 

write %2 add read %i 1 Simple Operations 36 Executable Abstract Syntax Tree class LLVMI32AddNode extends LLVMExpressionNode { @Specialization protected int executeI32(int left, int right) { return left + right; } } The result of the binary operator is the sum of the operands. C11 standard 

Defined Behavior in C 37 Implementing the semantics described in the standard is relatively straightforward C11 

Undefined Behavior 38 UB: … ranges from ignoring the situation completely with unpredictable results [to more sane behavior] int a = 1, b = INT_MAX; int val = a + b; printf("%d\n", val); UB 

What to do about UB? 39 Safe Strategy Increasingly powerful optimizations exploit UB 

What to do about UB? 39 Terminate the program Safe Strategy Increasingly powerful optimizations exploit UB 

What to do about UB? 40 write %2 add read %i 1 Executable Abstract Syntax Tree class LLVMI32AddNode extends LLVMExpressionNode { @Specialization protected int executeI32(int left, int right) { return Math.addExact(left, right); } } Safe Strategy 

Problem 41 Programmers often rely on Undefined Behavior being defined C11 

Problem 41 Programmers often rely on Undefined Behavior being defined C11 

What about the users? 42 6/9 of SPEC CINT 2006 benchmarks contain undefined integer operations alone (Dietz et al. 2018) 

For the kernel, we already really ignore some of the more idiotic C standard rules that introduce pointless undefined behavior: things like the strict aliasing rules are just insane, and the "overflow is u[nd]efined" is bad too. So we use -fno-strict-aliasing -fno-strict-overflow -fno-delete-null-pointer-checks to basically say "those optimizations are fundamentally stupid and wrong, and only encourage compilers to generate random code that doesn't actually match the source code". UB in the Linux Kernel 43 Linus 2017 

What to do about UB? 44 Terminate the program Continue execution Safe Strategy Many programs that we want to execute rely on UB having semantics 

What to do about UB? 45 write %2 add read %i 1 Executable Abstract Syntax Tree class LLVMI32AddNode extends LLVMExpressionNode { @Specialization protected int executeI32(int left, int right) { return left + right; } } Wraparound semantics for signed integer overflow Safe Strategy (Rigger et al. 2017) 

46 Is it always that simple? 

Undefined Behavior 47 void* memmove(void *dest , void const *src , size_t n ) { char *dp = dest; char const *sp = src; if (dp < sp) { while (n-- > 0) *dp++ = *sp++; } else { dp += n; sp += n; while (n-- > 0) *--dp = *--sp ; } return dest; } What’s the Undefined Behavior? 

Undefined Behavior 48 void* memmove(void *dest , void const *src , size_t n ) { char *dp = dest; char const *sp = src; if (dp < sp) { while (n-- > 0) *dp++ = *sp++; } else { dp += n; sp += n; while (n-- > 0) *--dp = *--sp ; } return dest; } UB Using < to compare pointers to two different objects is UB! 

Does code rely on it? 49 Response % of Respondants Yes 33% Yes, but it shouldn’t 12% No, but there might well be 29% No, that would be crazy 16% Don’t know 8% Q25 Can one do relational comparison (with <, >, <=, or >=) of two pointers to separately allocated objects (of compatible object types)? (Memarian et al. 2016) 

Integer Representation: Native Sulong 50 integer_rep(a) = a Native Strategy 0x8892042312 

Integer Representation: Safe Sulong 51 integer_rep(a) = a.offset satisfies the standard… but what about pointers to different objects? Safe Strategy Address offset = 0 data I32Array contents {0, 0, 0, 0} 

Integer Representation: Safe Sulong 52 Breaks antisymmetry as different objects might have the same hash code Safe Strategy integer_rep(a) = (long) System.identityHashCode(a.pointee) << 32 | offset; Address offset = 0 data I32Array contents {0, 0, 0, 0} 

Address offset = 0 data I32Array contents {0, 0, 0, 0} id = 1000 id = 1001 Integer Representation: Safe Sulong 53 integer_rep(a) = a.pointee.id Assign distinct IDs? Safe Strategy 

What to do about Undefined Behavior? 54 ~200 other instances of undefined behavior in addition to unspecified and implementation-defined behavior UB Unspecified- behavior Implementation- defined Behavior Locale-specific Behavior Defined Behavior 

What to do about Undefined Behavior? 55 Terminate the program Continue execution Discussion: How should Undefined Behavior be dealt with and who should deal with it? 

Is the C standard still up-to-date? 56 What should happen is a matter for implementations; it is impossible, in the language specification, to anticipate the details of the implementation scenario. – Stephen Kell (Kell 2017) 

Is the C standard still up-to-date? 57 I don’t even want to entertain the idea of a big family of Friendly C dialects, each making some niche audience happy– that is not really an improvement over our current situation. – John Regehr 

Summary 58 @RiggerManuel 

Bibliography • D. Lockhart, B. Ilbeyi and C. Batten, "Pydgin: generating fast instruction set simulators from simple architecture descriptions with meta-tracing JIT compilers," 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Philadelphia, PA, 2015, pp. 256-267. https://doi.org/10.1109/ISPASS.2015.7095811 • Kayvan Memarian, Justus Matthiesen, James Lingard, Kyndylan Nienhuis, David Chisnall, Robert N. M. Watson, and Peter Sewell. 2016. Into the depths of C: elaborating the de facto standards. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). ACM, New York, NY, USA, 1-15. DOI: https://doi.org/10.1145/2908080.2908081 • Manuel Rigger, Roland Schatz, Matthias Grimmer, and Hanspeter Mössenböck. 2017. Lenient Execution of C on a Java Virtual Machine: or: How I Learned to Stop Worrying and Run the Code. In Proceedings of the 14th International Conference on Managed Languages and Runtimes (ManLang 2017). ACM, New York, NY, USA, 35-47. DOI: https://doi.org/10.1145/3132190.3132204 • Manuel Rigger, Matthias Grimmer, Christian Wimmer, Thomas Würthinger, and Hanspeter Mössenböck. 2016. Bringing low-level languages to the JVM: efficient execution of LLVM IR on Truffle. In Proceedings of the 8th International Workshop on Virtual Machines and Intermediate Languages (VMIL 2016). ACM, New York, NY, USA, 6-15. DOI: https://doi.org/10.1145/2998415.2998416 • Manuel Rigger, Roland Schatz, René Mayrhofer, Matthias Grimmer, and Hanspeter Mössenböck. 2018. Sulong, and Thanks for All the Bugs: Finding Errors in C Programs by Abstracting from the Native Execution Model. In Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '18). ACM, New York, NY, USA, 377-391. DOI: https://doi.org/10.1145/3173162.3173174 • Stephen Kell. 2017. Some were meant for C: the endurance of an unmanageable language. In Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2017). ACM, New York, NY, USA, 229-245. DOI: https://doi.org/10.1145/3133850.3133867 • Thomas Würthinger, Christian Wimmer, Andreas Wöß, Lukas Stadler, Gilles Duboscq, Christian Humer, Gregor Richards, Doug Simon, and Mario Wolczko. 2013. One VM to rule them all. In Proceedings of the 2013 ACM international symposium on New ideas, new paradigms, and reflections on programming & software (Onward! 2013). ACM, New York, NY, USA, 187-204. DOI=http://dx.doi.org/10.1145/2509578.2509581 • Will Dietz, Peng Li, John Regehr, and Vikram Adve. 2012. Understanding integer overflow in C/C++. In Proceedings of the 34th International Conference on Software Engineering (ICSE '12). IEEE Press, Piscataway, NJ, USA, 760-770. 59