Improving TruffleRuby’s Startup Time with the SubstrateVM

Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
| Improving TruffleRuby’s Startup Time with the SubstrateVM Kevin Menard Principal Member of Technical Staff VM Research Group, Oracle Labs September 20, 2017

| Safe Harbor Statement The following is intended to provide some insight into a line of research in Oracle Labs. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Oracle reserves the right to alter its development plans and practices at any time, and the development, release, and timing of any features or functionality described in connection with any Oracle product or service remains at the sole discretion of Oracle. Any views expressed in this presentation are my own and do not necessarily reflect the views of Oracle. 3

| Improving TruffleRuby’s Startup Time with the SubstrateVM 4

| What is TruffleRuby? 5

| The TruffleRuby logo is copyright © 2017 Talkdesk, Inc., licensed under Creative Commons Attribiton 4.0 International (CC BY 4.0) The JRuby logo is copyright © 2011 Tony Price, licensed under the terms of Creative Commons Attribution-NoDerivs 3.0 Unported (CC BY-ND 3.0) The Rubinius logo is copyright © 2011 Shane Becker, licensed under the terms of Creative Commons Attribution-NoDerivatives 4.0 International (CC BY-ND 4.0) The Ruby logo is copyright © 2006 Yukihiro Matsumoto, licensed under the terms of Creative Commons Attribution-ShareAlike 2.5 (CC BY-SA 2.5)

| 96% Ruby core library specs 99% Ruby language specs 99% ActiveSupport tests

| What is Truffle? 8

| Truffle • Language toolkit for generating simple and fast runtimes • Generates extremely fast machine code via partial evaluation with Graal • Provides tooling for many common VM components: – Debugger, instrumentation, cache control, JIT interface, etc. • Polyglot is a first-class feature – Truffle languages can call in and out of each other

| TruffleRuby Milestones in the Last Year • JRuby+Truffle -> TruffleRuby • Running MRI C extensions – LLVM bitcode interpreter – Sulong – “Ruby’s C Extension Problem and How We're Solving It” – Chris Seaton • Java interop • Improved native calls with Truffle Native Function Interface (NFI) 10

| But, we have a problem… 11

| Start-up 12 Warm-up Steady t

| 13 t

| 14 t

| Startup Time 15

| 16 Time for Hello World 38ms 1,442ms 1,732ms

| The Ruby Spec Suite • Tests for the Ruby language itself – Language, core, standard library, C API, and others • RSpec-like runner, called MSpec • Language specs – 2,130 examples – 3,813 expectations 17

| 18 Time for Language Specs 1.5s 33.2s 47.4s

| Peak Performance 19

| Optcarrot • NES emulator written in Ruby • Used to measure Ruby 3x3 progress • Scored in frames per second (fps) – Real NES runs at 60 fps • Available at https://github.com/mame/optcarrot 20

| 21 Optcarrot 23.1 fps 45.7 fps 197.4 fps

| A balancing act: development VS production 22

| Traditional options • Always use MRI – Fast start-up, slower peak performance • Always use alternative Ruby – Fast peak performance, slow start-up time • MRI for development, alternative Ruby for production – Risky: likely production issues aren’t caught in development – Doable?: different Ruby engines can confuse tools like Bundler 23

| Can we do better? 24

| Introducing the SubstrateVM 25

| 26 TruffleRuby JVM GraalVM

| 27 TruffleRuby JVM GraalVM SubstrateVM

| 28 JDK Extra JARs TruffleRuby SubstrateVM Ahead-of-time Compiler TruffleRuby Binary SubstrateVM

| The Difference Between the JVM and SVM Models 29 public int add(int, int); Code: 0: iload_1 1: iload_2 2: invokestatic j/l/Math.addExact:(II)I 5: ireturn 0x0000000100c3b734 48897C2420 movq %rdi, var_8+0x28(%rsp) 0x0000000100c3b739 8974241C movl %esi, var_C+0x28(%rsp) 0x0000000100c3b73d 89542418 movl %edx, var_10+0x28(%rsp) 0x0000000100c3b741 8BFE movl %esi, %edi 0x0000000100c3b743 8BF2 movl %edx, %esi 0x0000000100c3b745 6690 nop 0x0000000100c3b747 E8344863FF call j.l.Math.addExact(int,int)int 0x0000000100c3b74c 90 nop 0x0000000100c3b74d 4183BF9000000000 cmpl $0x0, 0x90(%r15) 0x0000000100c3b755 0F8505000000 jne loc_100c3b760 0x0000000100c3b75b 4883C428 addq $0x28, %rsp 0x0000000100c3b75f C3 ret For the JVM, we ship Java bytecode, which is interpreted or dynamically compiled by the JVM: Using the SVM, we ship real machine code directly. There is no interpreter, but we also optionally include a dynamic compiler:

| 30 Java Main C1 C2 C3 C4 C5 C6 C9 C8 C7 CN SVM AOT Compiler Static Analysis: Find all reachable classes & methods Load each class and execute any static assignments and initializers Conservatively discard any class or method that is unused More details: “Polyglot Native: Java, Scala, Kotlin, and JVM Languages” – Christian Wimmer

| Taking Advantage of Ahead-of-time Compilation • Pre-parse core Ruby files and store constructed AST in the binary • Pre-load encoding tables • Pre-load transcoding mappings • Pre-construct common strings 31

| SVM Limitations for Language Implementors • Can’t dynamically load Java classes • Extra care needed if using “Unsafe” • Required to mark megamorphic calls with @TruffleBoundary – E.g., a “toString” call on an unknown receiver type could compile in the whole JDK 32

| Startup Time Revisited 33

| 34 Time for Hello World 38ms 1,442ms 1,732ms

| 35 Time for Hello World 38ms 1,442ms 1,732ms 100ms

| 36 Time for Language Specs 1.5s 33.2s 47.4s

| 37 Time for Language Specs 1.5s 33.2s 47.4s 6.8s

| Did we sacrifice peak performance? 38

| 39 Optcarrot 23.1 fps 45.7 fps 197.4 fps

| 40 Optcarrot 23.1 fps 45.7 fps 197.4 fps 168.7 fps

| Why the performance hit? 41

| 42 send(*DISPATCH[@opcode]) Creates an array and is run in a tight loop. SVM doesn’t optimize this as well as Graal on the JVM… yet. See also: “Fast Metaprogramming with Truffle” – Kevin Menard

| Summary • TruffleRuby’s startup time problem is fixed! – But always room for improvement • The SubstrateVM is publicly available as part of GraalVM – No longer vaporware! • Validates the Truffle AST interpreter approach to runtime construction 43

| Future Work • SubstrateVM – Compressed oops – Improved array handling • TruffleRuby – Precompute more during native image building – Embed the standard library in the image – Evaluate & reduce memory usage – Reduce overhead of native calls 44

| Building the TruffleRuby/SVM Binary • Download GraalVM – http://www.oracle.com/technetwork/oracle-labs/program- languages/downloads/index.html – Search “GraalVM OTN” • From the GraalVM root, run: – $ ./bin/native-image --ruby • Run the binary with: – $ ./ruby 45

| Benchmark Environment • Optcarrot – Intel Xeon E5-2699 v3 @ 2.30 GHz (2 CPUs, 18 cores each + hyperthreading) • TurboBoost disabled – 384 GB RAM – Oracle Enterprise Linux 6.8 with 4.1.12-37.5.1.el6uek kernel • Startup time – Intel i7-4390K @ 3.40 GHz (1 CPU, 6 cores + hyperthreading) – 48 GB RAM – Ubuntu 17.04 with 4.10.0-33-generic kernel 46

| Related Talks • “Polyglot Native: Java, Scala, Kotlin, and JVM Languages” – Christian Wimmer – JVM Language Summit 2017 • “Ruby’s C Extension Problem and How We're Solving It” – Chris Seaton – RubyConf 2016 • “Fast Metaprogramming with Truffle” – Kevin Menard – RubyKaigi 2015 47

| 48

| Team 49 Oracle Florian Angerer Danilo Ansaloni Stefan Anzinger Martin Balin Cosmin Basca Daniele Bonetta Dušan Bálek Matthias Brantner Lucas Braun Petr Chalupa Jürgen Christ Laurent Daynès Gilles Duboscq Svatopluk Dědic Martin Entlicher Pit Fender Francois Farquet Brandon Fish Matthias Grimmer Christian Häubl Peter Hofer Bastian Hossbach Christian Humer Tomáš Hůrka Mick Jordan Oracle (continued) Vojin Jovanovic Anantha Kandukuri Harshad Kasture Cansu Kaynak Peter Kessler Duncan MacGregor Jiří Maršík Kevin Menard Miloslav Metelka Tomáš Myšík Petr Pišl Oleg Pliss Jakub Podlešák Aleksandar Prokopec Tom Rodriguez Roland Schatz Benjamin Schlegel Chris Seaton Jiří Sedláček Doug Simon Štěpán Šindelář Zbyněk Šlajchrt Boris Spasojevic Lukas Stadler Codrut Stancu JKU Linz Hanspeter Mössenböck Benoit Daloze Josef Eisl Thomas Feichtinger Josef Haider Christian Huber David Leopoldseder Stefan Marr Manuel Rigger Stefan Rumzucker Bernhard Urban TU Berlin: Volker Markl Andreas Kunft Jens Meiners Tilmann Rabl University of Edinburgh Christophe Dubach Juan José Fumero Alfonso Ranjeet Singh Toomas Remmelg LaBRI Floréal Morandat University of California, Irvine Michael Franz Yeoul Na Mohaned Qunaibit Gulfem Savrun Yeniceri Wei Zhang Purdue University Jan Vitek Tomas Kalibera Petr Maj Lei Zhao T. U. Dortmund Peter Marwedel Helena Kotthaus Ingo Korb University of California, Davis Duncan Temple Lang Nicholas Ulle University of Lugano, Switzerland Walter Binder Sun Haiyang Oracle Interns Brian Belleville Ondrej Douda Juan Fumero Miguel Garcia Hugo Guiroux Shams Imam Berkin Ilbeyi Hugo Kapp Alexey Karyakin Stephen Kell Andreas Kunft Volker Lanting Gero Leinemann Julian Lettner Joe Nash Tristan Overney Aleksandar Pejovic David Piorkowski Philipp Riedmann Gregor Richards Robert Seilbeck Rifat Shariyar Oracle Alumni Erik Eckstein Michael Haupt Christos Kotselidis David Leibs Adam Welc Till Westmann Oracle (continued) Jan Štola Tomáš Stupka Farhan Tauheed Jaroslav Tulach Alexander Ulrich Michael Van De Vanter Aleksandar Vitorovic Christian Wimmer Christian Wirth Paul Wögerer Mario Wolczko Andreas Wöß Thomas Würthinger Tomáš Zezula Yudi Zheng Red Hat Andrew Dinn Andrew Haley Intel Michael Berg Twitter Chris Thalinger

| [email protected] @nirvdrum / @TruffleRuby https://github.com/graalvm/truffleruby (or just search for ‘truffleruby’)

| 51

Improving TruffleRuby’s Startup Time with the S...

Improving TruffleRuby’s Startup Time with the SubstrateVM

More Decks by Kevin Menard

Other Decks in Research

Featured

Transcript