Tweet
Tweet

Slide 1

Slide 1 text

PyPy and The Art of Generating Virtual Machines Antonio Cuni Maciej Fijalkowski Merlinux GmbH PyCon UK 2008 - Birmingham September 13 2008 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 1 / 45

Slide 2

Slide 2 text

PyPy PyPy is both: a Python interpreter written in Python a framework for writing dynamic languages Today we will focus on the latter, former is discussed on another talk. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 2 / 45

Slide 3

Slide 3 text

Agenda quick overview & motivations the translation framework JIT generator (demo) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 3 / 45

Slide 4

Slide 4 text

Interpreters ...are good to implement dynamic languages: Easy to write Portable Flexible and easy to evolve, if written in high-level language (without low-level details) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 4 / 45

Slide 5

Slide 5 text

Folk Wisdom ...about interpreters for Dynamic Languages: There are unavoidable tradeoffs between flexibility, maintainability, and speed Fast, Maintainable, Flexible -- pick one A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 5 / 45

Slide 6

Slide 6 text

What this means in Practice Current popular open source dynamic language implementations: are relatively slow are not very flexible are harder to maintain than we would like them to be A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 6 / 45

Slide 7

Slide 7 text

Not very flexible Low-level decisions permeate the entire code base. Not ideal to experiment - cannot simply plug-in a new garbage collector, memory model, or threading model Early decisions come back to haunt you. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 7 / 45

Slide 8

Slide 8 text

Hard to maintain because they are traditionally written in low-level languages: the community generates experts in the dynamic language but requires experts in C or C++ for its own maintenance every time a new VM is needed, the language’s community forks (CPython - Jython - IronPython) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 8 / 45

Slide 9

Slide 9 text

The PyPy Project We built enough infrastructure such that: speed is regained features requiring low-level manipulations are (re-)added as aspects interpreters are kept simple and uncluttered Targets as different as C and the industry OO VMs (JVM, CLR) are supported. A special aspect: Generating JIT compilers A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 9 / 45

Slide 10

Slide 10 text

PyPy as a project We operate both as an open source with production usage aspirations and research project. We focus on the whole system. We want the tool-chain itself to be as simple as possible (but not simpler). Some of what we do is relatively straight-forward, some is challenging (generating dynamic compilers!). A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 10 / 45

Slide 11

Slide 11 text

PyPy Approach A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 11 / 45

Slide 12

Slide 12 text

Going from interpreters to VMs In PyPy interpreters are written in RPython: A subset of Python amenable to static analysis Still fully garbage collected Rich built-in types RPython is still close to Python. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 12 / 45

Slide 13

Slide 13 text

Translation details (1) First, load and initialize RPython code inside a normal Python VM RPython translation starts from the resulting“live” bytecode Unified“intermediate code”representation: a forest of Control Flow Graphs A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 13 / 45

Slide 14

Slide 14 text

Translation details (2) PyPy uses abstract interpretation extensively: to construct Flow Graphs for type inference to gather info for some optimisations for Partial Evaluation in the generated Dynamic Compilers... also uses Flow Graph transformation and rewriting. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 14 / 45

Slide 15

Slide 15 text

Type Systems (1) We model the different targets through different type systems: LL (low-level C-like targets): data and function pointers, structures, arrays... OO (object oriented targets): classes and instances with inheritance and dispatching A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 15 / 45

Slide 16

Slide 16 text

Type Systems (2) Translation: starts from RPython Flow Graphs turns them into LL Flow Graphs or OO Flow Graphs the flowgraphs are transformed in various ways then they are sent to the backends. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 16 / 45

Slide 17

Slide 17 text

Translation aspects (1) The interpreters in RPython are free of low-level details (as required to target platforms as different as Posix/C and the JVM/.NET). Advanced features related to execution should not need wide-spread changes to the interpreters Instead, the interpreters should use support from the translation framework A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 17 / 45

Slide 18

Slide 18 text

Translation aspects (2) Examples: GC and memory management memory layout stack inspection and manipulation unboxed integers as tagged pointers A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 18 / 45

Slide 19

Slide 19 text

Implementation Translation aspects are implemented as transformation of low-level graphs Calls to library/helper code can be inserted too The helper code is also written in RPython and analyzed and translated A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 19 / 45

Slide 20

Slide 20 text

GC Framework The LL Type System is extended with allocation and address manipulation primitives, used to express GC in RPython directly. GCs are linked by substituting memory allocation operations with calls into them Transformation inserts bookkeeping code, e.g. to keep track of roots Inline fast paths of allocation and barriers A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 20 / 45

Slide 21

Slide 21 text

JIT motivation Flexibility vs. Performance: Interpreters are easy to write and evolve For high performance, dynamic compilation is required A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 21 / 45

Slide 22

Slide 22 text

Traditional JIT compilers Huge resource investment The richer the semantics, the harder to write Poor encoding of language semantics Hard to evolve Need for novel approaches! A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 22 / 45

Slide 23

Slide 23 text

PyPy Architecture A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 23 / 45

Slide 24

Slide 24 text

Basics Use partial evaluation techniques to generate a dynamic compiler from an interpreter Inspiration: Psyco Our translation tool-chain was designed for trying this A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 24 / 45

Slide 25

Slide 25 text

Futamura Partial evalution of computation process - an approach to a compiler-compiler, 1971 Generating compilers from interpreters with automatic specialization Relatively little practical impact so far A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 25 / 45

Slide 26

Slide 26 text

General idea Partial evaluation (PE): Assume the Python bytecode to be constant, and constant-propagate it into the Python interpreter. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 26 / 45

Slide 27

Slide 27 text

PE for dummies Example def f(x, y): x2 = x * x y2 = y * y return x2 + y2 case x=3 def f_3(y): y2 = y * y return 9 + y2 case x=10 def f_10(y): y2 = y * y return 100 + y2 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 27 / 45

Slide 28

Slide 28 text

PE for dummies Example def f(x, y): x2 = x * x y2 = y * y return x2 + y2 case x=3 def f_3(y): y2 = y * y return 9 + y2 case x=10 def f_10(y): y2 = y * y return 100 + y2 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 27 / 45

Slide 29

Slide 29 text

PE for dummies Example def f(x, y): x2 = x * x y2 = y * y return x2 + y2 case x=3 def f_3(y): y2 = y * y return 9 + y2 case x=10 def f_10(y): y2 = y * y return 100 + y2 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 27 / 45

Slide 30

Slide 30 text

Challenges A shortcoming of PE is that in many cases not much can be really assumed constant at compile-time: poor results Effective dynamic compilation requires feedback of runtime information into compile-time For a dynamic language: types are a primary example A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 28 / 45

Slide 31

Slide 31 text

Solution: Promotion Enhance PE with the ability to“promote”run-time values to compile-time Leverage the dynamic setting A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 29 / 45

Slide 32

Slide 32 text

Overall ingredients The pieces to enable effective dynamic compiler generation in PyPy: a few hints in the Python interpreter to guide the JIT generator promotion lazy allocation of objects (only on escape) use CPU stack and registers for the contents of the Python frame A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 30 / 45

Slide 33

Slide 33 text

Language-agnostic The dynamic generation process and primitives are language-agnostic. The language implementations should be able to evolve up to maintaining the hints. By construction all interpreter/language features are supported A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 31 / 45

Slide 34

Slide 34 text

pypy-c-jit PyPy 1.0 contains both the dynamic compiler generator and the start of its application to PyPy’s Python intepreter. JIT refactoring in-progress. included are backends for IA32 and PPC experimental/incomplete CLI backend integer arithmetic operations are optimized for these, we are in the speed range of gcc -O0 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 32 / 45

Slide 35

Slide 35 text

EXTRA MATERIAL More about the JIT Generation: The Rainbow interpreter Virtuals and Promotion A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 33 / 45

Slide 36

Slide 36 text

Execution steps Translation time pypy-c-jit is translated into an executable the JIT compiler is automatically generated Compile-time: the JIT compiler runs Runtime: the JIT compiled code runs Compile-time and runtime are intermixed A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 34 / 45

Slide 37

Slide 37 text

The Rainbow interpreter A special interpreter whose goal is to produce executable code Written in RPython Guided by a binding time analysis ( “color”of the graphs) Green operations: executed at compile-time Red operations: produce code that executes the operation at runtime A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 35 / 45

Slide 38

Slide 38 text

Rainbow architecture Translation time Low-level flowgraphs are produced The hint-annotator colors the variables The rainbow codewriter translates flowgraphs into rainbow bytecode Compile-time The rainbow interpreter executes the bytecode As a result, it produces executable code Runtime The produced code is executed A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 36 / 45

Slide 39

Slide 39 text

Rainbow architecture Translation time Low-level flowgraphs are produced The hint-annotator colors the variables The rainbow codewriter translates flowgraphs into rainbow bytecode Compile-time The rainbow interpreter executes the bytecode As a result, it produces executable code Runtime The produced code is executed A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 36 / 45

Slide 40

Slide 40 text

Rainbow architecture Translation time Low-level flowgraphs are produced The hint-annotator colors the variables The rainbow codewriter translates flowgraphs into rainbow bytecode Compile-time The rainbow interpreter executes the bytecode As a result, it produces executable code Runtime The produced code is executed A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 36 / 45

Slide 41

Slide 41 text

Coloring Green: compile-time value Red: runtime value The hints give constraints from which the colors of all values are derived We reuse the type inference framework to propagate colors A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 37 / 45

Slide 42

Slide 42 text

Partial Evaluation with Colors Green operations: unchanged, executed at compile-time Red operations: converted into corresponding code emitting code Example def f(x, y): x2 = x * x y2 = y * y return x2 + y2 case x=10 def f_10(y): y2 = y * y return 100 + y2 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 38 / 45

Slide 43

Slide 43 text

Partial Evaluation with Colors Green operations: unchanged, executed at compile-time Red operations: converted into corresponding code emitting code Example def f(x, y): x2 = x * x y2 = y * y return x2 + y2 case x=10 def f_10(y): y2 = y * y return 100 + y2 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 38 / 45

Slide 44

Slide 44 text

Partial Evaluation with Colors Green operations: unchanged, executed at compile-time Red operations: converted into corresponding code emitting code Example def f(x, y): x2 = x * x y2 = y * y return x2 + y2 case x=10 def f_10(y): y2 = y * y return 100 + y2 A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 38 / 45

Slide 45

Slide 45 text

Partial Evaluate Control Flow red split points: schedule multiple compilation states merge points: merge logic to reuse code for equivalent states Example if x: print "x is true" if y: print "y is true" case y != 0 if x: print "x is true" print "y is true" A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 39 / 45

Slide 46

Slide 46 text

Partial Evaluate Control Flow red split points: schedule multiple compilation states merge points: merge logic to reuse code for equivalent states Example if x: print "x is true" if y: print "y is true" case y != 0 if x: print "x is true" print "y is true" A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 39 / 45

Slide 47

Slide 47 text

Partial Evaluate Control Flow red split points: schedule multiple compilation states merge points: merge logic to reuse code for equivalent states Example if x: print "x is true" if y: print "y is true" case y != 0 if x: print "x is true" print "y is true" A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 39 / 45

Slide 48

Slide 48 text

Promotion Promotion is implemented generating a switch that grows to cover the seen runtime values First compilation stops at a promotion point and generates a switch with only a default case. The default will call back into the compiler with runtime values. On callback the compiler adds one more case to the switch and generates more code assuming the received value. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 40 / 45

Slide 49

Slide 49 text

Promotion (example) Example def f(x, y): x1 = hint(x, promote=True) return x1*x1 + y*y original def f_(x, y): switch x: pass default: compile_more(x) augmented def f_(x, y): switch x: case 3: return 9 + y*y default: compile_more(x) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 41 / 45

Slide 50

Slide 50 text

Promotion (example) Example def f(x, y): x1 = hint(x, promote=True) return x1*x1 + y*y original def f_(x, y): switch x: pass default: compile_more(x) augmented def f_(x, y): switch x: case 3: return 9 + y*y default: compile_more(x) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 41 / 45

Slide 51

Slide 51 text

Promotion (example) Example def f(x, y): x1 = hint(x, promote=True) return x1*x1 + y*y original def f_(x, y): switch x: pass default: compile_more(x) augmented def f_(x, y): switch x: case 3: return 9 + y*y default: compile_more(x) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 41 / 45

Slide 52

Slide 52 text

Virtuals + Promotion Example from PyPy (simplified!) def add_python_objects(obj1, obj2): obj1cls = hint(obj1.__class__, promote=True) obj2cls = hint(obj2.__class__, promote=True) if obj1cls is IntObject and obj2cls is IntObject: x = obj1.intval y = obj2.intval z = x + y return IntObject(intval=z) A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 42 / 45

Slide 53

Slide 53 text

Conclusion (JIT) Effective dynamic compiler generation flexibility and ease of evolution orthogonal to the performance question. Languages implemented as understandable interpreters. PyPy proves this a viable approach worth of further exploration. A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 43 / 45

Slide 54

Slide 54 text

Open Issues inlining control promotion switch explosion fallbacks jit only the hot-spots more hints needed in PyPy’s Python JIT backends for CLI/JVM A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 44 / 45

Slide 55

Slide 55 text

Virtualizable Frames frames need to live in the heap (tracebacks ...) and be introspectable jit code wants local variables to live in registers and on the stack => mark the frame class as“virtualizable” jit code uses lazy allocation and stores some contents (local variables...) in register and stack outside world access gets intercepted to be able to force lazy virtual data into the heap A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 45 / 45

Slide 56

Slide 56 text

Contact / Q&A Antonio Cuni, Maciej Fijalkowski at http://merlinux.eu PyPy: http://codespeak.net/pypy Blog: http://morepypy.blogspot.com A. Cuni, M. Fijalkowski (PyCon UK 2008) PyPy and The Art of Generating VMs September 13 2008 46 / 45