Accelerating Iterators in Optimizing AST Interpreters

0660a45bc590da31068780de9b34a0df?s=47 Wei Zhang
October 24, 2014

Accelerating Iterators in Optimizing AST Interpreters

0660a45bc590da31068780de9b34a0df?s=128

Wei Zhang

October 24, 2014
Tweet

Transcript

  1. 2.

    ZipPy • Python 3 implementation on JVM • Built using

    Truffle framework • Support the majority of the language features • https://bitbucket.org/ssllab/zippy 2
  2. 3.

    ZipPy on Truffle • Truffle optimizes a common base of

    dynamic languages • Simplifies type specialization • Offers a JIT compiler • It allows us to look at other things • Focus on features unique to Python 3
  3. 4.

    Python Iterators • Iterators are ubiquitous • Implement iterator protocol

    • Built-in iterators • User-defined iterators • Generators are user-defined iterators using special control-flow construct (yield) • Generators exist in other languages too, like C#, PHP,… 4 Client Iterator __next__() next value advance
  4. 5.

    Python Generators 5 def  fib(n):      a,  b  =

     0,  1      for  i  in  range(n):          a,  b  =  b,  a+b          yield  a   #  1,  1,  2,  3,  5,  8.. l  =  []   for  i  in  fib(10):      if  i  %  2  ==  0:          l.append(i)   #  [2,  8,  ..] generator  function consumer  loop
  5. 6.

    Python Generators 6 • We surveyed the use of Generators

    in Python programs • 90% of the top 50 Python projects on PyPI and GitHub use generators • Given its popularity the performance of generators are critical to Python programs Django LXML Jinja2 Flask pip Fabric Pandas Requests Reddit
  6. 8.

    Generator Execution 8 1.The implicit call to __next__ and resume

    execution l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 generator(body consumer(loop
  7. 9.

    Generator Execution 9 1.The implicit call to __next__ and resume

    execution 2.Evaluate the next value in generator body l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 2 generator(body consumer(loop
  8. 10.

    Generator Execution 10 1.The implicit call to __next__ and resume

    execution 2.Evaluate the next value in generator body 3.Suspend execution and return to the caller l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 2 3 generator(body consumer(loop
  9. 11.

    Generator Execution 11 1.The implicit call to __next__ and resume

    execution 2.Evaluate the next value in generator body 3.Suspend execution and return to the caller 4.Consume the generated value l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 2 3 4 generator(body consumer(loop
  10. 12.

    Generator Overheads 12 • Only step 2 and 4 do

    the real work • Python call is expensive • Resume and suspend add additional costs and prevents frame optimizations l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""resume&to&last&yield ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a """"suspend&execution 1 2 3 4 generator(body consumer(loop
  11. 13.

    Naive Inlining 13 • Desugar the consumer loop and inline

    __next__ directly • The suspend and resume handling still persists l"="[] g"="fib(10) while&True: ""resume&to&last&yield ""a,"b"="0,"1 ""for&i&in&range(n): """"a,"b"="b,"a+b """"yield&a """"suspend&execution ""i"="a ""if"i"%"2"=="0: """"l.append(i) except"StopIter: generator(body consumer(loop 0: n 1: a 2: b 3: i generator(frame 0: l 1: i caller(frame
  12. 14.

    Generator Peeling 14 • Specialize the loop over generator at

    runtime • Merge yield with consumer loop body
  13. 15.

    Generator Peeling 15 l"="[] n"="10 a,"b"="0,"1 for$i$in$range(n): ""a,"b"="b,"a+b ""i"="a ""if"i"%"2"=="0:

    """"l.append(i) 2 3 4 generator(body loop(body 1 • Specialize the loop over generator at runtime • Remove suspend and resume handling
  14. 16.

    Generator Peeling 16 l"="[] n"="10 a,"b"="0,"1 for$i$in$range(n): ""a,"b"="b,"a+b ""i"="a ""if"i"%"2"=="0:

    """"l.append(i) 2 3 4 generator(body loop(body 1 • Frames can be optimized during compilation 0: n 1: a 2: b 3: i generator(frame 0: l 1: i caller(frame
  15. 19.

    The End Result • Caller frame and generator frame can

    be optimized • Peeling inlines the call to __next__ • No suspend and resume handling • AST level transformation, independent from compilation • More details in the paper! 19
  16. 20.

    Speedups of Generator Peeling Measuring peak performance of ZipPy with

    and without Generator Peeling 20 0 8 15 23 30 nqueens euler11 euler31 eratos lyndon partitions pymaging python-graph simplejson sympy whoosh geomean 3.58 2.79 1.31 3.67 1.79 2.76 4.32 22.69 1.14 2.82 13.19 4.53
  17. 21.

    The Performance of ZipPy Measuring peak performance of ZipPy with

    Generator Peeling 21 0 1 10 100 1000 nqueens euler11 euler31 eratos lyndon partitions pymaging python-graph simplejson sympy whoosh geomean 20.59 56.53 2.37 14.58 3.16 95.96 40.29 162.88 3.32 13.09 57.43 29.05 3.58 2.79 1.31 3.67 1.79 2.76 4.32 22.69 1.14 2.82 13.19 4.53 11 22 7 12 3 65 25 24 1 8 6 12 1.16 1.39 0.71 1.23 0.54 1.1 1.72 2.37 1.68 0.64 0.75 2.14 1 1 1 1 1 1 1 1 1 1 1 1 CPython 3.4 Jython 2.7 PyPy 2.3 ZipPy Baseline ZipPy + Peeling
  18. 22.

    In Summary • We present a dynamic program transformation that

    optimizes generators for optimizing AST interpreters • As a result, programmers are free to enjoy generators’ upsides 22