Accelerating Iterators in Optimizing AST Interpreters

Transcript

1. Accelerating Iterators in Optimizing AST Interpreters Wei Zhang, Per Larsen, Stefan Brunthaler, Michael Franz UC Irvine 1
2. ZipPy • Python 3 implementation on JVM • Built using Truffle framework • Support the majority of the language features • https://bitbucket.org/ssllab/zippy 2
3. ZipPy on Truffle • Truffle optimizes a common base of dynamic languages • Simplifies type specialization • Offers a JIT compiler • It allows us to look at other things • Focus on features unique to Python 3
4. Python Iterators • Iterators are ubiquitous • Implement iterator protocol • Built-in iterators • User-defined iterators • Generators are user-defined iterators using special control-flow construct (yield) • Generators exist in other languages too, like C#, PHP,… 4 Client Iterator __next__() next value advance
5. Python Generators 5 def  fib(n):      a,  b  =  0,  1      for  i  in  range(n):          a,  b  =  b,  a+b          yield  a   #  1,  1,  2,  3,  5,  8.. l  =  []   for  i  in  fib(10):      if  i  %  2  ==  0:          l.append(i)   #  [2,  8,  ..] generator  function consumer  loop
6. Python Generators 6 • We surveyed the use of Generators in Python programs • 90% of the top 50 Python projects on PyPI and GitHub use generators • Given its popularity the performance of generators are critical to Python programs Django LXML Jinja2 Flask pip Fabric Pandas Requests Reddit
7. 7 Python generators are slow…
8. Generator Execution 8 1.The implicit call to __next__ and resume execution l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 generator(body consumer(loop
9. Generator Execution 9 1.The implicit call to __next__ and resume execution 2.Evaluate the next value in generator body l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 2 generator(body consumer(loop
10. Generator Execution 10 1.The implicit call to __next__ and resume execution 2.Evaluate the next value in generator body 3.Suspend execution and return to the caller l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 2 3 generator(body consumer(loop
11. Generator Execution 11 1.The implicit call to __next__ and resume execution 2.Evaluate the next value in generator body 3.Suspend execution and return to the caller 4.Consume the generated value l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a 1 2 3 4 generator(body consumer(loop
12. Generator Overheads 12 • Only step 2 and 4 do the real work • Python call is expensive • Resume and suspend add additional costs and prevents frame optimizations l"="[] for$i$in$fib(10): ""if"i"%"2"=="0: """"l.append(i) def$fib(n): ""resume&to&last&yield ""a,"b"="0,"1 ""for$i$in$range(n): """"a,"b"="b,"a+b """"yield$a """"suspend&execution 1 2 3 4 generator(body consumer(loop
13. Naive Inlining 13 • Desugar the consumer loop and inline __next__ directly • The suspend and resume handling still persists l"="[] g"="fib(10) while&True: ""resume&to&last&yield ""a,"b"="0,"1 ""for&i&in&range(n): """"a,"b"="b,"a+b """"yield&a """"suspend&execution ""i"="a ""if"i"%"2"=="0: """"l.append(i) except"StopIter: generator(body consumer(loop 0: n 1: a 2: b 3: i generator(frame 0: l 1: i caller(frame
14. Generator Peeling 14 • Specialize the loop over generator at runtime • Merge yield with consumer loop body
15. Generator Peeling 15 l"="[] n"="10 a,"b"="0,"1 for$i$in$range(n): ""a,"b"="b,"a+b ""i"="a ""if"i"%"2"=="0: """"l.append(i) 2 3 4 generator(body loop(body 1 • Specialize the loop over generator at runtime • Remove suspend and resume handling
16. Generator Peeling 16 l"="[] n"="10 a,"b"="0,"1 for$i$in$range(n): ""a,"b"="b,"a+b ""i"="a ""if"i"%"2"=="0: """"l.append(i) 2 3 4 generator(body loop(body 1 • Frames can be optimized during compilation 0: n 1: a 2: b 3: i generator(frame 0: l 1: i caller(frame
17. 17 Before ForNode YieldNode
18. 18 After PeeledLoopNode FrameTransferNode
19. The End Result • Caller frame and generator frame can be optimized • Peeling inlines the call to __next__ • No suspend and resume handling • AST level transformation, independent from compilation • More details in the paper! 19
20. Speedups of Generator Peeling Measuring peak performance of ZipPy with and without Generator Peeling 20 0 8 15 23 30 nqueens euler11 euler31 eratos lyndon partitions pymaging python-graph simplejson sympy whoosh geomean 3.58 2.79 1.31 3.67 1.79 2.76 4.32 22.69 1.14 2.82 13.19 4.53
21. The Performance of ZipPy Measuring peak performance of ZipPy with Generator Peeling 21 0 1 10 100 1000 nqueens euler11 euler31 eratos lyndon partitions pymaging python-graph simplejson sympy whoosh geomean 20.59 56.53 2.37 14.58 3.16 95.96 40.29 162.88 3.32 13.09 57.43 29.05 3.58 2.79 1.31 3.67 1.79 2.76 4.32 22.69 1.14 2.82 13.19 4.53 11 22 7 12 3 65 25 24 1 8 6 12 1.16 1.39 0.71 1.23 0.54 1.1 1.72 2.37 1.68 0.64 0.75 2.14 1 1 1 1 1 1 1 1 1 1 1 1 CPython 3.4 Jython 2.7 PyPy 2.3 ZipPy Baseline ZipPy + Peeling
22. In Summary • We present a dynamic program transformation that optimizes generators for optimizing AST interpreters • As a result, programmers are free to enjoy generators’ upsides 22
23. Thank You! 23