PyPy Status Update

Transcript

1. PyPy Status Update Antonio Cuni PyCon Otto April 07 2017

   antocuni (PyCon Otto) PyPy Status Update April 07 2017 1 / 19

2. About me PyPy core dev pdb++, fancycompleter, capnpy, ... Consultant,

   trainer http://antocuni.eu antocuni (PyCon Otto) PyPy Status Update April 07 2017 2 / 19

3. What is PyPy Alternative, fast Python implementation Performance: JIT compiler,

   advanced GC PyPy 5.7.1 x86, ARM, s390x, PPC http://pypy.org antocuni (PyCon Otto) PyPy Status Update April 07 2017 3 / 19

4. Speed 7.4 times faster than CPython http://speed.pypy.org antocuni (PyCon Otto)

   PyPy Status Update April 07 2017 4 / 19

5. Usual questions Is it stable/mature? Can I use it in

   production? Does package X work? What about Python 3? antocuni (PyCon Otto) PyPy Status Update April 07 2017 5 / 19

6. PyPy in the real world (1) High frequency trading platform

   for sports bets low latency is a must PyPy used in production since 2012 ~100 PyPy processes running 24/7 up to 10x speedups after careful tuning and optimizing for PyPy antocuni (PyCon Otto) PyPy Status Update April 07 2017 6 / 19

7. PyPy in the real world (2) Real-time online advertising auctions

   tight latency requirement (<100ms) high throughput (hundreds of thousands of requests per second) 30% speedup We run PyPy basically everywhere Julian Berman antocuni (PyCon Otto) PyPy Status Update April 07 2017 7 / 19

8. PyPy in the real world (3) IoT on the cloud

   5-10x faster We do not even run benchmarks on CPython because we just know that PyPy is way faster Tobias Oberstein antocuni (PyCon Otto) PyPy Status Update April 07 2017 8 / 19

9. Usual questions Is it stable/mature? Can I use it in

   production? Does package X work? What about Python 3? antocuni (PyCon Otto) PyPy Status Update April 07 2017 9 / 19

10. cpyext Pure Python pacakges just work (TM) C extensions goes

    through cpyext Lots of progresses in the compatibility area Still slow Crossing the Python/C border takes ages lots of low hanging fruits to be taken :) Use CFFI if possible antocuni (PyCon Otto) PyPy Status Update April 07 2017 10 / 19

11. demo live demo time :) antocuni (PyCon Otto) PyPy Status

    Update April 07 2017 11 / 19

12. numpy It just works ~5 failing tests Better GC integration

    needed The GC does not know how much raw memory numpy objects are keeping alive It looks like a memory leak antocuni (PyCon Otto) PyPy Status Update April 07 2017 12 / 19

13. Other libs cython it works pandas ~95% of passing tests

    most failures are because it cannot find Tkinter libs We saw some segfaults scipy ~99% of passing tests Numpy memory leaks? matplotlib Saving to disk works Jupyter inline works no GUI backends antocuni (PyCon Otto) PyPy Status Update April 07 2017 13 / 19

14. Usual questions Is it stable/mature? Can I use it in

    production? Does package X work? What about Python 3? antocuni (PyCon Otto) PyPy Status Update April 07 2017 14 / 19

15. PyPy 3.5 PyPy 5.7.1 supports Python 3.5 (beta) Funded by

    a big grant by Mozilla (thanks!) https://morepypy.blogspot.com/2016/08/ pypy-gets-funding-from-mozilla-for.html Still some failing tests Very good results on asyncio antocuni (PyCon Otto) PyPy Status Update April 07 2017 15 / 19

16. PyPy 3.5 asyncio Between 2.87 and 8.08 times faster antocuni

    (PyCon Otto) PyPy Status Update April 07 2017 16 / 19

17. Usual questions Is it stable/mature? Can I use it in

    production? Does package X work? What about Python 3? Bonus topic: vmprof antocuni (PyCon Otto) PyPy Status Update April 07 2017 17 / 19

18. VMProf Statistical profiler Low overhead (~5%) For CPython and PyPy

    Distinguish between: Interpreter Native code JIT (PyPy only) GC (PyPy only) antocuni (PyCon Otto) PyPy Status Update April 07 2017 18 / 19

19. VMProf example $ python -m vmprof --web foobar.py $ pypy

    -m vmprof --web foobar.py $ pypy -m vmprof --web --jitlog foobar.py antocuni (PyCon Otto) PyPy Status Update April 07 2017 19 / 19