Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyPy Status Update

PyPy Status Update

Antonio Cuni

April 07, 2017
Tweet

More Decks by Antonio Cuni

Other Decks in Programming

Transcript

  1. PyPy Status Update Antonio Cuni PyCon Otto April 07 2017

    antocuni (PyCon Otto) PyPy Status Update April 07 2017 1 / 19
  2. About me PyPy core dev pdb++, fancycompleter, capnpy, ... Consultant,

    trainer http://antocuni.eu antocuni (PyCon Otto) PyPy Status Update April 07 2017 2 / 19
  3. What is PyPy Alternative, fast Python implementation Performance: JIT compiler,

    advanced GC PyPy 5.7.1 x86, ARM, s390x, PPC http://pypy.org antocuni (PyCon Otto) PyPy Status Update April 07 2017 3 / 19
  4. Usual questions Is it stable/mature? Can I use it in

    production? Does package X work? What about Python 3? antocuni (PyCon Otto) PyPy Status Update April 07 2017 5 / 19
  5. PyPy in the real world (1) High frequency trading platform

    for sports bets low latency is a must PyPy used in production since 2012 ~100 PyPy processes running 24/7 up to 10x speedups after careful tuning and optimizing for PyPy antocuni (PyCon Otto) PyPy Status Update April 07 2017 6 / 19
  6. PyPy in the real world (2) Real-time online advertising auctions

    tight latency requirement (<100ms) high throughput (hundreds of thousands of requests per second) 30% speedup We run PyPy basically everywhere Julian Berman antocuni (PyCon Otto) PyPy Status Update April 07 2017 7 / 19
  7. PyPy in the real world (3) IoT on the cloud

    5-10x faster We do not even run benchmarks on CPython because we just know that PyPy is way faster Tobias Oberstein antocuni (PyCon Otto) PyPy Status Update April 07 2017 8 / 19
  8. Usual questions Is it stable/mature? Can I use it in

    production? Does package X work? What about Python 3? antocuni (PyCon Otto) PyPy Status Update April 07 2017 9 / 19
  9. cpyext Pure Python pacakges just work (TM) C extensions goes

    through cpyext Lots of progresses in the compatibility area Still slow Crossing the Python/C border takes ages lots of low hanging fruits to be taken :) Use CFFI if possible antocuni (PyCon Otto) PyPy Status Update April 07 2017 10 / 19
  10. numpy It just works ~5 failing tests Better GC integration

    needed The GC does not know how much raw memory numpy objects are keeping alive It looks like a memory leak antocuni (PyCon Otto) PyPy Status Update April 07 2017 12 / 19
  11. Other libs cython it works pandas ~95% of passing tests

    most failures are because it cannot find Tkinter libs We saw some segfaults scipy ~99% of passing tests Numpy memory leaks? matplotlib Saving to disk works Jupyter inline works no GUI backends antocuni (PyCon Otto) PyPy Status Update April 07 2017 13 / 19
  12. Usual questions Is it stable/mature? Can I use it in

    production? Does package X work? What about Python 3? antocuni (PyCon Otto) PyPy Status Update April 07 2017 14 / 19
  13. PyPy 3.5 PyPy 5.7.1 supports Python 3.5 (beta) Funded by

    a big grant by Mozilla (thanks!) https://morepypy.blogspot.com/2016/08/ pypy-gets-funding-from-mozilla-for.html Still some failing tests Very good results on asyncio antocuni (PyCon Otto) PyPy Status Update April 07 2017 15 / 19
  14. PyPy 3.5 asyncio Between 2.87 and 8.08 times faster antocuni

    (PyCon Otto) PyPy Status Update April 07 2017 16 / 19
  15. Usual questions Is it stable/mature? Can I use it in

    production? Does package X work? What about Python 3? Bonus topic: vmprof antocuni (PyCon Otto) PyPy Status Update April 07 2017 17 / 19
  16. VMProf Statistical profiler Low overhead (~5%) For CPython and PyPy

    Distinguish between: Interpreter Native code JIT (PyPy only) GC (PyPy only) antocuni (PyCon Otto) PyPy Status Update April 07 2017 18 / 19
  17. VMProf example $ python -m vmprof --web foobar.py $ pypy

    -m vmprof --web foobar.py $ pypy -m vmprof --web --jitlog foobar.py antocuni (PyCon Otto) PyPy Status Update April 07 2017 19 / 19