Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyPy Status Update

PyPy Status Update

Antonio Cuni

April 07, 2017
Tweet

More Decks by Antonio Cuni

Other Decks in Programming

Transcript

  1. PyPy Status Update
    Antonio Cuni
    PyCon Otto
    April 07 2017
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 1 / 19

    View Slide

  2. About me
    PyPy core dev
    pdb++, fancycompleter, capnpy, ...
    Consultant, trainer
    http://antocuni.eu
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 2 / 19

    View Slide

  3. What is PyPy
    Alternative, fast Python implementation
    Performance: JIT compiler, advanced GC
    PyPy 5.7.1
    x86, ARM, s390x, PPC
    http://pypy.org
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 3 / 19

    View Slide

  4. Speed
    7.4 times faster than CPython
    http://speed.pypy.org
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 4 / 19

    View Slide

  5. Usual questions
    Is it stable/mature? Can I use it in production?
    Does package X work?
    What about Python 3?
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 5 / 19

    View Slide

  6. PyPy in the real world (1)
    High frequency trading platform for sports bets
    low latency is a must
    PyPy used in production since 2012
    ~100 PyPy processes running 24/7
    up to 10x speedups
    after careful tuning and optimizing for PyPy
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 6 / 19

    View Slide

  7. PyPy in the real world (2)
    Real-time online advertising auctions
    tight latency requirement (<100ms)
    high throughput (hundreds of thousands of requests per
    second)
    30% speedup
    We run PyPy basically everywhere
    Julian Berman
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 7 / 19

    View Slide

  8. PyPy in the real world (3)
    IoT on the cloud
    5-10x faster
    We do not even run benchmarks on CPython
    because we just know that PyPy is way faster
    Tobias Oberstein
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 8 / 19

    View Slide

  9. Usual questions
    Is it stable/mature? Can I use it in production?
    Does package X work?
    What about Python 3?
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 9 / 19

    View Slide

  10. cpyext
    Pure Python pacakges just work (TM)
    C extensions goes through cpyext
    Lots of progresses in the compatibility area
    Still slow
    Crossing the Python/C border takes ages
    lots of low hanging fruits to be taken :)
    Use CFFI if possible
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 10 / 19

    View Slide

  11. demo
    live demo time :)
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 11 / 19

    View Slide

  12. numpy
    It just works
    ~5 failing tests
    Better GC integration needed
    The GC does not know how much raw memory
    numpy objects are keeping alive
    It looks like a memory leak
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 12 / 19

    View Slide

  13. Other libs
    cython
    it works
    pandas
    ~95% of passing tests
    most failures are because it cannot find Tkinter libs
    We saw some segfaults
    scipy
    ~99% of passing tests
    Numpy memory leaks?
    matplotlib
    Saving to disk works
    Jupyter inline works
    no GUI backends
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 13 / 19

    View Slide

  14. Usual questions
    Is it stable/mature? Can I use it in production?
    Does package X work?
    What about Python 3?
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 14 / 19

    View Slide

  15. PyPy 3.5
    PyPy 5.7.1 supports Python 3.5 (beta)
    Funded by a big grant by Mozilla (thanks!)
    https://morepypy.blogspot.com/2016/08/
    pypy-gets-funding-from-mozilla-for.html
    Still some failing tests
    Very good results on asyncio
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 15 / 19

    View Slide

  16. PyPy 3.5 asyncio
    Between 2.87 and 8.08 times faster
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 16 / 19

    View Slide

  17. Usual questions
    Is it stable/mature? Can I use it in production?
    Does package X work?
    What about Python 3?
    Bonus topic: vmprof
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 17 / 19

    View Slide

  18. VMProf
    Statistical profiler
    Low overhead (~5%)
    For CPython and PyPy
    Distinguish between:
    Interpreter
    Native code
    JIT (PyPy only)
    GC (PyPy only)
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 18 / 19

    View Slide

  19. VMProf example
    $ python -m vmprof --web foobar.py
    $ pypy -m vmprof --web foobar.py
    $ pypy -m vmprof --web --jitlog
    foobar.py
    antocuni (PyCon Otto) PyPy Status Update April 07 2017 19 / 19

    View Slide