Upgrade to Pro — share decks privately, control downloads, hide ads and more …

High Performance Python

ianozsvald
November 01, 2013

High Performance Python

memory_profiler, line_profiler, Cython, Numba for Python 2.7

ianozsvald

November 01, 2013
Tweet

More Decks by ianozsvald

Other Decks in Technology

Transcript

  1. www.morconsulting.c
    High Performance Python – find your
    bottleneck and go much faster
    London Python Usergroup Oct 2013
    Ian Ozsvald @IanOzsvald MorConsulting.com

    View full-size slide

  2. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    “High Performance Python”
    • Publishing mid 2014
    • Please join the mailing
    list via IanOzsvald.com

    View full-size slide

  3. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    About Ian Ozsvald
    • “Exploiter of Data” in MorConsulting.com
    • Teach: PyCon, EuroSciPy, EuroPython
    • Various ML/Parallel/Data projects
    • ShowMeDo.com
    • IanOzsvald.com

    View full-size slide

  4. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    I teach High Performance...
    • because it is embarrassingly easy to
    collect lots of data
    • turning data->actionable information is
    hard

    View full-size slide

  5. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    The Julia Set Fractal

    View full-size slide

  6. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    The code (1/2)

    View full-size slide

  7. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    The code (2/2)

    View full-size slide

  8. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    cProfile & runsnakerun

    View full-size slide

  9. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    kernprof.py -v -l julia.py

    View full-size slide

  10. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    memory_profiler.py -v -l julia.py

    View full-size slide

  11. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    mprof (memory_profiler)

    View full-size slide

  12. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Cython (annotated->gcc)

    View full-size slide

  13. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Cython (annotated->gcc)

    View full-size slide

  14. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Cython (annotated->gcc)

    View full-size slide

  15. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Cython results
    Straight CPython (in VM): 8.9s
    Cython1: 5.5s (uses Python lists)
    Cython3: 3s (uses numpy arrays)
    Cython4: 0.15s (expanded math)
    66* speed­up

    View full-size slide

  16. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Numba (continuum.io)
    • Just in Time Compiler (like PyPy/JVM)
    • Numpy-compatible (unlike PyPy)
    • LLVM toolchain
    • Multiple targets (e.g. GPU)
    • Installation is a touch tricky...

    View full-size slide

  17. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Numba (continuum.io)

    View full-size slide

  18. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Numba results
    Straight CPython (in VM): 8.9s
    Numba: 1.4s (using numpy arrays – faster
    than Cython3 at 3s)
    Numba2: 0.45s (expanded math)
    22* speed­up (vs 66* for Cython)

    View full-size slide

  19. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Missed out...
    • Ignored PyPy, ShedSkin, Parakeet...
    • Didn't look at parallelisation
    • Didn't look at other memory profilers
    • This will go into the book and maybe
    training in London

    View full-size slide

  20. [email protected] @IanOzsvald
    London Python Usergroup October 2013
    Thank you!
    • Training interest?
    • Want to interview more CTOs
    • Join book mailing list (on ianozsvald.com)
    [email protected]
    • @IanOzsvald
    • MorConsulting.com
    • GitHub/IanOzsvald

    View full-size slide