Upgrade to Pro — share decks privately, control downloads, hide ads and more …

High Performance Python

3d644406158b4d440111903db1f62622?s=47 ianozsvald
November 01, 2013

High Performance Python

memory_profiler, line_profiler, Cython, Numba for Python 2.7

3d644406158b4d440111903db1f62622?s=128

ianozsvald

November 01, 2013
Tweet

Transcript

  1. www.morconsulting.c High Performance Python – find your bottleneck and go

    much faster London Python Usergroup Oct 2013 Ian Ozsvald @IanOzsvald MorConsulting.com
  2. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 “High Performance Python”

    • Publishing mid 2014 • Please join the mailing list via IanOzsvald.com
  3. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 About Ian Ozsvald

    • “Exploiter of Data” in MorConsulting.com • Teach: PyCon, EuroSciPy, EuroPython • Various ML/Parallel/Data projects • ShowMeDo.com • IanOzsvald.com
  4. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 I teach High

    Performance... • because it is embarrassingly easy to collect lots of data • turning data->actionable information is hard
  5. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 The Julia Set

    Fractal
  6. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 The code (1/2)

  7. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 The code (2/2)

  8. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 cProfile & runsnakerun

  9. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 kernprof.py -v -l

    julia.py
  10. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 memory_profiler.py -v -l

    julia.py
  11. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 mprof (memory_profiler)

  12. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython (annotated->gcc)

  13. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython (annotated->gcc)

  14. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython (annotated->gcc)

  15. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython results Straight

    CPython (in VM): 8.9s Cython1: 5.5s (uses Python lists) Cython3: 3s (uses numpy arrays) Cython4: 0.15s (expanded math) 66* speed­up
  16. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Numba (continuum.io) •

    Just in Time Compiler (like PyPy/JVM) • Numpy-compatible (unlike PyPy) • LLVM toolchain • Multiple targets (e.g. GPU) • Installation is a touch tricky...
  17. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Numba (continuum.io)

  18. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Numba results Straight

    CPython (in VM): 8.9s Numba: 1.4s (using numpy arrays – faster than Cython3 at 3s) Numba2: 0.45s (expanded math) 22* speed­up (vs 66* for Cython)
  19. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Missed out... •

    Ignored PyPy, ShedSkin, Parakeet... • Didn't look at parallelisation • Didn't look at other memory profilers • This will go into the book and maybe training in London
  20. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Thank you! •

    Training interest? • Want to interview more CTOs • Join book mailing list (on ianozsvald.com) • Ian@IanOzsvald.com • @IanOzsvald • MorConsulting.com • GitHub/IanOzsvald