High Performance Python

memory_profiler, line_profiler, Cython, Numba for Python 2.7

Transcript

1. www.morconsulting.c High Performance Python – find your bottleneck and go much faster London Python Usergroup Oct 2013 Ian Ozsvald @IanOzsvald MorConsulting.com
2. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 “High Performance Python” • Publishing mid 2014 • Please join the mailing list via IanOzsvald.com
3. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 About Ian Ozsvald • “Exploiter of Data” in MorConsulting.com • Teach: PyCon, EuroSciPy, EuroPython • Various ML/Parallel/Data projects • ShowMeDo.com • IanOzsvald.com
4. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 I teach High Performance... • because it is embarrassingly easy to collect lots of data • turning data->actionable information is hard
5. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 The Julia Set Fractal
6. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 The code (1/2)
7. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 The code (2/2)
8. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 cProfile & runsnakerun
9. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 kernprof.py -v -l julia.py
10. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 memory_profiler.py -v -l julia.py
11. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 mprof (memory_profiler)
12. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython (annotated->gcc)
13. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython (annotated->gcc)
14. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython (annotated->gcc)
15. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Cython results Straight CPython (in VM): 8.9s Cython1: 5.5s (uses Python lists) Cython3: 3s (uses numpy arrays) Cython4: 0.15s (expanded math) 66* speed­up
16. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Numba (continuum.io) • Just in Time Compiler (like PyPy/JVM) • Numpy-compatible (unlike PyPy) • LLVM toolchain • Multiple targets (e.g. GPU) • Installation is a touch tricky...
17. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Numba (continuum.io)
18. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Numba results Straight CPython (in VM): 8.9s Numba: 1.4s (using numpy arrays – faster than Cython3 at 3s) Numba2: 0.45s (expanded math) 22* speed­up (vs 66* for Cython)
19. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Missed out... • Ignored PyPy, ShedSkin, Parakeet... • Didn't look at parallelisation • Didn't look at other memory profilers • This will go into the book and maybe training in London
20. Ian@MorConsulting.com @IanOzsvald London Python Usergroup October 2013 Thank you! • Training interest? • Want to interview more CTOs • Join book mailing list (on ianozsvald.com) • Ian@IanOzsvald.com • @IanOzsvald • MorConsulting.com • GitHub/IanOzsvald