High Performance Python (Hacker News London Oct 2013)

Short talk on ways to profile the speed and memory usage of a CPU-bound function in Python, then a short demo with Cython and Numba on ways to speed it up.

Transcript

1. www.morconsulting.c High Performance Python – find your bottleneck and go much faster HackerNewsLondon Oct 2013 Ian Ozsvald @IanOzsvald MorConsulting.com
2. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 “High Performance Python” • Publishing mid 2014 • Please join the mailing list via IanOzsvald.com
3. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 About Ian Ozsvald • “Exploiter of Data” in MorConsulting.com • Teach: PyCon, EuroSciPy, EuroPython • SocialTies.com (an “exploiter of tweets”) • ShowMeDo.com • IanOzsvald.com
4. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 I teach High Performance... • because it is embarrassingly easy to collect lots of data • turning data->actionable information is hard • Is there interest in a data science talk?
5. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 The Julia Set Fractal
6. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 The code (1/2)
7. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 The code (2/2)
8. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 The Julia Set Fractal
9. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 cProfile & runsnakerun
10. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 kernprof.py -v -l julia.py
11. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 memory_profiler.py -v -l julia.py
12. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Cython (annotated->gcc)
13. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Cython (annotated->gcc)
14. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Cython (annotated->gcc)
15. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Cython results Straight CPython (in VM): 8.9s Cython1: 5.5s (uses Python lists) Cython3: 3s (uses numpy arrays) Cython4: 0.15s (expanded math) 66* speed­up
16. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Numba (continuum.io)
17. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Numba (continuum.io)
18. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Numba results Straight CPython (in VM): 8.9s Numba: 1.4s (using numpy arrays – faster than Cython3 at 3s) Numba2: 0.45s (expanded math) 22* speed­up (vs 66* for Cython)
19. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Missed out... • Ignored PyPy, ShedSkin, Parakeet... • Didn't look at parallelisation • Didn't look at other memory profilers • This will go into the book and maybe training in London
20. Ian@MorConsulting.com @IanOzsvald Hacker News London October 2013 Thank you! • Contract around HPC? • Want to interview more CTOs • Join book mailing list (on ianozsvald.com) • Ian@IanOzsvald.com • @IanOzsvald • MorConsulting.com • GitHub/IanOzsvald