Fast Python? Don't Bother

PyConUK 2016 session on not using Python for high performance computation, but instead using Chapel.

Transcript

1. Copyright © 2016 Russel Winder 1 Fast Python? Don’t Bother Russel Winder russel@winder.org.uk https://www.russel.org.uk @russel_winder
2. Copyright © 2016 Russel Winder 2 There is a rumour I only come to PyCon UK to talk about languages other than Python…
3. Copyright © 2016 Russel Winder 3 …this is not true…
4. Copyright © 2016 Russel Winder 4 …well not entirely.
5. Copyright © 2016 Russel Winder 5 A gentle rant, but…
6. Copyright © 2016 Russel Winder 6 …a bit of a rant nonetheless.
7. Copyright © 2016 Russel Winder 7 Python is… • A dynamically typed (or not) language. • Quite popular, in data science, and data visualisation. • Quite popular for sys. admin. • Quite popular for Web-y stuff. • Having it’s development driven by IDEs. Type signatures.
8. Copyright © 2016 Russel Winder 8 Slow
9. Copyright © 2016 Russel Winder 9 Python is not… • A high performance computational language. • A language in which you can write parallel programs.
10. Copyright © 2016 Russel Winder 10
11. Copyright © 2016 Russel Winder 11 Maybe the GIL is a good thing, it stops us using Python when we shouldn’t.
12. Copyright © 2016 Russel Winder 12 Why?
13. Copyright © 2016 Russel Winder 13 Parallelism
14. Copyright © 2016 Russel Winder 14 So how to do computation? • Cython • Numba • NumPy
15. Copyright © 2016 Russel Winder 15 Cython • Write Python and annotate the code to make it appear like C so it is actually C by source-to-source translation. Hats off to Pyrex.
16. Copyright © 2016 Russel Winder 16 Why not just write C if you want to write C?
17. Copyright © 2016 Russel Winder 17 Because C is a programming language for writing operating systems, not applications?
18. Copyright © 2016 Russel Winder 18 What we want is Fortran or even FORTRAN.
19. Copyright © 2016 Russel Winder 19 Or not.
20. Copyright © 2016 Russel Winder 20 Numba • Write Python code and decorate with decorator so as to perform non-Python translation via LLVM to native code.
21. Copyright © 2016 Russel Winder 21 Does Python really have native code semantics?
22. Copyright © 2016 Russel Winder 22 Does RPython really have native code semantics?
23. Copyright © 2016 Russel Winder 23 No
24. Copyright © 2016 Russel Winder 24 NumPy • And the whole SciPy, Pandas, etc. kit and caboodle. • A C subsystem, providing opaque data types and operations well integrated with Python as an API.
25. Copyright © 2016 Russel Winder 25 It’s just a C system with a nice API. Lots of niceness to the API.
26. Copyright © 2016 Russel Winder 26 Is C good enough for computation in the modern era?
27. Copyright © 2016 Russel Winder 27 No
28. Copyright © 2016 Russel Winder 28 Why?
29. Copyright © 2016 Russel Winder 29 Parallelism
30. Copyright © 2016 Russel Winder 30 Multi-multicore processors GPGPU
31. Copyright © 2016 Russel Winder 31 OpenCL Vulkan CUDA
32. Copyright © 2016 Russel Winder 32 Actors CSP Dataflow Data Parallelism
33. Copyright © 2016 Russel Winder 33 Parallel Processing • Threads and thread pools: – C – but very low level. – C++ – better than C. – Java: Akka, Quasar, Hadoop, Apache Spark, GPars. – D – actors and data parallelism. • Partitioned Global Address Space (PGAS): – Chapel – X10 There are man, many more: Pony, Nim, etc.
34. Copyright © 2016 Russel Winder 34 Where does Python fit in?
35. Copyright © 2016 Russel Winder 35 It doesn’t for the computation…
36. Copyright © 2016 Russel Winder 36 …but there is a way forward:
37. Copyright © 2016 Russel Winder 37 Microservices Architecture
38. Copyright © 2016 Russel Winder 38 Processes communicating over a network. Processes on same computer communicating using inter-process communication.
39. Copyright © 2016 Russel Winder 39 Processes communicating is the only way pure Python code can create parallelism so it must be Pythonic.
40. Copyright © 2016 Russel Winder 40 Intermix Python processes with non-Python ones.
41. Copyright © 2016 Russel Winder 41 Intermix: • Python • C • C++ • Rust • D • Chapel
42. Copyright © 2016 Russel Winder 42 Peek at D and Chapel. https://github.com/russel/Pi_Quadrature
43. Copyright © 2016 Russel Winder 43 Follow Up… Short hands-on workshop: Monday 2016-09-19T14:30+01:00 Room A
44. Copyright © 2016 Russel Winder 44 Fast Python? Don’t Bother Russel Winder russel@winder.org.uk https://www.russel.org.uk @russel_winder