Pro Yearly is on sale from $80 to $50! »

Fast Python? Don't Bother

Ca04455902d1b207348b6d406432718f?s=47 Russel Winder
September 17, 2016

Fast Python? Don't Bother

PyConUK 2016 session on not using Python for high performance computation, but instead using Chapel.

Ca04455902d1b207348b6d406432718f?s=128

Russel Winder

September 17, 2016
Tweet

Transcript

  1. Copyright © 2016 Russel Winder 1 Fast Python? Don’t Bother

    Russel Winder russel@winder.org.uk https://www.russel.org.uk @russel_winder
  2. Copyright © 2016 Russel Winder 2 There is a rumour

    I only come to PyCon UK to talk about languages other than Python…
  3. Copyright © 2016 Russel Winder 3 …this is not true…

  4. Copyright © 2016 Russel Winder 4 …well not entirely.

  5. Copyright © 2016 Russel Winder 5 A gentle rant, but…

  6. Copyright © 2016 Russel Winder 6 …a bit of a

    rant nonetheless.
  7. Copyright © 2016 Russel Winder 7 Python is… • A

    dynamically typed (or not) language. • Quite popular, in data science, and data visualisation. • Quite popular for sys. admin. • Quite popular for Web-y stuff. • Having it’s development driven by IDEs. Type signatures.
  8. Copyright © 2016 Russel Winder 8 Slow

  9. Copyright © 2016 Russel Winder 9 Python is not… •

    A high performance computational language. • A language in which you can write parallel programs.
  10. Copyright © 2016 Russel Winder 10

  11. Copyright © 2016 Russel Winder 11 Maybe the GIL is

    a good thing, it stops us using Python when we shouldn’t.
  12. Copyright © 2016 Russel Winder 12 Why?

  13. Copyright © 2016 Russel Winder 13 Parallelism

  14. Copyright © 2016 Russel Winder 14 So how to do

    computation? • Cython • Numba • NumPy
  15. Copyright © 2016 Russel Winder 15 Cython • Write Python

    and annotate the code to make it appear like C so it is actually C by source-to-source translation. Hats off to Pyrex.
  16. Copyright © 2016 Russel Winder 16 Why not just write

    C if you want to write C?
  17. Copyright © 2016 Russel Winder 17 Because C is a

    programming language for writing operating systems, not applications?
  18. Copyright © 2016 Russel Winder 18 What we want is

    Fortran or even FORTRAN.
  19. Copyright © 2016 Russel Winder 19 Or not.

  20. Copyright © 2016 Russel Winder 20 Numba • Write Python

    code and decorate with decorator so as to perform non-Python translation via LLVM to native code.
  21. Copyright © 2016 Russel Winder 21 Does Python really have

    native code semantics?
  22. Copyright © 2016 Russel Winder 22 Does RPython really have

    native code semantics?
  23. Copyright © 2016 Russel Winder 23 No

  24. Copyright © 2016 Russel Winder 24 NumPy • And the

    whole SciPy, Pandas, etc. kit and caboodle. • A C subsystem, providing opaque data types and operations well integrated with Python as an API.
  25. Copyright © 2016 Russel Winder 25 It’s just a C

    system with a nice API. Lots of niceness to the API.
  26. Copyright © 2016 Russel Winder 26 Is C good enough

    for computation in the modern era?
  27. Copyright © 2016 Russel Winder 27 No

  28. Copyright © 2016 Russel Winder 28 Why?

  29. Copyright © 2016 Russel Winder 29 Parallelism

  30. Copyright © 2016 Russel Winder 30 Multi-multicore processors GPGPU

  31. Copyright © 2016 Russel Winder 31 OpenCL Vulkan CUDA

  32. Copyright © 2016 Russel Winder 32 Actors CSP Dataflow Data

    Parallelism
  33. Copyright © 2016 Russel Winder 33 Parallel Processing • Threads

    and thread pools: – C – but very low level. – C++ – better than C. – Java: Akka, Quasar, Hadoop, Apache Spark, GPars. – D – actors and data parallelism. • Partitioned Global Address Space (PGAS): – Chapel – X10 There are man, many more: Pony, Nim, etc.
  34. Copyright © 2016 Russel Winder 34 Where does Python fit

    in?
  35. Copyright © 2016 Russel Winder 35 It doesn’t for the

    computation…
  36. Copyright © 2016 Russel Winder 36 …but there is a

    way forward:
  37. Copyright © 2016 Russel Winder 37 Microservices Architecture

  38. Copyright © 2016 Russel Winder 38 Processes communicating over a

    network. Processes on same computer communicating using inter-process communication.
  39. Copyright © 2016 Russel Winder 39 Processes communicating is the

    only way pure Python code can create parallelism so it must be Pythonic.
  40. Copyright © 2016 Russel Winder 40 Intermix Python processes with

    non-Python ones.
  41. Copyright © 2016 Russel Winder 41 Intermix: • Python •

    C • C++ • Rust • D • Chapel
  42. Copyright © 2016 Russel Winder 42 Peek at D and

    Chapel. https://github.com/russel/Pi_Quadrature
  43. Copyright © 2016 Russel Winder 43 Follow Up… Short hands-on

    workshop: Monday 2016-09-19T14:30+01:00 Room A
  44. Copyright © 2016 Russel Winder 44 Fast Python? Don’t Bother

    Russel Winder russel@winder.org.uk https://www.russel.org.uk @russel_winder