Four Python Pains (May 2011)

This presentation was supposed to be given at a workshop in May 2011 that was eventually cancelled. It was supposed to spark a (constructive) conversation, not a controversy ;)

Transcript

1. Four Python Pains Stefane Fermigier Scripting Languages Workshop, IRILL, May 2011 Friday, April 20, 2012
2. Part1: Some context Friday, April 20, 2012
3. Friday, April 20, 2012
4. Friday, April 20, 2012
5. Friday, April 20, 2012
6. Friday, April 20, 2012
7. Friday, April 20, 2012
8. Friday, April 20, 2012
9. Friday, April 20, 2012
10. Wrong (on several points) Friday, April 20, 2012
11. Python (“Scripting Language”) • Has built-in syntactical support for arbitrarily complex data structures (lists, dictionaries) • Supports significantly complex applications: ERP5, Plone, Nuxeo CPS (~320 KLOC), OpenERP (~150 KLOC), Komodo (~250 KLOC) • Is compiled (to bytecode) Friday, April 20, 2012
12. Scala (“Sys. Prog. Language”) • Is interpreted (it as a REPL) • Is a high-level language - in other words, has a high CPU instructions / statement ratio (comparable or higher than Python) Friday, April 20, 2012
13. Other counterexamples • Java, pre-generics (Java SE < 5, i.e. at the time of Ousterhout’s paper), had really weak support for static typing • Clojure (that many consider in the same league as Scala) is not even statically typed Friday, April 20, 2012
14. Friday, April 20, 2012
15. Friday, April 20, 2012
16. At which expense? Friday, April 20, 2012
17. • Execution speed? • Safety? • Maintainability? • What else? Friday, April 20, 2012
18. Or maybe Python is a scripting language after all ? Friday, April 20, 2012
19. Uses of Python • Education • Small to medium sized sysadmin and software engineering project • Medium to large web frameworks (Zope, Django...) and applications (Youtube...) • Scientific computing (numpy, scipy...) • Scripting of games, desktop apps, etc. Friday, April 20, 2012
20. • Speed • Scalability • Type-safety • Adherence to external code bases, and an implementation (CPython) Recap: 4 pain points Friday, April 20, 2012
21. Part 2: Discussion Friday, April 20, 2012
22. Speed • Python (seems to be) slower by 10x than Clojure and V8 (JavaScript) • Both Clojure and JavaScript are dynamically typed languages • TODO: look at Clojure and V8 and see how they do it Friday, April 20, 2012
23. Friday, April 20, 2012
24. Scalability • CPython multi-threading / multi-core performance is constrained by “the GIL” (global interpretor lock) • It doesn’t have to be there (Jython doesn’t have one) but removing it from Python is controversial • Python has support for some par prog constructs (see: http://wiki.python.org/moin/ ParallelProcessing) but not the fancy new ones (a la Scala, Clojure, Go, etc.) Friday, April 20, 2012
25. Friday, April 20, 2012
26. Type Safety • Static typic is good to create safer programs, and also to ease refactoring (necessary condition for agility) • Since Python 3, type annotations are possible on variables, parameters, etc. • Some tools (e.g. Jetbrains’ PyCharm) have support for type inference on Python, and enable: code completion, errors detections, refactoring... Friday, April 20, 2012
27. Friday, April 20, 2012
28. Adherence to CPython • There are at least 4 major Python implementations: CPython, Jython, IronPython and PyPy • But only CPython is mainstream • Why? Because all the “system programming lang” extensions written for CPython cannot be used with the others • And we’re talking about hundreds of packages Friday, April 20, 2012
29. Solution? • There are also tools, languages and extensions that ease up the task of writing a CPython extension: SWIG, ctypes, Pyrex, Cython... • By encapsulating part of the complexity and low-level details of creating a Python extension, it could make is both easier, more robust and easier to share extensions between language implementations Friday, April 20, 2012
30. Conclusion Friday, April 20, 2012
31. • Python could maybe be made 10x faster by applying ideas from Clojure and V8 • Python could be made more scalable by removing the GIL (probably needs to rewrite the interpreter, though) and adding modern parallel programming paradigms Friday, April 20, 2012
32. • Python programs could be made more robust with the help of static typing analysis tools + a little help from the developers (i.e. add a few type annotations in their programs and libraries) • Python is a scripting language after all (but not only), and could benefit from a way of writing extensions that is higher level than the common one (but then: how to convert the existing code base?) Friday, April 20, 2012