Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyConZA 2015: "How PyPy runs your program" by Maciej Fijałkowski

Pycon ZA
October 02, 2015

PyConZA 2015: "How PyPy runs your program" by Maciej Fijałkowski

In this talk we would like to have a short introduction on how Python programs are compiled and executed, with special attention towards just-in-time compilation done by PyPy. PyPy is the most advanced Python interpreter around, and while it should generally just speed up your programs, there is a wide range of performance that you can get out of PyPy, ranging from slightly faster than CPython to C speeds, depending on how you write your programs.

We will split the talk in two parts. In the first part we will explain how things work, and what can and cannot be optimized, as well as describe the basic heuristics of the JIT compiler and optimizer. In the next part we will do a brief survey of existing tools for looking at performance of Python programs, with a specific focus on PyPy. We'll mostly focus on vmprof with a brief mention of others.

As a result of this talk, an audience member should be better equipped with the tools to write new software and improve existing software with performance in mind.

Pycon ZA

October 02, 2015
Tweet

More Decks by Pycon ZA

Other Decks in Programming

Transcript

  1. How PyPy runs your programs Maciej Fijałkowski PyCon ZA 2015

    Oct 2nd, 2015 fijal How PyPy runs your programs
  2. About me PyPy core developer for 8 years running consulting

    business http://baroquesoftware.com always interested in tooling and improving experience originally from Poland, living in Cape Town fijal How PyPy runs your programs
  3. This talk The idea is to learn: how pypy runs

    your programs how to assess the performance of your program additionally, why a lot of common folklore is not true fijal How PyPy runs your programs
  4. The basics of PyPy python interpreter, just that uses magic

    to run code faster (most of the time) completely different codebase, not written in C ~7x faster than cpython fijal How PyPy runs your programs
  5. PyPy - the wider angle download it, should come in

    your distribution x86, x86_64, arm; windows, os x, linux open source (MIT) fijal How PyPy runs your programs
  6. PyPy - usage mostly long running server programs call C

    using cffi, a lot of libraries just work use virtualenv (you should anyway) fijal How PyPy runs your programs
  7. PyPy - magic just in time compiler, replaces bytecode to

    assembler under your feet takes a while to warm up, which defeats most short running programs most of the time faster, sometimes slower heavily optimizing, tons of heuristics for typical python programs fijal How PyPy runs your programs
  8. PyPy - smallish example take python code run python in

    interpreted mode (slow) run python in meta-interpreter mode (VERY SLOW) compile to optimized assembler repeat if necessary fijal How PyPy runs your programs
  9. Tracing JIT follow what the program is doing enter special

    mode where all the operations are recorded compile the recorded list of operations add a bunch of “guards” that check that we’re following the correct path and correct optimizations if guard fails, jump to interpreter if guard fails enough jump to metainterpreter repeat until all the paths are compiled to assembler fijal How PyPy runs your programs
  10. Performance you need a metric (response time, number of requests)

    the less you’re trying to measure, the better benchmarks are a vast improvement repeatability is the key fijal How PyPy runs your programs
  11. Optimization for dummies Obligatory citation premature optimization is the root

    of all evil (D. Knuth) Pareto principle, or 80-20 rule 80% of the time will be spent in 20% of the program 20% of 1 mln is 200 000 Two golden rules: 1. Identify the slow spots 2. Optimize them fijal How PyPy runs your programs
  12. Why we’re here? because the points above don’t really work

    tradeoffs between productivity and performance once you fix obvious mistakes, profiles tend to look flat let’s look at some of them fijal How PyPy runs your programs
  13. the basics have a metric a number, the shorter iteration

    the better use science! fijal How PyPy runs your programs
  14. a word about timeit don’t use it really, never no,

    not even that time minimum is a terrible thing disables the GC fijal How PyPy runs your programs
  15. more complicated example django admin 30 req/s in 2015, that’s

    90mln operations to show you a simple website fijal How PyPy runs your programs
  16. baroquesoftware.com monetizing open source is a difficult question, warrants another

    talk small consultancy on execution of programs talk to me fijal How PyPy runs your programs