TOYING WITH PYTHON BYTECODE

Ever wondered what happens when you execute your Python programs? Would you like to gain insights into writing performance-oriented code or be able to explain why Python is an interpreted or compiled language? 

If so, then please drop in. I will take you on a walk in the forest of abstract syntax trees grown by lexers and parsers where compilers generate streams of bits understood and run by interpreters. Together, we will examine the "intermediate language" responsible for expressing your code as machine instructions, and look at the ways in which we can understand it, for fun and profit, through an application of the dis module. 

Using these learnings, we will pair up and optimise some problematic Python code in the second part of this session. We will also reflect on how this code appears from the perspective of the Interpreter.

This is a beginner level talk and the only prerequisites are a curious mind, basic Python knowledge and a laptop with Jupyter Notebook installed running Python 3.x. If you don't have the latter, rest assured there will be enough people with one around.

Transcript

1. Jan Chwiejczak iamjanhak www.github.com/janhak/bytecode
2. whoami  Python Dev  I work with robots  Cambridge Medical Robotics  iamjanhak  janhak/bytecode UPGRADE JOIN CMR
3. What I would like to explore  Gain insight into how Python executes code  Get hands on practice using the dis module to look at Python Bytecode  For this talk by Python I mean CPython www.github.com/janhak/bytecode
4. Python execution model www.github.com/janhak/bytecode
5. Lexing www.github.com/janhak/bytecode Source code Lexer Tokens
6. Parsing www.github.com/janhak/bytecode Stream of tokens Parser AST
7. Abstract Syntax Tree missing docs https://greentreesnakes.readthedocs.io/
8. Compiling www.github.com/janhak/bytecode Structured code Compiler ByteCode
9. Interpreter www.github.com/janhak/bytecode ByteCode Interpreter Program Output
10. Simple stack based interpreter  Let’s start with minimal interpreter that understands three instructions:  LOAD_VALUE  ADD_TWO_VALUES  PRINT_ANSWER
11. Interpreter Code Execution  Suppose we want to execute “7 + 5”  LOAD_VALUE - 0 # the first number  LOAD_VALUE - 1 # the second number  ADD_TWO_VALUES - None  PRINT_ANSWER - None First number Second number First number Result
12. What is bytecode?
13. www.github.com/janhak/bytecode AN INTERMEDIATE REPRESENTATION OF YOUR PROGRAM
14. www.github.com/janhak/bytecode WHAT THE INTERPRETER WORKS WITH WHEN IT RUNS YOUR PROGRAM
15. www.github.com/janhak/bytecode MACHINE CODE FOR A VIRTUAL MACHINE
16. www.github.com/janhak/bytecode A SERIES OF INSTRUCTIONS FOR STACK OPERATIONS
17. www.github.com/janhak/bytecode BUNCH OF .PYC FILES
18. Finally! Let’s switch to the real deal
19. Dis module in action! instruction line no index into bytecode operation name argument index argument value www.github.com/janhak/bytecode
20. Thanks Thank you for coming and contributing to the learning of others
21. Further Resources:  A Python Interpreter written in Python:  http://www.aosabook.org/en/500L/a-python-interpreter-written-in-python.html  https://github.com/nedbat/byterun  Hand crafted ByteCode:  http://multigrad.blogspot.co.uk/2014/06/fun-with-python-bytecode.html  Anjana Vakil presentation:  https://speakerdeck.com/vakila/exploring-python-bytecode  Docs for the dis module:  https://docs.python.org/3/library/dis.html  Exploring ceval.c at the heart of interpreter:  https://tech.blog.aknin.name/category/my-projects/pythons-innards/