Type Python, Press Enter, What Happens?

This talk discusses how the Python interpreter starts, from the perspective of the operating system (OS). Together, we will see the ins & outs of processes: fork(), exec(), stdin, and stdout. It focuses on OS concepts and requires no background knowledge, using analogies to Python data structures. (The talk does not discuss Python’s own initialization, such as site.py or global variables.)

Transcript

1. Type Python, Press Enter, What Happens? Asheesh Laroia, Philip James (@asheeshlaroia, @phildini) PyCon 2015
2. Who are these guys?
3. paulproteus@vellum:~ $
4. paulproteus@vellum:~ $ python
5. paulproteus@vellum:~ $ python Python 2.7.6 (default, Apr 11 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
6. paulproteus@vellum:~ $
7. paulproteus@vellum:~ $ python
8. paulproteus@vellum:~ $
9. paulproteus@vellum:~ $ echo $PATH
10. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin:
11. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $
12. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $ python
13. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $ python
14. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $ python
15. paulproteus@vellum:~ $
16. paulproteus@vellum:~ $ hexdump -C /usr/bin/python
17. paulproteus@vellum:~ $ hexdump -C /usr/bin/python 00000000 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............| 00000010 02 00 3e 00 01 00 00 00 55 49 57 00 00 00 00 00 |..>.....UIW.....| 00000020 40 00 00 00 00 00 00 00 08 15 33 00 00 00 00 00 |@.........3.....| 00000030 00 00 00 00 40 00 38 00 09 00 40 00 1c 00 1b 00 |....@.8...@.....| 00000040 06 00 00 00 05 00 00 00 40 00 00 00 00 00 00 00 |........@.......| 00000050 40 00 40 00 00 00 00 00 40 00 40 00 00 00 00 00 |@.@.....@.@.....| 00000060 f8 01 00 00 00 00 00 00 f8 01 00 00 00 00 00 00 |................| 00000070 08 00 00 00 00 00 00 00 03 00 00 00 04 00 00 00 |................| 00000080 38 02 00 00 00 00 00 00 38 02 40 00 00 00 00 00 |8.......8.@.....| 00000090 38 02 40 00 00 00 00 00 1c 00 00 00 00 00 00 00 |8.@.............|
18. paulproteus@vellum:~ $
19. paulproteus@vellum:~ $ file /usr/bin/python
20. paulproteus@vellum:~ $ file /usr/bin/python /usr/bin/python: ELF 64-bit LSB executable
21. for dir in os.getenv( 'PATH').split(':'): try: os.stat(dir + '/python') return dir except: pass raise CommandNotFound()
22. bash pid: 1221
23. bash pid: 1221 paulproteus@vellum:~ $
24. bash pid: 1221 paulproteus@vellum:~ $ exec python
25. python pid: 1221
26. python pid: 1221
27. python pid: 1221 paulproteus@vellum:~ $
28. python pid: 1221 paulproteus@vellum:~ $ [Process completed]
29. bash pid: 1221
30. bash pid: 1221 fork()
31. bash pid: 1221 fork() bash pid: 1222
32. bash pid: 1221 fork() bash pid: 1222 exec()
33. bash pid: 1221 fork() python pid: 1222 exec()
34. paulproteus@vellum:~ $
35. paulproteus@vellum:~ $ pstree
36. paulproteus@vellum:~ $ pstree init─┬ ... ├─cron ... └─sshd─┬─bash───python └─bash───pstree
37. Process Creation 1. Find binary in $PATH 2. Fork 3. Exec
38. paulproteus@vellum:~ $ python -v
39. # installing zipimport hook import zipimport # builtin # installed zipimport hook # /usr/lib/python2.7/site.pyc matches /usr/lib/python2.7/site.py import site # precompiled from /usr/ lib/python2.7/site.pyc # /usr/lib/python2.7/os.pyc matches /usr/lib/python2.7/os.py paulproteus@vellum:~ $ python -v
40. bash pid: 1221 fork() python pid: 1222 exec( ‘/usr/bin/python’, [‘python’, ‘-v’] )
41. bash pid: 1221 fork() python pid: 1222 argv = [‘python’, ‘-v’] exec( ‘/usr/bin/python’, [‘python’, ‘-v’] )
42. python pid: 1222 argv = [‘python’]
43. Python 2.7.6 (default, Apr 11 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
45. /dev/pts/3
46. python pid: 1222 0: /dev/pts/3
47. python pid: 1222 0: /dev/pts/3 stdin
48. python pid: 1222 0: /dev/pts/3 2
49. python pid: 1222 0: /dev/pts/3 termios 2
50. python pid: 1222 0: /dev/pts/3 termios 2
51. paulproteus@vellum:~ $
52. paulproteus@vellum:~ $ strace python
53. paulproteus@vellum:~ $ strace python [...] read(0,
54. paulproteus@vellum:~ $ strace python [...] read(0, 2
55. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) 2
56. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, 2
57. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, 2 +
58. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, “+”, 1) 2 +
59. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, “+”, 1) read(0, “2”, 1) read(0, “\n”, 1) 2 + 2 Enter
60. while True: line = ‘’ chr = sys.stdin.read(1) line += chr if chr == ‘\n’: print eval(line)
61. while True: line = ‘’ chr = sys.stdin.read(1) line += chr if chr == ‘\n’: print eval(line)
62. python pid: 1222 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3
63. bash pid: 1221 python pid: 1222 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3
64. bash pid: 1221 python pid: 1222 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 foreground
65. Special Types of Input
66. Ctrl C
67. Ctrl C termios
68. sigint Ctrl C termios
69. python pid: 1222 sigint Ctrl C termios
70. >>>
71. >>> Crtl C
72. >>> Crtl C KeyboardInterrupt >>>
73. python pid: 1222 sigint
74. python pid: 1222 sigint sig pipe
75. cat pid: 1222 0: /dev/pts/3 Ctrl D termios
76. cat pid: 1222 0: /dev/pts/3 Ctrl D termios
77. python pid: 1222 0: /dev/pts/3 Ctrl D termios
78. >>> import sys >>> sys.exit()
79. >>> raise SystemExit()
80. paulproteus@vellum:~ $ strace python
81. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …>
82. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …> paulproteus@vellum:~ $
83. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …> paulproteus@vellum:~ $ echo $?
84. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …> paulproteus@vellum:~ $ echo $? 0
85. >>> import sys >>> sys.exit(1)
86. >>> import sys >>> sys.exit(1) paulproteus@vellum:~ $ echo $? 1
87. bash pid: 1221
88. python pid: 1222 argv = [‘python’] bash pid: 1221
89. python pid: 1222 argv = [‘python’] 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 bash pid: 1221
90. python pid: 1222 argv = [‘python’] sigint sig pipe 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 bash pid: 1221
91. Be Curious! • pstree • strace • file/hexdump
92. Talks to explore more • Systems Programming as a Swiss Army Knife (Julia Evans, Room 517c, Saturday @1050am)* • Exploring is never boring: understanding CPython without reading the code (Allison Kaptur, Room 710a, Saturday @ 245pm) • Where in your RAM is "python san_diego.py”? (Ying Li, Room 517c, Sunday @ 110pm) • Getting comfortable with web security: A hands-on session (Asheesh Laroia, Jacky Chang, Nicole Zuckerman, Room 512cg, Thursday @ 9am)* • Finding Spammers & Scammers through Rate Tracking with Python & Redis (Mica Swyers, Jay Chan, Room 511, Sunday @ 150pm) Talks by friends *time machine may be required
93. Questions? Thanks! Asheesh Laroia, Philip James (@asheeshlaroia, @phildini)