Philip James, Asheesh Laroia - Type python, press enter. What happens?

Philip James, Asheesh Laroia - Type python, press enter. What happens?

This talk discusses how the Python interpreter starts, from the perspective of the operating system (OS). Together, we will see the ins & outs of processes: fork(), exec(), stdin, and stdout.

It focuses on OS concepts and requires no background knowledge, using analogies to Python data structures. (The talk does not discuss Python’s own initialization, such as site.py or global variables.)

https://us.pycon.org/2015/schedule/presentation/340/

D5710b3bca38f1233274b4cbc523dc4b?s=128

PyCon 2015

April 18, 2015
Tweet

Transcript

  1. Type Python, Press Enter, What Happens? Asheesh Laroia, Philip James

    (@asheeshlaroia, @phildini) PyCon 2015
  2. Who are these guys?

  3. paulproteus@vellum:~ $

  4. paulproteus@vellum:~ $ python

  5. paulproteus@vellum:~ $ python Python 2.7.6 (default, Apr 11 2014, 22:59:56)

    [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
  6. paulproteus@vellum:~ $

  7. paulproteus@vellum:~ $ python

  8. paulproteus@vellum:~ $

  9. paulproteus@vellum:~ $ echo $PATH

  10. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin:

  11. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $

  12. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $ python

  13. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $ python

  14. paulproteus@vellum:~ $ echo $PATH /home/paulproteus/bin:/usr/bin:/usr/local/bin: paulproteus@vellum:~ $ python

  15. paulproteus@vellum:~ $

  16. paulproteus@vellum:~ $ hexdump -C /usr/bin/python

  17. paulproteus@vellum:~ $ hexdump -C /usr/bin/python 00000000 7f 45 4c 46

    02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............| 00000010 02 00 3e 00 01 00 00 00 55 49 57 00 00 00 00 00 |..>.....UIW.....| 00000020 40 00 00 00 00 00 00 00 08 15 33 00 00 00 00 00 |@.........3.....| 00000030 00 00 00 00 40 00 38 00 09 00 40 00 1c 00 1b 00 |....@.8...@.....| 00000040 06 00 00 00 05 00 00 00 40 00 00 00 00 00 00 00 |........@.......| 00000050 40 00 40 00 00 00 00 00 40 00 40 00 00 00 00 00 |@.@.....@.@.....| 00000060 f8 01 00 00 00 00 00 00 f8 01 00 00 00 00 00 00 |................| 00000070 08 00 00 00 00 00 00 00 03 00 00 00 04 00 00 00 |................| 00000080 38 02 00 00 00 00 00 00 38 02 40 00 00 00 00 00 |8.......8.@.....| 00000090 38 02 40 00 00 00 00 00 1c 00 00 00 00 00 00 00 |8.@.............|
  18. paulproteus@vellum:~ $

  19. paulproteus@vellum:~ $ file /usr/bin/python

  20. paulproteus@vellum:~ $ file /usr/bin/python /usr/bin/python: ELF 64-bit LSB executable

  21. for dir in os.getenv( 'PATH').split(':'): try: os.stat(dir + '/python') return

    dir except: pass raise CommandNotFound()
  22. bash pid: 1221

  23. bash pid: 1221 paulproteus@vellum:~ $

  24. bash pid: 1221 paulproteus@vellum:~ $ exec python

  25. python pid: 1221

  26. python pid: 1221

  27. python pid: 1221 paulproteus@vellum:~ $

  28. python pid: 1221 paulproteus@vellum:~ $ [Process completed]

  29. bash pid: 1221

  30. bash pid: 1221 fork()

  31. bash pid: 1221 fork() bash pid: 1222

  32. bash pid: 1221 fork() bash pid: 1222 exec()

  33. bash pid: 1221 fork() python pid: 1222 exec()

  34. paulproteus@vellum:~ $

  35. paulproteus@vellum:~ $ pstree

  36. paulproteus@vellum:~ $ pstree init─┬ ... ├─cron ... └─sshd─┬─bash───python └─bash───pstree

  37. Process Creation 1. Find binary in $PATH 2. Fork 3.

    Exec
  38. paulproteus@vellum:~ $ python -v

  39. # installing zipimport hook import zipimport # builtin # installed

    zipimport hook # /usr/lib/python2.7/site.pyc matches /usr/lib/python2.7/site.py import site # precompiled from /usr/ lib/python2.7/site.pyc # /usr/lib/python2.7/os.pyc matches /usr/lib/python2.7/os.py paulproteus@vellum:~ $ python -v
  40. bash pid: 1221 fork() python pid: 1222 exec( ‘/usr/bin/python’, [‘python’,

    ‘-v’] )
  41. bash pid: 1221 fork() python pid: 1222 argv = [‘python’,

    ‘-v’] exec( ‘/usr/bin/python’, [‘python’, ‘-v’] )
  42. python pid: 1222 argv = [‘python’]

  43. Python 2.7.6 (default, Apr 11 2014, 22:59:56) [GCC 4.8.2] on

    linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
  44. None
  45. /dev/pts/3

  46. python pid: 1222 0: /dev/pts/3

  47. python pid: 1222 0: /dev/pts/3 stdin

  48. python pid: 1222 0: /dev/pts/3 2

  49. python pid: 1222 0: /dev/pts/3 termios 2

  50. python pid: 1222 0: /dev/pts/3 termios 2

  51. paulproteus@vellum:~ $

  52. paulproteus@vellum:~ $ strace python

  53. paulproteus@vellum:~ $ strace python [...] read(0,

  54. paulproteus@vellum:~ $ strace python [...] read(0, 2

  55. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) 2

  56. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, 2

  57. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, 2

    +
  58. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, “+”,

    1) 2 +
  59. paulproteus@vellum:~ $ strace python [...] read(0, “2”, 1) read(0, “+”,

    1) read(0, “2”, 1) read(0, “\n”, 1) 2 + 2 Enter
  60. while True: line = ‘’ chr = sys.stdin.read(1) line +=

    chr if chr == ‘\n’: print eval(line)
  61. while True: line = ‘’ chr = sys.stdin.read(1) line +=

    chr if chr == ‘\n’: print eval(line)
  62. python pid: 1222 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3

  63. bash pid: 1221 python pid: 1222 0: /dev/pts/3 1: /dev/pts/3

    2: /dev/pts/3 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3
  64. bash pid: 1221 python pid: 1222 0: /dev/pts/3 1: /dev/pts/3

    2: /dev/pts/3 0: /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 foreground
  65. Special Types of Input

  66. Ctrl C

  67. Ctrl C termios

  68. sigint Ctrl C termios

  69. python pid: 1222 sigint Ctrl C termios

  70. >>>

  71. >>> Crtl C

  72. >>> Crtl C KeyboardInterrupt >>>

  73. python pid: 1222 sigint

  74. python pid: 1222 sigint sig pipe

  75. cat pid: 1222 0: /dev/pts/3 Ctrl D termios

  76. cat pid: 1222 0: /dev/pts/3 Ctrl D termios

  77. python pid: 1222 0: /dev/pts/3 Ctrl D termios

  78. >>> import sys >>> sys.exit()

  79. >>> raise SystemExit()

  80. paulproteus@vellum:~ $ strace python

  81. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …>

  82. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …> paulproteus@vellum:~

    $
  83. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …> paulproteus@vellum:~

    $ echo $?
  84. paulproteus@vellum:~ $ strace python [...] exit(0 <no return …> paulproteus@vellum:~

    $ echo $? 0
  85. >>> import sys >>> sys.exit(1)

  86. >>> import sys >>> sys.exit(1) paulproteus@vellum:~ $ echo $? 1

  87. bash pid: 1221

  88. python pid: 1222 argv = [‘python’] bash pid: 1221

  89. python pid: 1222 argv = [‘python’] 0: /dev/pts/3 1: /dev/pts/3

    2: /dev/pts/3 bash pid: 1221
  90. python pid: 1222 argv = [‘python’] sigint sig pipe 0:

    /dev/pts/3 1: /dev/pts/3 2: /dev/pts/3 bash pid: 1221
  91. Be Curious! • pstree • strace • file/hexdump

  92. Talks to explore more • Systems Programming as a Swiss

    Army Knife (Julia Evans, Room 517c, Saturday @1050am)* • Exploring is never boring: understanding CPython without reading the code (Allison Kaptur, Room 710a, Saturday @ 245pm) • Where in your RAM is "python san_diego.py”? (Ying Li, Room 517c, Sunday @ 110pm) • Getting comfortable with web security: A hands-on session (Asheesh Laroia, Jacky Chang, Nicole Zuckerman, Room 512cg, Thursday @ 9am)* • Finding Spammers & Scammers through Rate Tracking with Python & Redis (Mica Swyers, Jay Chan, Room 511, Sunday @ 150pm) Talks by friends *time machine may be required
  93. Questions? Thanks! Asheesh Laroia, Philip James (@asheeshlaroia, @phildini)