Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyCon 2016, File descriptors, Unix sockets and other POSIX magic

PyCon 2016, File descriptors, Unix sockets and other POSIX magic

Have you ever wondered how the OS manages open files and network connections, what this 'file descriptor' thing actually is all about, or what's so special about Unix sockets? In my talk I will give you a quick tour into the I/O layer and process model of Unix-like operating systems. You will learn how to securely identify and efficiently share resources between processes.

https://us.pycon.org/2016/schedule/presentation/2019/

33bd15feb2558d0050e863875e0f5f60?s=128

Christian Heimes

May 30, 2016
Tweet

More Decks by Christian Heimes

Other Decks in Programming

Transcript

  1. File descriptors, Unix sockets and other POSIX magic PyCon US

    2016 Christian Heimes christian@python.org @ChristianHeimes
  2. File descriptors, PyCon US 2016 2 Who am I •

    from Hamburg / Germany • Python core contributor since 2008 • PEP 379, 452, 456 • ssl, hashlib • Python Security Response Team • I put bytes and b'' into Python 2.6
  3. File descriptors, PyCon US 2016 3 Professional Life • Senior

    Software Engineer with Red Hat • Security Engineering & Identity Management • FreeIPA IdM • Dogtag PKI
  4. File descriptors, PyCon US 2016 4 Custodia Secrets for Containers

  5. File descriptors, PyCon US 2016 5 Recipes Concepts & Tools

    Linux / Unix only Python 3
  6. File descriptors, PyCon US 2016 6 Agenda • File descriptor

    • Operating System 101 • File descriptors and processes • Networking and sockets • Unix sockets, containers and sandboxing • Bonus track
  7. File descriptors, PyCon US 2016 7 Simplifications ahead

  8. Introducing file descriptors

  9. File descriptors, PyCon US 2016 9 Everything is a file

  10. File descriptors, PyCon US 2016 10 Every I/O operating uses

    a file descriptor
  11. File descriptors, PyCon US 2016 11 File descriptors are used

    for • reading / writing files • directories • devices • inter-process communication • network communication • I/O multiplexing • file system monitoring • …
  12. File descriptors, PyCon US 2016 12

  13. File descriptors, PyCon US 2016 13 Standard numbers • 0:

    standard input (sys.stdin) • 1: standard output (sys.stdout) • 2: standard error output (sys.stderr) • -1: error (Python raises an exception)
  14. File descriptors, PyCon US 2016 14 Simple example with open('example.txt')

    as f: print(f.read()) with open('example.txt') as f: print(f.read())
  15. File descriptors, PyCon US 2016 15 Reference file by descriptor

    with open('example.txt') as f: stat = os.stat(f.fileno()) os.chmod(f.fileno(), 0o640) with open('example.txt') as f: stat = os.stat(f.fileno()) os.chmod(f.fileno(), 0o640)
  16. File descriptors, PyCon US 2016 16 Directory file descriptor flags

    = os.O_RDONLY | os.O_DIRECTORY etcdir = os.open('/etc', flags) # read /etc/passwd stat = os.stat('passwd', dir_fd=etcdir)) os.close(etcdir) flags = os.O_RDONLY | os.O_DIRECTORY etcdir = os.open('/etc', flags) # read /etc/passwd stat = os.stat('passwd', dir_fd=etcdir)) os.close(etcdir)
  17. File descriptors, PyCon US 2016 17 Control hardware (silence please)

  18. File descriptors, PyCon US 2016 18

  19. File descriptors, PyCon US 2016 19 Open CD drive with

    Python import os, fcntl CDROMEJECT = 0x5309 # Linux specific! fd = os.open('/dev/cdrom', os.O_RDONLY | os.O_NONBLOCK) fcntl.ioctl(fd, CDROMEJECT, 0) os.close(fd) import os, fcntl CDROMEJECT = 0x5309 # Linux specific! fd = os.open('/dev/cdrom', os.O_RDONLY | os.O_NONBLOCK) fcntl.ioctl(fd, CDROMEJECT, 0) os.close(fd)
  20. Operating System 101

  21. File descriptors, PyCon US 2016 21 Dark Ages application application

    application application disk disk memory memory network network
  22. File descriptors, PyCon US 2016 22 Modern Operating Systems application

    application application application disk disk memory memory network network Kernel Kernel
  23. File descriptors, PyCon US 2016 23 syscall

  24. File descriptors, PyCon US 2016 24 Simple example # read.py

    with open('example.txt') as f: print(f.read()) # read.py with open('example.txt') as f: print(f.read())
  25. File descriptors, PyCon US 2016 25 Simple example $ strace

    ./read.py open("example.txt", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0664, st_size=19, …}) = 0 lseek(3, 0, SEEK_CUR) = 0 read(3, "Python is awesome!\n", 20) = 19 read(3, "", 1) = 0 write(1, "Python is awesome!\n", 19) = 19 write(1, "\n", 1) = 1 close(3) = 0 $ strace ./read.py open("example.txt", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0664, st_size=19, …}) = 0 lseek(3, 0, SEEK_CUR) = 0 read(3, "Python is awesome!\n", 20) = 19 read(3, "", 1) = 0 write(1, "Python is awesome!\n", 19) = 19 write(1, "\n", 1) = 1 close(3) = 0
  26. File descriptors, PyCon US 2016 26 open file table (global)

  27. File descriptors, PyCon US 2016 27 file descriptor table (process)

  28. File descriptors, PyCon US 2016 28 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 open file table entry 23 ... ... ... ... process 2 file descriptor table open 'example.txt'
  29. File descriptors, PyCon US 2016 29 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 open file table entry 23 ... entry 117 ... ... process 2 file descriptor table open 'example.txt' a second time
  30. File descriptors, PyCon US 2016 30 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 fd 5 open file table entry 23 ... entry 117 ... ... process 2 file descriptor table duplicate file descriptor 4
  31. File descriptors, PyCon US 2016 31 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 fd 5 open file table ... ... entry 117 ... ... process 2 file descriptor table rename and close
  32. File descriptors, PyCon US 2016 32 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 fd 5 open file table entry 23 ... entry 117 ... ... process 2 file descriptor table fd 7 send file descriptor to another process
  33. File descriptors, PyCon US 2016 33 process and global state

    file descriptor table maps to open file table cloexec flags open file table position mode owner locks credentials reference count ...
  34. File descriptors, PyCon US 2016 34

  35. File descriptors, PyCon US 2016 35 Spawn a new process

    fork & exec
  36. File descriptors, PyCon US 2016 36 fork() • creates a

    clone of the current process • inherits copy of file descriptor table • inherits copy of memory • shallow copy with Copy-on-Write
  37. File descriptors, PyCon US 2016 37 fork() example import os

    f = open('example.txt', mode='rb', buffering=0) pid = os.fork() print("getpid {:>5}, fork {:>5}, fd {}: {}".format( os.getpid(), pid, f.fileno(), f.read(6))) f.close() $ python3 fork.py getpid 17364, fork 17365, fd 3: b'Python' getpid 17365, fork 0, fd 3: b' is aw' import os f = open('example.txt', mode='rb', buffering=0) pid = os.fork() print("getpid {:>5}, fork {:>5}, fd {}: {}".format( os.getpid(), pid, f.fileno(), f.read(6))) f.close() $ python3 fork.py getpid 17364, fork 17365, fd 3: b'Python' getpid 17365, fork 0, fd 3: b' is aw'
  38. File descriptors, PyCon US 2016 38 exec() • replaces current

    program code • file descriptors are inherited by default • unless cloexec flag is set • potential security risk • PEP 446 -- Make newly created file descriptors non-inheritable (Victor Stinner)
  39. File descriptors, PyCon US 2016 39 Summary • applications must

    go through the Kernel • syscalls call into kernel space • file descriptors point to global table • global tables contains mode and position • new process with fork() and exec()
  40. subprocess.PIPE $ ls | grep py

  41. File descriptors, PyCon US 2016 41

  42. File descriptors, PyCon US 2016 43 pipe example import os

    readend, writeend = os.pipe() pid = os.fork() if pid != 0: # parent process os.close(writeend) with open(readend) as f: print(f.read()) else: # child process os.close(readend) os.dup2(writeend, 1) # 1: stdout os.execl('/bin/ls', # programm 'ls', '-l', 'example.txt') # argv 0, 1, 2 import os readend, writeend = os.pipe() pid = os.fork() if pid != 0: # parent process os.close(writeend) with open(readend) as f: print(f.read()) else: # child process os.close(readend) os.dup2(writeend, 1) # 1: stdout os.execl('/bin/ls', # programm 'ls', '-l', 'example.txt') # argv 0, 1, 2 1 2 3 4 6 5 7
  43. network sockets

  44. File descriptors, PyCon US 2016 45

  45. File descriptors, PyCon US 2016 46

  46. File descriptors, PyCon US 2016 47 socket parameters • addressing

    / routing • IPv4: AF_INET • IPv6: AF_INET6 • flow control • TCP: SOCK_STREAM • UDP: SOCK_DGRAM
  47. File descriptors, PyCon US 2016 48 socket server from socket

    import socket, SOCK_STREAM,AF_INET server = socket(AF_INET, SOCK_STREAM) server.bind(('0.0.0.0', 443)) server.listen(1) while True: conn, addr = server.accept() from socket import socket, SOCK_STREAM,AF_INET server = socket(AF_INET, SOCK_STREAM) server.bind(('0.0.0.0', 443)) server.listen(1) while True: conn, addr = server.accept()
  48. File descriptors, PyCon US 2016 49 socket client from socket

    import (socket, SOCK_STREAM, AF_INET, AF_INET6) cl4 = socket(AF_INET, SOCK_STREAM) cl4.connect(('104.130.43.121', 443)) cl6 = socket(AF_INET6, SOCK_STREAM) cl6.connect(('2001:4802:7901:0:e60a:1375:0:5', 443)) from socket import (socket, SOCK_STREAM, AF_INET, AF_INET6) cl4 = socket(AF_INET, SOCK_STREAM) cl4.connect(('104.130.43.121', 443)) cl6 = socket(AF_INET6, SOCK_STREAM) cl6.connect(('2001:4802:7901:0:e60a:1375:0:5', 443))
  49. Unix Sockets (Unix Domain Sockets / Local Sockets)

  50. File descriptors, PyCon US 2016 51

  51. File descriptors, PyCon US 2016 52

  52. File descriptors, PyCon US 2016 53 Unix sockets from socket

    import (socket, socketpair, SOCK_STREAM, AF_UNIX) server = socket(AF_UNIX, SOCK_STREAM) server.bind('/path/to/file') client = socket(AF_UNIX, SOCK_STREAM) client.connect('/path/to/file') a, b = socketpair() from socket import (socket, socketpair, SOCK_STREAM, AF_UNIX) server = socket(AF_UNIX, SOCK_STREAM) server.bind('/path/to/file') client = socket(AF_UNIX, SOCK_STREAM) client.connect('/path/to/file') a, b = socketpair()
  53. File descriptors, PyCon US 2016 54 peer credentials import socket,

    struct def getpeercred(sock): fmt = "iII" size = struct.calcsize(fmt) raw = sock.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, size) pid, uid, gid = struct.unpack(fmt, raw) return pid, uid, gid >>> getpeercred(uds) (31362, 0, 0) import socket, struct def getpeercred(sock): fmt = "iII" size = struct.calcsize(fmt) raw = sock.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, size) pid, uid, gid = struct.unpack(fmt, raw) return pid, uid, gid >>> getpeercred(uds) (31362, 0, 0)
  54. File descriptors, PyCon US 2016 55 peer security context import

    socket SO_PEERSEC = getattr(socket, 'SO_PEERSEC', 31) def getpeersec(sock): raw = sock.getsockopt(socket.SOL_SOCKET, SO_PEERSEC, 256) return raw.rstrip(b'\x00').decode('utf-8') >>> getpeersec(uds) 'system_u:system_r:svirt_lxc_net_t:s0:c560,c872' import socket SO_PEERSEC = getattr(socket, 'SO_PEERSEC', 31) def getpeersec(sock): raw = sock.getsockopt(socket.SOL_SOCKET, SO_PEERSEC, 256) return raw.rstrip(b'\x00').decode('utf-8') >>> getpeersec(uds) 'system_u:system_r:svirt_lxc_net_t:s0:c560,c872'
  55. File descriptors, PyCon US 2016 56 Unix sockets and containers

    • Unix sockets work between containers • SELinux! • Kernel translates PID name space • pid 31362 • Multi-Category Security (MCS) Separation • system_u:system_r:svirt_lxc_net_t:s0:c560,c872
  56. File descriptors, PyCon US 2016 57 Docker # cat /proc/31336/cgroup

    7:pids:/system.slice/docker-de20f4cb....scope # docker inspect de20f4cb... ... "ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c560,c872", ... "Config": { "Labels": { "io.kubernetes.container.name": "custodia-pwmgr", "io.kubernetes.pod.name": "custodia-pwmgr-tqxtz", "io.kubernetes.pod.namespace": "default", ... }} # cat /proc/31336/cgroup 7:pids:/system.slice/docker-de20f4cb....scope # docker inspect de20f4cb... ... "ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c560,c872", ... "Config": { "Labels": { "io.kubernetes.container.name": "custodia-pwmgr", "io.kubernetes.pod.name": "custodia-pwmgr-tqxtz", "io.kubernetes.pod.namespace": "default", ... }}
  57. File descriptors, PyCon US 2016 58 sending file descriptors #

    sender with open(filename) as f: send_fds(sender, b'\x01', [f.fileno()]) # receiver msg, fds = recv_fds(rcv, msglen=1, maxfds=1) with open(fds[0]) as f: f.read() # sender with open(filename) as f: send_fds(sender, b'\x01', [f.fileno()]) # receiver msg, fds = recv_fds(rcv, msglen=1, maxfds=1) with open(fds[0]) as f: f.read()
  58. File descriptors, PyCon US 2016 59 seccomp sandboxing syscall filter

    OpenSSH, Tor, Firefox media plugins, Google Chrome sandbox disk disk network network broker broker • read, write, close • open, socket, unlink • fork, exec
  59. Beyond

  60. File descriptors, PyCon US 2016 61 memory mapped I/0 •

    mmap • numpy.memmap
  61. File descriptors, PyCon US 2016 62 memory fd • temporary

    file in memory • file sealing
  62. File descriptors, PyCon US 2016 63 Efficient I/O with zero-copy

    • sendfile() • copy_file_range() • splice() / vmsplice() / tee() • AF_KTLS (Kernel space TLS)
  63. File descriptors, PyCon US 2016 64 event-driven I/O • select

    • poll • epoll
  64. File descriptors, PyCon US 2016 65 Open Space 4pm, room

    C120+C121 @ChristianHeimes / christian@python.org
  65. THANK YOU plus.google.com/+RedHat linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHatNews

  66. Images sources • https://commons.wikimedia.org/wiki/File:Washington,_D.C._Miss_Helen_ Ringwald_works_with_the_pneumatic_tubes.jpg ( • https://commons.wikimedia.org/wiki/File:Rohrpostbuechse.jpg (CC BY-

    SA 3.0) • https://commons.wikimedia.org/wiki/File:Datasette_c2n.jpg (CC BY-SA 3.0)