Systems programming is for everyone

Systems programming is for everyone

80c37b7308df099b8b9ec660146cf3da?s=128

Julia Evans

July 02, 2016
Tweet

Transcript

  1. SYSTEMS PROGRAMMING IS FOR EVERYONE by Julia Evans Stripe twitter:

    @b0rk blog: jvns.ca Tweet questions to @b0rk
  2. ὁὁ bugs ὁὁ

  3. perl | go | c++ | fortran php | python

    | java | smalltalk INTERCAL | BASIC | erlang
  4. MOSTLY LINUX ὂ

  5. YOUR PROGRAM = BLACK BOX

  6. DEBUGGING: look at the source code add print statements know

    the programming language
  7. DEBUGGING: look at the source code add print statements know

    the programming language ★★★ be a wizard★★★
  8. None
  9. THIS TALK Wizard school (or, an operating systems primer) Chapter

    1: The Case of the Mystery Config File Chapter 2: The Case of the French Website Chapter 3: The Case of the Slow Program
  10. WIZARD SCHOOL -OR- WHY YOU SHOULD ❤ YOUR OPERATING SYSTEM

  11. WHAT IS AN OPERATING SYSTEM FOR?

  12. When I go to http://google.com, kernel code runs for: Typing

    in the address Handling every network packet Writing history files to disk Allocating memory Communicating with the graphics card
  13. HOW TO CALL OPERATING SYSTEM CODE

  14. ★★★ SYSTEM CALLS!!! ★★★

  15. SYSTEM CALLS: AN OS'S INTERFACE open a file! (o p

    e n ) start a program! (e x e c v e ) change a file's permissions! (c h m o d )
  16. WHAT WE'VE LEARNED Your OS does tons of stuff Programs

    tell it what to do using system calls
  17. USING SYSTEMS KNOWLEDGE TO DEBUG

  18. CHAPTER 1: THE CASE OF THE MYSTERY CONFIG FILE

  19. Does bash use . b a s h _ p

    r o f i l e or . b a s h r c ??!??
  20. STRACE = ⚡ WIZARDRY ⚡

  21. STRACE = TRACING SYSTEM CALLS

  22. HOW TO STRACE $ s t r a c e

    g o o g l e - c h r o m e e x e c v e ( " / u s r / b i n / g o o g l e - c h r o m e " , [ " g o o g l e - c h r o m e " ] , [ / * 5 1 v a r s * / ] b r k ( 0 ) = 0 x 1 2 4 f 0 0 0 a c c e s s ( " / e t c / l d . s o . n o h w c a p " , F _ O K ) = - 1 E N O E N T ( N o s u c h f i l e o
  23. None
  24. OPEN strace ­e open bash

  25. BASHRC WINS!

  26. OTHER AWESOME SYSTEM CALLS w r i t e for

    log files e x e c v e for starting programs r e c v f r o m for receiving data
  27. STRACE ZINE

  28. OPENSNOOP LINUX ὂ & OS X ἴ ! https://github.com/iovisor/bcc

  29. opensnoop(8) NAME opensnoop ­ Trace open() syscalls. Uses Linux eBPF/bcc.

    ... AUTHOR Brendan Gregg
  30. CHAPTER 2: THE CASE OF THE POLISH WEBSITE

  31. None
  32. None
  33. ???

  34. NETWORK SPYING TO THE RESCUE

  35. TCPDUMP sudo tcpdump port 5000 ­i any ­o polish­website.pcap

  36. WIRESHARK wireshark polish­website.pcap

  37. Accept­Language: en­US,en;q=0.5\r\n

  38. None
  39. None
  40. None
  41. None
  42. None
  43. None
  44. WIRESHARK: ASSIGNER OF BLAME

  45. why tcpdump is fast: BPF (berkeley packet filter) src ip

    192.168.0.1 or dst ip 192.168.0.1
  46. NETWORK SPYING TOOLS ngrep tcpdump wireshark mitmproxy

  47. CHAPTER 3: THE CASE OF THE SLOW PROGRAM

  48. 3 SLOW PROGRAMS 1. CPU time 2. too many writes

    3. waiting for a slow server
  49. MYSTERY PROGRAM #1

  50. $ time python mystery_1.py 0.09user 0.01system 0:02.11elapsed 5%CPU

  51. WHAT IS IT WAITING FOR?

  52. LET'S LOOK INTO THE KERNEL'S SOUL

  53. /PROC/P I D /STACK $ p g r e p

    - f m y s t e r y _ 1 3 1 7 2 8 $ s u d o c a t / p r o c / 3 1 7 2 8 / s t a c k [ < f f f f f f f f 8 1 7 0 2 4 6 7 > ] s k _ w a i t _ d a t a + 0 x 1 0 7 / 0 x 1 2 0 [ < f f f f f f f f 8 1 7 6 7 1 1 2 > ] t c p _ r e c v m s g + 0 x 2 e 2 / 0 x b 8 0 [ < f f f f f f f f 8 1 7 9 4 d 6 e > ] i n e t _ r e c v m s g + 0 x 7 e / 0 x b 0 [ < f f f f f f f f 8 1 6 f d b 6 b > ] s o c k _ r e c v m s g + 0 x 3 b / 0 x 5 0 [ < f f f f f f f f 8 1 6 f d d c 1 > ] S Y S C _ r e c v f r o m + 0 x e 1 / 0 x 1 6 0 [ < f f f f f f f f 8 1 6 f f 1 c e > ] S y S _ r e c v f r o m + 0 x e / 0 x 1 0 [ < f f f f f f f f 8 1 8 2 4 4 f 2 > ] e n t r y _ S Y S C A L L _ 6 4 _ f a s t p a t h + 0 x 1 6 / 0 x 7 1 [ < f f f f f f f f f f f f f f f f > ] 0 x f f f f f f f f f f f f f f f
  54. WE WIN! IT WAS THE NETWORK!

  55. OUR SERVER @app.route('/') def slow(): time.sleep(2) return "Hi!" app.run()

  56. MYSTERY PROGRAM #2 $ time python mystery_2.py 2.74user 0.00system 2.74

    seconds 99%CPU
  57. USE A PYTHON PROFILER

  58. total = 0 for i in xrange(14000000): total += i

  59. MYSTERY PROGRAM #3

  60. (REALLY A MYSTERY)

  61. $ time python mystery_3.py 0.03user 4.11system 4.40 seconds elapsed 94%CPU

  62. DEMO DEMO

  63. 29,81% [kernel] _aesni_enc1 1,88% [kernel] ecryptfs_write_begin ▒ 1,51% [kernel] raw_spin_lock

  64. MYSTERY SOLVED filename = '/home/bork/fake.txt'

  65. ❤ dstat ❤

  66. ❤ perf top ❤

  67. WORKS GREAT WITH THE JVM & NODE

  68. rust_mutex system_call_fastpath ttwu_.. en.. enq.. enqu.. futex_wait main::.. [unknown] _raw_..

    [unknown] futex_wake do_futex __ticket_spin_lock __schedule __lll_lock_wait wake_futex acti.. schedule sys_futex futex_wait_que.. dea.. __lll_unlock_wake wake_up_state _raw_spin_lock __tic.. __ticket_spin_lock de.. try_to_wake_up se.. _raw_spin_lock ttwu_.. system_call_fastpath deq.. d.. sys_futex futex_wait_setup do_futex
  69. YOUR PROGRAM = BLACK BOX

  70. THERE ARE A LOT OF AWESOME TOOLS

  71. LEARN YOUR OPERATING SYSTEM (or your browser.)

  72. Recurse Center

  73. PROGRAMMING PROGRAMMING EXPERIMENTS EXPERIMENTS

  74. RULES OF RULES OF PROGRAMMING PROGRAMMING EXPERIMENTS EXPERIMENTS it doesn't

    have to work 1. you don't have to finish it 2. you have to learn something 3.
  75. EXPERIMENT 1: WRITE AN EXPERIMENT 1: WRITE AN OPERATING SYSTEM

    OPERATING SYSTEM remember it doesn't have to work
  76. WHAT I LEARNED WHAT I LEARNED having more than once

    process is hard filesystems are hard operating systems are SO COMPLICATED
  77. EXPERIMENT 2: EXPERIMENT 2: DATABASE SURGERY DATABASE SURGERY

  78. HOW DOES SQLITE HOW DOES SQLITE WORK? WORK?

  79. FUN.SQLITE FUN.SQLITE id | word 1 | greedy 2 |

    greediness 3 | greener
  80. $ hexdump fun.sqlite |.............{.n| |.a.R.D.4.%......| |................| |...y.n._.N.>.,.$| |................| |..............F.| |..EAcevedo.E...D|

    |Accra's.D...CAcc| |ra.C..#BAccentur| |e's.B...AAccentu| |re.A..!@Acapulco| |'s.@...?Acapulco|
  81. A DATABASE IS A A DATABASE IS A TREE TREE

  82. static MemPage *btreePageFromDbPage(DbPage *pDbPage, Pgno pgno, BtSha // actual code

    printf("Read a btree page, page number %d\n", pgno); // added by me // actual code }
  83. sqlite> select * from fun where id = 1; Read

    a btree page, page number 1 Read a btree page, page number 5 Read a btree page, page number 828 Read a btree page, page number 10 Read a btree page, page number 2 Read a btree page, page number 76 Read a btree page, page number 6 1|A's
  84. sqlite> select * from fun where id = 20; Read

    a btree page, page number 1 Read a btree page, page number 5 Read a btree page, page number 828 Read a btree page, page number 10 Read a btree page, page number 2 Read a btree page, page number 76 Read a btree page, page number 6 20|Aaliyah
  85. sqlite> select * from fun where id = 80000; Read

    a btree page, page number 1 Read a btree page, page number 5 Read a btree page, page number 1198 Read a btree page, page number 992 Read a btree page, page number 2 Read a btree page, page number 1813 Read a btree page, page number 449 80000|scarfs
  86. WHAT I LEARNED WHAT I LEARNED databases tables are trees

    databases are made of pages i can read some of the SQLite source code!
  87. EXPERIMENT 3: EXPERIMENT 3: WRITE A TCP STACK WRITE A

    TCP STACK
  88. EXPERIMENT 3: EXPERIMENT 3: WRITE A TCP STACK WRITE A

    TCP STACK IN PYTHON IN PYTHON
  89. None
  90. ip_header = IP(dst=dest_ip, src=src_ip) syn = TCP(dport=80, sport=59333, ack=0, flags="S")

    # Send the SYN packet to Google response = srp(ip_header + syn)
  91. WHAT I LEARNED WHAT I LEARNED how TCP packets are

    put together! you can write a 10% working TCP from scratch in 2 weeks python can't keep up
  92. EXPERIMENT 4: EXPERIMENT 4: CONCURRENCY CONCURRENCY

  93. int counter; void *AddThings(void *threadid) { for (int i =

    0; i < 10000; i++) counter += 1; pthread_exit(NULL); }
  94. WRONG ANSWER WRONG ANSWER

  95. MUTEX MUTEX pthread_mutex_lock(&mutex); counter += 1;

  96. "ATOM" "ATOM" __sync_add_and_fetch(&counter, 1);

  97. WHAT I LEARNED WHAT I LEARNED atoms are faster than

    mutexes
  98. I BLOG MY EXPERIMENTS I BLOG MY EXPERIMENTS

  99. "can you discuss the pros and cons of using a

    lock-free approach for implementing a thread-safe hashmap?"
  100. DO ENOUGH DO ENOUGH EXPERIMENTS EXPERIMENTS END UP WITH ACTUAL

    END UP WITH ACTUAL KNOWLEDGE KNOWLEDGE
  101. THANKS! Julia Evans twitter: @b0rk learn more by reading my

    blog: http://jvns.ca Come ask me questions & get a zine!