Julia Evans - Systems programming as a swiss army knife

Slide 1

Slide 1 text

SPYING ON YOUR PROGRAMS by Julia Evans Stripe twitter: @b0rk blog: jvns.ca Tweet questions to @b0rk

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Slide 4

Slide 4 text

LINUX-ONLY

Slide 5

Slide 5 text

YOUR PROGRAM = BLACK BOX

Slide 6

Slide 6 text

DEBUGGING: look at the source code add print statements know the programming language

Slide 7

Slide 7 text

DEBUGGING: look at the source code add print statements know the programming language ★★★ be a wizard★★★

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

THIS TALK Wizard school (or, an operating systems primer) Chapter 1: The Case of the Mystery Config File Chapter 2: The Case of the French Website Chapter 3: The Case of the Slow Program

Slide 10

Slide 10 text

WIZARD SCHOOL -OR- WHY YOU SHOULD ❤ YOUR OPERATING SYSTEM

Slide 11

Slide 11 text

WHAT IS AN OPERATING SYSTEM FOR?

Slide 12

Slide 12 text

When I go to http://google.com, kernel code runs for: Typing in the address Handling every network packet Writing history files to disk Allocating memory Communicating with the graphics card

Slide 13

Slide 13 text

HOW TO CALL OPERATING SYSTEM CODE

Slide 14

Slide 14 text

★★★ SYSTEM CALLS!!! ★★★

Slide 15

Slide 15 text

SYSTEM CALLS: AN OS'S INTERFACE open a file! (o p e n ) start a program! (e x e c v e ) change a file's permissions! (c h m o d )

Slide 16

Slide 16 text

WHAT WE'VE LEARNED Your OS does tons of stuff Programs tell it what to do using system calls

Slide 17

Slide 17 text

USING SYSTEMS KNOWLEDGE TO DEBUG

Slide 18

Slide 18 text

CHAPTER 1: THE CASE OF THE MYSTERY CONFIG FILE

Slide 19

Slide 19 text

Does bash use . b a s h _ p r o f i l e or . b a s h r c ??!??

Slide 20

Slide 20 text

STRACE = WIZARDRY

Slide 21

Slide 21 text

STRACE = TRACING SYSTEM CALLS

Slide 22

Slide 22 text

HOW TO STRACE $ s t r a c e g o o g l e - c h r o m e e x e c v e ( " / u s r / b i n / g o o g l e - c h r o m e " , [ " g o o g l e - c h r o m e " ] , [ / * 5 1 v a r s * / ] ) b r k ( 0 ) = 0 x 1 2 4 f 0 0 0 a c c e s s ( " / e t c / l d . s o . n o h w c a p " , F _ O K ) = - 1 E N O E N T ( N o s u c h f i l e o r

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

OPEN s t r a c e - e o p e n b a s h

Slide 25

Slide 25 text

BASHRC WINS!

Slide 26

Slide 26 text

OTHER AWESOME SYSTEM CALLS w r i t e for log files e x e c v e for starting programs r e c v f r o m for receiving data

Slide 27

Slide 27 text

STRACE ZINE

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

CHAPTER 2: THE CASE OF THE FRENCH WEBSITE

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

???

Slide 34

Slide 34 text

NETWORK SPYING TO THE RESCUE

Slide 35

Slide 35 text

s u d o n g r e p - d l o 5 0 0 0 i n t e r f a c e : l o ( 1 2 7 . 0 . 0 . 0 / 2 5 5 . 0 . 0 . 0 ) m a t c h : 5 0 0 0 # # # # T 1 2 7 . 0 . 0 . 1 : 4 5 4 3 8 - > 1 2 7 . 0 . 0 . 1 : 5 0 0 0 [ A P ] G E T / H T T P / 1 . 1 . . H o s t : l o c a l h o s t : 5 0 0 0 . . C o n n e c t i o n : k e e p - a l i v e . . C a c h e - C o n t r o l : m a x - a g e = 0 . . A c c e p t : t e x t / h t m l , a p p l i c a t i o n / x h t m l + x m l , a p p l i c a t i o n / x m l ; q = 0 . 9 , i m a g e / w e b p , * / * ; q = 0 . 8 . . U s e r - A g e n t : M o z i l l a / 5 . 0 ( X 1 1 ; L i n u x x 8 6 _ 6 4 ) A p p l e W e b K i t / 5 3 7 . 3 6 ( K H T M L , l i k e G e c k o ) C h r o m e / 4 1 . 0 . 2 2 7 2 . 5 3 S a r i / 5 3 7 . 3 6 . . D N T : 1 . . A c c e p t - E n c o d i n g : g z i p , d e f l a t e , s d c h . . A c c e p t - L a n g u a g e : e n - U S , e n ; q = 0 . 8 . . C o o k i e : u s e r n a m e - l o c a l h o s t - 8 8 8 8 = " 2 | 1 : 0 | 1 0 : 1 4 2 8 4 1 1 8 7 9 | 2 3 : u s e r n a m e - l o c a l h o s t - 8 8 8 8 | 4 8 : M j Y z M T c 2 N G M t Y T A 1 M C 0 0 Y j N k L T k y Y T k t N f a b 7 e e 2 7 9 " . . . . # # # # # # # # # # # # # # # # # # # # # # # T 1 2 7 . 0 . 0 . 1 : 4 5 4 4 0 - > 1 2 7 . 0 . 0 . 1 : 5 0 0 0 [ A P ] G E T / H T T P / 1 . 1 . . U s e r - A g e n t : c u r l / 7 . 2 2 . 0 ( x 8 6 _ 6 4 - p c - l i n u x - g n u ) l i b c u r l / 7 . 2 2 . 0 O p e n S S L / 1 . 0 . 1 z l i b / 1 . 2 . 3 . 4 l i b i d n / 1 . 2 3 l i b r t m p / 2 . 3 . . H o s t : l o c a l h o s t : 5 0 0 0 . . A c c e p t : * / * . . . . # # # # # # # # # # # # # # # # # #

Slide 36

Slide 36 text

A c c e p t - L a n g u a g e : e n - U S

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

NETWORK SPYING TOOLS ngrep tcpdump wireshark mitmproxy

Slide 39

Slide 39 text

CHAPTER 3: THE CASE OF THE SLOW PROGRAM

Slide 40

Slide 40 text

3 SLOW PROGRAMS 1. CPU time 2. too many writes 3. waiting for a slow server

Slide 41

Slide 41 text

MYSTERY PROGRAM #1

Slide 42

Slide 42 text

$ t i m e p y t h o n m y s t e r y _ 1 . p y 0 . 0 9 u s e r 0 . 0 1 s y s t e m 0 : 0 2 . 1 1 e l a p s e d 5 % C P U

Slide 43

Slide 43 text

WHAT IS IT WAITING FOR?

Slide 44

Slide 44 text

LET'S LOOK INTO THE KERNEL'S SOUL

Slide 45

Slide 45 text

/PROC/P I D /STACK $ p g r e p - f m y s t e r y _ 1 3 1 7 2 8 $ s u d o c a t / p r o c / 3 1 7 2 8 / s t a c k [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 c 0 3 9 > ] s k _ w a i t _ d a t a + 0 x d 9 / 0 x e 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 9 8 b d f > ] t c p _ r e c v m s g + 0 x 6 7 f / 0 x b 5 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 c 1 7 2 b > ] i n e t _ r e c v m s g + 0 x 6 b / 0 x 8 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 7 8 9 5 > ] s o c k _ r e c v m s g + 0 x c 5 / 0 x e 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 6 3 7 9 9 e > ] S Y S C _ r e c v f r o m + 0 x e e / 0 x 1 7 0 [ < f f f f f f f f 8 1 6 3 8 7 1 e > ] S y S _ r e c v f r o m + 0 x e / 0 x 1 0 [ < f f f f f f f f 8 1 7 6 d 5 0 5 > ] r e t u r n _ t o _ h a n d l e r + 0 x 0 / 0 x 2 b [ < f f f f f f f f 8 1 7 6 d 6 6 d > ] s y s t e m _ c a l l _ f a s t p a t h + 0 x 1 a / 0 x 1 f [ < f f f f f f f f f f f f f f f f > ] 0 x f f f f f f f f f f f f f f f f

Slide 46

Slide 46 text

WE WIN! IT WAS THE NETWORK!

Slide 47

Slide 47 text

OUR SERVER @ a p p . r o u t e ( ' / ' ) d e f s l o w ( ) : t i m e . s l e e p ( 2 ) r e t u r n " H i ! " a p p . r u n ( )

Slide 48

Slide 48 text

MYSTERY PROGRAM #2 $ t i m e p y t h o n m y s t e r y _ 2 . p y 2 . 7 4 u s e r 0 . 0 0 s y s t e m 0 : 0 2 . 7 4 e l a p s e d 9 9 % C P U

Slide 49

Slide 49 text

USE A PYTHON PROFILER

Slide 50

Slide 50 text

t o t a l = 0 f o r i i n x r a n g e ( 1 4 0 0 0 0 0 0 ) : t o t a l + = i

Slide 51

Slide 51 text

MYSTERY PROGRAM #3

Slide 52

Slide 52 text

(REALLY A MYSTERY)

Slide 53

Slide 53 text

$ t i m e p y t h o n m y s t e r y _ 3 . p y 0 : 0 2 . 6 1 e l a p s e d 6 2 % C P U $ t i m e p y t h o n m y s t e r y _ 3 . p y 0 : 1 0 . 6 1 e l a p s e d 1 0 % C P U

Slide 54

Slide 54 text

DEMO DEMO

Slide 55

Slide 55 text

WE WIN

Slide 56

Slide 56 text

YOUR PROGRAM = BLACK BOX

Slide 57

Slide 57 text

THERE ARE A LOT OF AWESOME TOOLS

Slide 58

Slide 58 text

LEARN YOUR OPERATING SYSTEM

Slide 59

Slide 59 text

Hacker School Recurse Center

Slide 60

Slide 60 text

THANKS! Julia Evans twitter: @b0rk learn more by reading my blog: http://jvns.ca Come get a strace zine!!!!!