Slide 1

Slide 1 text

See Python, See Python Go, Go Python Go Presented by Andrey Petrov at PyCon 2016 Andrey Petrov · @shazow

Slide 2

Slide 2 text

Andrey Petrov @shazow urllib3, ssh-chat, briefmetrics, colorblendy, tweepsect, funny tweets, and more Andrey Petrov · @shazow

Slide 3

Slide 3 text

Do you Go? Andrey Petrov · @shazow

Slide 4

Slide 4 text

Do you Go? Simple syntax that is easy to learn Andrey Petrov · @shazow

Slide 5

Slide 5 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Andrey Petrov · @shazow

Slide 6

Slide 6 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Statically typed Andrey Petrov · @shazow

Slide 7

Slide 7 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Statically typed Statically linked binaries Andrey Petrov · @shazow

Slide 8

Slide 8 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Statically typed Statically linked binaries Cross-compiles to ~every platform Andrey Petrov · @shazow

Slide 9

Slide 9 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Statically typed Statically linked binaries Cross-compiles to ~every platform Easy concurrency Andrey Petrov · @shazow

Slide 10

Slide 10 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Statically typed Statically linked binaries Cross-compiles to ~every platform Easy concurrency Great standard library Andrey Petrov · @shazow

Slide 11

Slide 11 text

Do you Go? Simple syntax that is easy to learn Compiles superfast Statically typed Statically linked binaries Cross-compiles to ~every platform Easy concurrency Great standard library Gophers! Andrey Petrov · @shazow

Slide 12

Slide 12 text

Python Go Andrey Petrov · @shazow Credit: Renee French & Alice Tribuleva

Slide 13

Slide 13 text

Go in Python i m p o r t o s o s . s y s t e m ( " g o r u n m a i n . g o " ) Andrey Petrov · @shazow

Slide 14

Slide 14 text

Go in Python i m p o r t o s o s . s y s t e m ( " g o r u n m a i n . g o " ) lol just kidding Andrey Petrov · @shazow

Slide 15

Slide 15 text

Fun Facts Python speaks with C Andrey Petrov · @shazow

Slide 16

Slide 16 text

Fun Facts Python speaks with C Go speaks with C Andrey Petrov · @shazow

Slide 17

Slide 17 text

Fun Facts Python speaks with C Go speaks with C Therefore, Python speaks with Go? Andrey Petrov · @shazow

Slide 18

Slide 18 text

Challenges 1. Runtime barriers Andrey Petrov · @shazow

Slide 19

Slide 19 text

Challenges 1. Runtime barriers Garbage Collectors (GC) Global Interpreter Lock (GIL) Just In Time compiling (JIT) Resource Pools (Threads) Andrey Petrov · @shazow

Slide 20

Slide 20 text

Challenges 1. Runtime barriers Garbage Collectors (GC) Global Interpreter Lock (GIL) Just In Time compiling (JIT) Resource Pools (Threads) 2. Syntax and feature barriers Andrey Petrov · @shazow

Slide 21

Slide 21 text

Challenges 1. Runtime barriers Garbage Collectors (GC) Global Interpreter Lock (GIL) Just In Time compiling (JIT) Resource Pools (Threads) 2. Syntax and feature barriers Go: Interface, Goroutines, etc. Python: Classes, Generators, etc. Oodles of other language-specific constructs Andrey Petrov · @shazow

Slide 22

Slide 22 text

Go runtime Python runtime C GIL GC Interpreter Ponies GC Goroutines Gophers Andrey Petrov · @shazow

Slide 23

Slide 23 text

Let's take a moment and imagine... Andrey Petrov · @shazow

Slide 24

Slide 24 text

Running a webserver in Go p a c k a g e m a i n i m p o r t ( " f m t " " n e t / h t t p " ) f u n c i n d e x ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { f m t . F p r i n t f ( w , " H e l l o , w o r l d . \ n " ) } f u n c m a i n ( ) { h t t p . H a n d l e F u n c ( " / " , i n d e x ) h t t p . L i s t e n A n d S e r v e ( " 1 2 7 . 0 . 0 . 1 : 5 0 0 0 " , n i l ) } Andrey Petrov · @shazow

Slide 25

Slide 25 text

Running a webserver in Python f r o m f l a s k i m p o r t F l a s k a p p = F l a s k ( _ _ n a m e _ _ ) @ a p p . r o u t e ( ' / ' ) d e f i n d e x ( ) : r e t u r n ' H e l l o , w o r l d ! \ n ' i f _ _ n a m e _ _ = = ' _ _ m a i n _ _ ' : a p p . r u n ( h o s t = ' 1 2 7 . 0 . 0 . 1 ' , p o r t = 5 0 0 0 ) Andrey Petrov · @shazow

Slide 26

Slide 26 text

Running a Go webserver in Python?? f r o m g o h t t p i m p o r t r o u t e , r u n @ r o u t e ( ' / ' ) d e f i n d e x ( w , r e q ) : w . w r i t e ( " H e l l o , w o r l d . \ n " ) i f _ _ n a m e _ _ = = ' _ _ m a i n _ _ ' : r u n ( h o s t = ' 1 2 7 . 0 . 0 . 1 ' , p o r t = 5 0 0 0 ) Andrey Petrov · @shazow

Slide 27

Slide 27 text

Running a Go webserver in Python?? f r o m g o h t t p i m p o r t r o u t e , r u n @ r o u t e ( ' / ' ) d e f i n d e x ( w , r e q ) : w . w r i t e ( " H e l l o , w o r l d . \ n " ) i f _ _ n a m e _ _ = = ' _ _ m a i n _ _ ' : r u n ( h o s t = ' 1 2 7 . 0 . 0 . 1 ' , p o r t = 5 0 0 0 ) Compare f r o m f l a s k i m p o r t F l a s k a p p = F l a s k ( _ _ n a m e _ _ ) @ a p p . r o u t e ( ' / ' ) d e f i n d e x ( ) : r e t u r n ' H e l l o , w o r l d ! \ n ' i f _ _ n a m e _ _ = = ' _ _ m a i n _ _ ' : a p p . r u n ( h o s t = ' 1 2 7 . 0 . 0 . 1 ' , p o r t = 5 0 0 0 ) Andrey Petrov · @shazow

Slide 28

Slide 28 text

Comparing handlers Go (net/http) f u n c i n d e x ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { f m t . F p r i n t f ( w , " H e l l o , w o r l d . \ n " ) } Go in Python (gohttplib) d e f i n d e x ( w , r e q ) : w . w r i t e ( " H e l l o , w o r l d . \ n " ) Python (flask) d e f i n d e x ( ) : r e t u r n ' H e l l o , w o r l d ! \ n ' Andrey Petrov · @shazow

Slide 29

Slide 29 text

gohttplib This is an actual functioning thing. github.com/shazow/gohttplib Andrey Petrov · @shazow

Slide 30

Slide 30 text

gohttplib This is an actual functioning thing. github.com/shazow/gohttplib Sponsored by Glider Labs Andrey Petrov · @shazow

Slide 31

Slide 31 text

Here's how it works... Andrey Petrov · @shazow

Slide 32

Slide 32 text

The Plan 1. Go: Export Go functions to a C shared library 2. C: 3. Python: Call C and wrap it in a Python-shaped bow 4. Make it actually work ¯\_(ツ)_/¯ Andrey Petrov · @shazow

Slide 33

Slide 33 text

Andrey Petrov · @shazow

Slide 34

Slide 34 text

Calling C from Go p a c k a g e m a i n / * i n t t h e _ a n s w e r ( ) { r e t u r n 4 2 ; } * / i m p o r t " C " i m p o r t " f m t " f u n c m a i n ( ) { r : = C . t h e _ a n s w e r ( ) f m t . P r i n t l n ( r ) } Andrey Petrov · @shazow

Slide 35

Slide 35 text

Calling C from Go p a c k a g e m a i n / * i n t t h e _ a n s w e r ( ) { r e t u r n 4 2 ; } * / i m p o r t " C " i m p o r t " f m t " f u n c m a i n ( ) { r : = C . t h e _ a n s w e r ( ) f m t . P r i n t l n ( r ) } $ g o b u i l d - o a n s w e r $ . / a n s w e r 4 2 Andrey Petrov · @shazow

Slide 36

Slide 36 text

Calling Go from C p a c k a g e m a i n i m p o r t " C " / / e x p o r t T h e A n s w e r f u n c T h e A n s w e r ( ) C . i n t { r e t u r n C . i n t ( 4 2 ) } f u n c m a i n ( ) { } Andrey Petrov · @shazow

Slide 37

Slide 37 text

Calling Go from C p a c k a g e m a i n i m p o r t " C " / / e x p o r t T h e A n s w e r f u n c T h e A n s w e r ( ) C . i n t { r e t u r n C . i n t ( 4 2 ) } f u n c m a i n ( ) { } $ g o b u i l d - b u i l d m o d e = c - s h a r e d - o l i b a n s w e r . s o Andrey Petrov · @shazow

Slide 38

Slide 38 text

Calling Go from C p a c k a g e m a i n i m p o r t " C " / / e x p o r t T h e A n s w e r f u n c T h e A n s w e r ( ) C . i n t { r e t u r n C . i n t ( 4 2 ) } f u n c m a i n ( ) { } $ g o b u i l d - b u i l d m o d e = c - s h a r e d - o l i b a n s w e r . s o # i n c l u d e < s t d i o . h > # i n c l u d e " l i b a n s w e r . h " i n t m a i n ( ) { i n t r = T h e A n s w e r ( ) ; p r i n t f ( " % d \ n " , r ) ; r e t u r n 0 ; } Andrey Petrov · @shazow

Slide 39

Slide 39 text

Andrey Petrov · @shazow

Slide 40

Slide 40 text

See CPython C CPython Extension Interface: no dependencies, but lots of boilerplate CFFI: a little more magic but does more work for us and more portable Andrey Petrov · @shazow

Slide 41

Slide 41 text

Calling C from Python # a n s w e r _ b u i l d . p y : f r o m c f f i i m p o r t F F I f f i = F F I ( ) f f i . c d e f ( " i n t t h e _ a n s w e r ( ) ; " ) f f i . s e t _ s o u r c e ( " _ a n s w e r " , " " " i n t t h e _ a n s w e r ( ) { r e t u r n 4 2 ; } " " " ) i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : f f i . c o m p i l e ( ) Andrey Petrov · @shazow

Slide 42

Slide 42 text

Calling C from Python # a n s w e r _ b u i l d . p y : f r o m c f f i i m p o r t F F I f f i = F F I ( ) f f i . c d e f ( " i n t t h e _ a n s w e r ( ) ; " ) f f i . s e t _ s o u r c e ( " _ a n s w e r " , " " " i n t t h e _ a n s w e r ( ) { r e t u r n 4 2 ; } " " " ) i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : f f i . c o m p i l e ( ) $ p y t h o n a n s w e r _ b u i l d . p y $ l s _ a n s w e r . c _ a n s w e r . o _ a n s w e r . s o a n s w e r _ b u i l d . p y Andrey Petrov · @shazow

Slide 43

Slide 43 text

Calling C from Python # a n s w e r _ b u i l d . p y : f r o m c f f i i m p o r t F F I f f i = F F I ( ) f f i . c d e f ( " i n t t h e _ a n s w e r ( ) ; " ) f f i . s e t _ s o u r c e ( " _ a n s w e r " , " " " i n t t h e _ a n s w e r ( ) { r e t u r n 4 2 ; } " " " ) i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : f f i . c o m p i l e ( ) $ p y t h o n a n s w e r _ b u i l d . p y $ l s _ a n s w e r . c _ a n s w e r . o _ a n s w e r . s o a n s w e r _ b u i l d . p y # a n s w e r . p y : f r o m _ a n s w e r i m p o r t l i b p r i n t ( l i b . t h e _ a n s w e r ( ) ) Andrey Petrov · @shazow

Slide 44

Slide 44 text

Calling Python from C Simple function pointer that can be used in C: @ f f i . c a l l b a c k ( " i n t ( i n t , i n t ) " ) d e f a d d ( x , y ) : r e t u r n x + y Andrey Petrov · @shazow

Slide 45

Slide 45 text

Calling Python from C Simple function pointer that can be used in C: @ f f i . c a l l b a c k ( " i n t ( i n t , i n t ) " ) d e f a d d ( x , y ) : r e t u r n x + y ~handwaving~ Andrey Petrov · @shazow

Slide 46

Slide 46 text

Calling Python from C Simple function pointer that can be used in C: @ f f i . c a l l b a c k ( " i n t ( i n t , i n t ) " ) d e f a d d ( x , y ) : r e t u r n x + y ~handwaving~ s t a t i c i n t ( * a d d ) ( i n t x , i n t b ) ; That's all we need for now. More on CFFI embedding here. Andrey Petrov · @shazow

Slide 47

Slide 47 text

Challenges 1. Runtime barriers Garbage Collectors (GC) Global Interpreter Lock (GIL) Just In Time compiling in PyPy (JIT) Thread Pools 2. Syntax and feature barriers Go: Interface, Goroutines, etc. Python: Classes, Generators, etc. Oodles of other language-specific constructs Andrey Petrov · @shazow

Slide 48

Slide 48 text

Overcoming Challenges 1. Don't share memory Andrey Petrov · @shazow

Slide 49

Slide 49 text

Go runtime Python runtime C GIL GC Interpreter Ponies GC Goroutines Gophers Andrey Petrov · @shazow

Slide 50

Slide 50 text

Challenge 1: Don't share memory h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { . . . } ) Our Python code will need to provide this callback, but we can't pass * h t t p . R e q u e s t to Python. Andrey Petrov · @shazow

Slide 51

Slide 51 text

Challenge 1: Don't share memory h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { . . . } ) Our Python code will need to provide this callback, but we can't pass * h t t p . R e q u e s t to Python. t y p e d e f s t r u c t R e q u e s t _ { c o n s t c h a r * M e t h o d ; c o n s t c h a r * H o s t ; . . . } R e q u e s t ; Andrey Petrov · @shazow

Slide 52

Slide 52 text

Challenge 1: Don't share memory h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { . . . } ) Our Python code will need to provide this callback, but we can't pass * h t t p . R e q u e s t to Python. t y p e d e f s t r u c t R e q u e s t _ { c o n s t c h a r * M e t h o d ; c o n s t c h a r * H o s t ; . . . } R e q u e s t ; h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { / / W r a p r e l e v a n t r e q u e s t f i e l d s i n a C - f r i e n d l y d a t a s t r u c t u r e . c r e q : = C . R e q u e s t { M e t h o d : C . C S t r i n g ( r e q . M e t h o d ) , H o s t : C . C S t r i n g ( r e q . H o s t ) , . . . } . . . Andrey Petrov · @shazow

Slide 53

Slide 53 text

Overcoming Challenges 1. Don't share memory 2. Translation layer Andrey Petrov · @shazow

Slide 54

Slide 54 text

Challenge 2: Translation layer h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { . . . } ) Our Python code will need to use the h t t p . R e s p o n s e W r i t e r interface. Andrey Petrov · @shazow

Slide 55

Slide 55 text

Challenge 2: Translation layer h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { . . . } ) Our Python code will need to use the h t t p . R e s p o n s e W r i t e r interface. R e s p o n s e W r i t e r . W r i t e ( [ ] b y t e ) ( i n t , e r r o r ) R e s p o n s e W r i t e r . W r i t e H e a d e r ( i n t ) Andrey Petrov · @shazow

Slide 56

Slide 56 text

Challenge 2: Translation layer h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { . . . } ) Our Python code will need to use the h t t p . R e s p o n s e W r i t e r interface. R e s p o n s e W r i t e r . W r i t e ( [ ] b y t e ) ( i n t , e r r o r ) R e s p o n s e W r i t e r . W r i t e H e a d e r ( i n t ) / / e x p o r t R e s p o n s e W r i t e r _ W r i t e f u n c R e s p o n s e W r i t e r _ W r i t e ( w P t r C . u i n t , c b u f * C . c h a r , l e n g t h C . i n t ) C . i n t { b u f : = C . G o B y t e s ( u n s a f e . P o i n t e r ( c b u f ) , l e n g t h ) . . . n , _ : = ( * ( * h t t p . R e s p o n s e W r i t e r ) ( w ) ) . W r i t e ( b u f ) r e t u r n C . i n t ( n ) } / / e x p o r t R e s p o n s e W r i t e r _ W r i t e H e a d e r f u n c R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) { . . . ( * ( * h t t p . R e s p o n s e W r i t e r ) ( w ) ) . W r i t e H e a d e r ( i n t ( h e a d e r ) ) } Andrey Petrov · @shazow

Slide 57

Slide 57 text

Challenge 2: Translation layer (Cont.) We exported these functions in Go R e s p o n s e W r i t e r _ W r i t e ( w P t r C . u i n t , c b u f * C . c h a r , l e n g t h C . i n t ) C . i n t R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) Now we can wrap them in Python l i b = f f i . d l o p e n ( . . . ) c l a s s R e s p o n s e W r i t e r : d e f _ _ i n i t _ _ ( s e l f , w ) : s e l f . _ w = w d e f w r i t e ( s e l f , b o d y ) : n = l i b . R e s p o n s e W r i t e r _ W r i t e ( s e l f . _ w , b o d y , l e n ( b o d y ) ) i f n ! = l e n ( b o d y ) : r a i s e I O E r r o r ( " F a i l e d t o w r i t e t o R e s p o n s e W r i t e r . " ) d e f s e t _ s t a t u s ( s e l f , c o d e ) : l i b . R e s p o n s e W r i t e r _ W r i t e H e a d e r ( s e l f . _ w , c o d e ) Andrey Petrov · @shazow

Slide 58

Slide 58 text

Challenge 2: Translation layer (Cont.) Original interface in Go: t y p e h t t p . R e s p o n s e W r i t e r i n t e r f a c e { W r i t e H e a d e r ( i n t ) } Andrey Petrov · @shazow

Slide 59

Slide 59 text

Challenge 2: Translation layer (Cont.) Original interface in Go: t y p e h t t p . R e s p o n s e W r i t e r i n t e r f a c e { W r i t e H e a d e r ( i n t ) } Exported interface from Go: f u n c R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) Andrey Petrov · @shazow

Slide 60

Slide 60 text

Challenge 2: Translation layer (Cont.) Original interface in Go: t y p e h t t p . R e s p o n s e W r i t e r i n t e r f a c e { W r i t e H e a d e r ( i n t ) } Exported interface from Go: f u n c R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) C header: v o i d R e s p o n s e W r i t e r _ W r i t e H e a d e r ( u n s i g n e d i n t p 0 , i n t p 1 ) ; Andrey Petrov · @shazow

Slide 61

Slide 61 text

Challenge 2: Translation layer (Cont.) Original interface in Go: t y p e h t t p . R e s p o n s e W r i t e r i n t e r f a c e { W r i t e H e a d e r ( i n t ) } Exported interface from Go: f u n c R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) C header: v o i d R e s p o n s e W r i t e r _ W r i t e H e a d e r ( u n s i g n e d i n t p 0 , i n t p 1 ) ; Accessing C from Python: R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w , h e a d e r ) Andrey Petrov · @shazow

Slide 62

Slide 62 text

Challenge 2: Translation layer (Cont.) Original interface in Go: t y p e h t t p . R e s p o n s e W r i t e r i n t e r f a c e { W r i t e H e a d e r ( i n t ) } Exported interface from Go: f u n c R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) C header: v o i d R e s p o n s e W r i t e r _ W r i t e H e a d e r ( u n s i g n e d i n t p 0 , i n t p 1 ) ; Accessing C from Python: R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w , h e a d e r ) Python wrapper: R e s p o n s e W r i t e r . s e t _ s t a t u s ( s e l f , c o d e ) Andrey Petrov · @shazow

Slide 63

Slide 63 text

Overcoming Challenges 1. Don't share memory 2. Translation layer 3. Seriously, don't share memory Andrey Petrov · @shazow

Slide 64

Slide 64 text

Challenge 3: Seriously, don't share memory Wait, what's a Go interface? Andrey Petrov · @shazow

Slide 65

Slide 65 text

Challenge 3: Seriously, don't share memory Wait, what's a Go interface? To use it, we need to pass a pointer through ☞ Go ⇢ C ⇢ Python ⇢ C ⇢ Go ☞ Andrey Petrov · @shazow

Slide 66

Slide 66 text

Challenge 3: Seriously, don't share memory Wait, what's a Go interface? To use it, we need to pass a pointer through ☞ Go ⇢ C ⇢ Python ⇢ C ⇢ Go ☞ Solution: Pointer Proxy! Andrey Petrov · @shazow

Slide 67

Slide 67 text

Pointer Proxy PtrProxy ResponseWriter 0xd34db33f PtrId 1234 Ref Deref Free Safe for Go Safe for C Andrey Petrov · @shazow

Slide 68

Slide 68 text

Pointer Proxy t y p e p t r P r o x y s t r u c t { s y n c . M u t e x c o u n t u i n t l o o k u p m a p [ u i n t ] u n s a f e . P o i n t e r } f u n c ( p * p t r P r o x y ) R e f ( p t r u n s a f e . P o i n t e r ) C . u i n t { . . . } f u n c ( p * p t r P r o x y ) D e r e f ( i d C . u i n t ) ( u n s a f e . P o i n t e r , b o o l ) { . . . } f u n c ( p * p t r P r o x y ) F r e e ( i d C . u i n t ) { . . . } Now we can pass an opaque uint across enemy lines and it's perfectly safe. Andrey Petrov · @shazow

Slide 69

Slide 69 text

Quick flashback: Translation layer One pointer proxy to rule them all. v a r c p o i n t e r s = P t r P r o x y ( ) Andrey Petrov · @shazow

Slide 70

Slide 70 text

Quick flashback: Translation layer One pointer proxy to rule them all. v a r c p o i n t e r s = P t r P r o x y ( ) In the callback, reference to bind them. ⛓ h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { / / W r a p r e l e v a n t r e q u e s t f i e l d s i n a C - f r i e n d l y d a t a s t r u c t u r e . c r e q : = C . R e q u e s t { . . . } w P t r : = c p o i n t e r s . R e f ( u n s a f e . P o i n t e r ( & w ) ) . . . c p o i n t e r s . F r e e ( w P t r ) } ) Andrey Petrov · @shazow

Slide 71

Slide 71 text

Quick flashback: Translation layer One pointer proxy to rule them all. v a r c p o i n t e r s = P t r P r o x y ( ) In the callback, reference to bind them. ⛓ h t t p . H a n d l e F u n c ( p a t t e r n , f u n c ( w h t t p . R e s p o n s e W r i t e r , r e q * h t t p . R e q u e s t ) { / / W r a p r e l e v a n t r e q u e s t f i e l d s i n a C - f r i e n d l y d a t a s t r u c t u r e . c r e q : = C . R e q u e s t { . . . } w P t r : = c p o i n t e r s . R e f ( u n s a f e . P o i n t e r ( & w ) ) . . . c p o i n t e r s . F r e e ( w P t r ) } ) With pointer proxy, dereference to find them. f u n c R e s p o n s e W r i t e r _ W r i t e H e a d e r ( w P t r C . u i n t , h e a d e r C . i n t ) { w , _ : = c p o i n t e r s . D e r e f ( w P t r ) ( * ( * h t t p . R e s p o n s e W r i t e r ) ( w ) ) . W r i t e H e a d e r ( i n t ( h e a d e r ) ) } Andrey Petrov · @shazow

Slide 72

Slide 72 text

Recap If our languages can speak with C, they can speak with each other. Andrey Petrov · @shazow

Slide 73

Slide 73 text

Recap If our languages can speak with C, they can speak with each other. Be careful going in and out of runtimes. Andrey Petrov · @shazow

Slide 74

Slide 74 text

Recap If our languages can speak with C, they can speak with each other. Be careful going in and out of runtimes. Be super-careful with sharing memory. Andrey Petrov · @shazow

Slide 75

Slide 75 text

Recap If our languages can speak with C, they can speak with each other. Be careful going in and out of runtimes. Be super-careful with sharing memory. We'll need a translation layer to use non-trivial language constructs. Andrey Petrov · @shazow

Slide 76

Slide 76 text

Other Considerations Andrey Petrov · @shazow

Slide 77

Slide 77 text

Other Considerations Memory leaks Andrey Petrov · @shazow

Slide 78

Slide 78 text

Other Considerations Memory leaks Race conditions Andrey Petrov · @shazow

Slide 79

Slide 79 text

Other Considerations Memory leaks Race conditions Context switching overhead Andrey Petrov · @shazow

Slide 80

Slide 80 text

Other Considerations Memory leaks Race conditions Context switching overhead Probably security issues because C is hard Andrey Petrov · @shazow

Slide 81

Slide 81 text

Other Considerations Memory leaks Race conditions Context switching overhead Probably security issues because C is hard Architecture campanelle Andrey Petrov · @shazow

Slide 82

Slide 82 text

A Palate Cleanser Andrey Petrov · @shazow

Slide 83

Slide 83 text

A Palate Cleanser ✨ LOLBENCHMARKS! Andrey Petrov · @shazow

Slide 84

Slide 84 text

LOLBENCHMARKS Andrey Petrov · @shazow

Slide 85

Slide 85 text

LOLBENCHMARKS Name Total Req/Sec Time/Req go-net/http 1.115 8969.89 0.111 gohttp-c 1.181 8470.97 0.118 gohttp-python 1.285 7779.87 0.129 gunicorn-flask 7.826 1277.73 0.783 werkzeug-flask 15.029 665.37 1.503 Conditions: a b doing 10,000 requests with 10 concurrency on my . Andrey Petrov · @shazow

Slide 86

Slide 86 text

Thank you github.com/shazow/gohttplib hrku.co/gopythongo twitter.com/shazow shazow.net Andrey Petrov · @shazow