Go: a simple programming environment

By Andrew Gerrand
http://kod.io

Transcript

1. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 1/42 Go: a simple programming environment Andrew Gerrand Google
2. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 2/42 Why Go?
3. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 3/42 Software should be simple
4. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 4/42 Software can be simple
5. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 5/42 What I mean by "simple" Small Readable Consistent Orthogonal Predictable Robust Useful by default
6. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 6/42 Go at a glance Compiled Statically typed Clean syntax Simple type system Concurrency primitives Rich standard library Great tools Open source This talk is just a taste.
7. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 7/42 The Gopher
8. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 8/42 Go at Google Go was created at Google to solve Google's problems: Network servers that do a lot of stuff concurrently Run on huge clusters of networked machines Maintained by large teams of programmers Currently used by/for: YouTube (see the vitess project (https://code.google.com/p/vitess/) ) dl.google.com (http://talks.golang.org/2013/oscon-dl.slide) Logs Analysis teams Site Reliability and Operations teams Many others (the details are confidential, sorry!)
9. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 9/42 Go beyond Google (two short lists) Companies: Canonical CloudFlare Heroku Mozilla ngmoco:) Soundcloud Zynga Open source projects: Camlistore Docker
10. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 10/42 Hello, Go p a c k a g e m a i n i m p o r t " f m t " f u n c m a i n ( ) { f m t . P r i n t l n ( " H e l l o , G o " ) } Run
11. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 11/42 go Goroutines are lightweight threads that are managed by the Go runtime. To run a function in a new goroutine, just put " g o " before the function call. p a c k a g e m a i n i m p o r t ( " f m t " " t i m e " ) f u n c m a i n ( ) { g o s a y ( " l e t ' s g o ! " , 3 * t i m e . S e c o n d ) g o s a y ( " h o ! " , 2 * t i m e . S e c o n d ) g o s a y ( " h e y ! " , 1 * t i m e . S e c o n d ) t i m e . S l e e p ( 4 * t i m e . S e c o n d ) } f u n c s a y ( t e x t s t r i n g , d e l a y t i m e . D u r a t i o n ) { t i m e . S l e e p ( d e l a y ) f m t . P r i n t l n ( t e x t ) } Run
12. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 12/42 chan Channels are typed conduits for sychronization and communication between goroutines. They're a versatile and expressive means of modelling concurrent processes. But we're not going to look at them today. (There's no time!)
13. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 13/42 sync Channels are great, but sometimes other concurrency mechanisms are a better fit. The s y n c package provides mutexes, condition variables, and more useful primitives. f u n c m a i n ( ) { w g : = n e w ( s y n c . W a i t G r o u p ) w g . A d d ( 3 ) g o s a y ( w g , " l e t ' s g o ! " , 3 * t i m e . S e c o n d ) g o s a y ( w g , " h o ! " , 2 * t i m e . S e c o n d ) g o s a y ( w g , " h e y ! " , 1 * t i m e . S e c o n d ) w g . W a i t ( ) } f u n c s a y ( w g * s y n c . W a i t G r o u p , t e x t s t r i n g , d e l a y t i m e . D u r a t i o n ) { t i m e . S l e e p ( d e l a y ) f m t . P r i n t l n ( t e x t ) w g . D o n e ( ) } Run
14. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 14/42 net/http (1/2) The n e t / h t t p package provides an HTTP client. The client handles HTTP K e e p - A l i v e using a pool of connections, by default. (This is configurable, of course.) f u n c m a i n ( ) { r , e r r : = h t t p . G e t ( " h t t p : / / w w w . g o l a n g . o r g / " ) i f e r r ! = n i l { l o g . F a t a l ( e r r ) } i f r . S t a t u s C o d e ! = h t t p . S t a t u s O K { l o g . F a t a l ( r . S t a t u s ) } i o . C o p y ( o s . S t d o u t , r . B o d y ) } Run
15. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 15/42 net/http (2/2) The n e t / h t t p package also provides an HTTP server. This is a high-performance, DoS-hardened, production-ready web server. It serves d l . g o o g l e . c o m . f u n c m a i n ( ) { h t t p . H a n d l e F u n c ( " / " , h a n d l e r ) e r r : = h t t p . L i s t e n A n d S e r v e ( " l o c a l h o s t : 8 0 8 0 " , n i l ) i f e r r ! = n i l { l o g . F a t a l ( e r r ) } } f u n c h a n d l e r ( w h t t p . R e s p o n s e W r i t e r , r * h t t p . R e q u e s t ) { f m t . F p r i n t ( w , " H e l l o , w e b " ) } Run
16. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 16/42 html/template The h t m l / t e m p l a t e package provides an HTML templating system that automatically escapes content depending on its context. c o n s t h t m l = ` < s c r i p t > v a r f o o = { { . F o o } } ; < / s c r i p t > < a h r e f = " { { . U R L } } " > { { . T e x t } } < / a > ` f u n c m a i n ( ) { t m p l : = t e m p l a t e . M u s t ( t e m p l a t e . N e w ( " e x a m p l e " ) . P a r s e ( h t m l ) ) d a t a : = s t r u c t { F o o s t r i n g U R L , T e x t s t r i n g } { F o o : ` S o m e " q u o t e d " s t r i n g ` , U R L : ` " o n C l i c k = " a l e r t ( ' x s s ! ' ) ; ` , T e x t : " T h e < - o p e r a t o r i s f o r c h a n n e l s e n d s a n d r e c e i v e s " , } t m p l . E x e c u t e ( o s . S t d o u t , d a t a ) } Run
17. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 17/42 flag The f l a g package provides a simple API for parsing command-line flags. Example invocation (a little different than GNU getopt): $ f l a g - m e s s a g e ' H o l d o n . . . ' - d e l a y 5 m v a r ( m e s s a g e = f l a g . S t r i n g ( " m e s s a g e " , " H e l l o ! " , " w h a t t o s a y " ) d e l a y = f l a g . D u r a t i o n ( " d e l a y " , 2 * t i m e . S e c o n d , " h o w l o n g t o w a i t " ) ) f u n c m a i n ( ) { f l a g . P a r s e ( ) f m t . P r i n t l n ( * m e s s a g e ) t i m e . S l e e p ( * d e l a y ) } Run
18. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 18/42 An example isgo1point1outyet.com (http://isgo1point1outyet.com)
19. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 19/42 Structure The program has two parts that execute concurrently: a poller that continuously checks whether Go 1.2 has been tagged, and an HTTP server that provides the user interface. Go 1.2 can be considered "tagged" when this URL returns a " 2 0 0 O K " response: h t t p s : / / c o d e . g o o g l e . c o m / p / g o / s o u r c e / d e t a i l ? r = g o 1 . 2
20. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 20/42 The Server type The S e r v e r struct contains the server state: the poll URL and period, and a boolean value ("Is Go 1.2 out yet?") and a mutex to protect it. t y p e S e r v e r s t r u c t { u r l s t r i n g p e r i o d t i m e . D u r a t i o n m u s y n c . R W M u t e x y e s b o o l }
21. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 21/42 Sharing state We must share state ("Is Go 1.2 out?") between the repo poller and the user interface. To read from a S e r v e r value s , take the read lock (multiple goroutines can do this simultaneously): s . m u . R L o c k ( ) y e s : = s . y e s s . m u . R U n l o c k ( ) To write, take the write lock (only one goroutine can do this at a time): s . m u . L o c k ( ) s . y e s = t r u e s . m u . U n l o c k ( )
22. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 22/42 Polling (1/2) The i s T a g g e d function returns t r u e if the given URL returns " 2 0 0 O K " : f u n c i s T a g g e d ( u r l s t r i n g ) b o o l { r , e r r : = h t t p . H e a d ( u r l ) i f e r r ! = n i l { l o g . P r i n t ( e r r ) r e t u r n f a l s e } r e t u r n r . S t a t u s C o d e = = h t t p . S t a t u s O K }
23. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 23/42 Polling (2/2) The p o l l method loops until i s T a g g e d returns t r u e . Then it updates the state ("Go 1.2 is out!") and returns. f u n c ( s * S e r v e r ) p o l l ( ) { f o r ! i s T a g g e d ( s . u r l ) { t i m e . S l e e p ( s . p e r i o d ) } s . m u . L o c k ( ) s . y e s = t r u e s . m u . U n l o c k ( ) }
24. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 24/42 Constructing a Server The N e w S e r v e r function constructs a new S e r v e r , starts the polling goroutine, and returns a * S e r v e r value. f u n c N e w S e r v e r ( u r l s t r i n g , p e r i o d t i m e . D u r a t i o n ) * S e r v e r { s : = & S e r v e r { u r l : u r l , p e r i o d : p e r i o d } g o s . p o l l ( ) r e t u r n s }
25. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 25/42 Serving the user interface The S e r v e H T T P method serves an HTTP request. It puts the s . y e s and s . u r l values into a struct, and uses the struct to render the template as the HTTP response. f u n c ( s * S e r v e r ) S e r v e H T T P ( w h t t p . R e s p o n s e W r i t e r , r * h t t p . R e q u e s t ) { s . m u . R L o c k ( ) d a t a : = s t r u c t { Y e s b o o l U R L s t r i n g } { Y e s : s . y e s , U R L : s . u r l , } s . m u . R U n l o c k ( ) e r r : = t m p l . E x e c u t e ( w , d a t a ) i f e r r ! = n i l { l o g . P r i n t ( e r r ) } } This is an implementation of the h t t p . H a n d l e r interface.
26. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 26/42 The HTML user interface The t m p l variable is a template that provides the HTML user inteface. It is a global variable, so the template is parsed just once at init time. v a r t m p l = t e m p l a t e . M u s t ( t e m p l a t e . N e w ( " r o o t " ) . P a r s e ( ` < ! D O C T Y P E h t m l > < h t m l > < b o d y > < c e n t e r > < h 2 > I s G o 1 . 1 o u t y e t ? < / h 2 > < h 1 > { { i f . Y e s } } < a h r e f = " { { . U R L } } " > Y E S ! < / a > { { e l s e } } N o . { { e n d } } < / h 1 > < / c e n t e r > < / b o d y > < / h t m l > ` ) )
27. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 27/42 Putting it all together The m a i n function creates a new S e r v e r and sets up the web server. Some command-line flags enable run time configuration. c o n s t d e f a u l t C h a n g e U R L = " h t t p s : / / c o d e . g o o g l e . c o m / p / g o / s o u r c e / d e t a i l ? r = g o 1 . 2 " v a r ( h t t p A d d r = f l a g . S t r i n g ( " h t t p " , " l o c a l h o s t : 8 0 8 0 " , " L i s t e n a d d r e s s " ) p o l l P e r i o d = f l a g . D u r a t i o n ( " p o l l " , 5 * t i m e . S e c o n d , " P o l l p e r i o d " ) c h a n g e U R L = f l a g . S t r i n g ( " u r l " , d e f a u l t C h a n g e U R L , " C h a n g e U R L " ) ) f u n c m a i n ( ) { f l a g . P a r s e ( ) h t t p . H a n d l e ( " / " , N e w S e r v e r ( * c h a n g e U R L , * p o l l P e r i o d ) ) l o g . F a t a l ( h t t p . L i s t e n A n d S e r v e ( * h t t p A d d r , n i l ) ) } The whole program is just 87 lines of code.
28. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 28/42 Demo
29. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 29/42 Testing: writing tests The t e s t i n g package provides a lightweight test framework. f u n c T e s t I n d e x ( t * t e s t i n g . T ) { v a r t e s t s = [ ] s t r u c t { s s t r i n g s e p s t r i n g o u t i n t } { { " " , " " , 0 } , { " " , " a " , - 1 } , { " f o " , " f o o " , - 1 } , { " f o o " , " f o o " , 0 } , { " o o f o f o o f o o o " , " f " , 2 } , / / e t c } f o r _ , t e s t : = r a n g e t e s t s { a c t u a l : = s t r i n g s . I n d e x ( t e s t . s , t e s t . s e p ) i f a c t u a l ! = t e s t . o u t { t . E r r o r f ( " I n d e x ( % q , % q ) = % v ; w a n t % v " , t e s t . s , t e s t . s e p , a c t u a l , t e s t . o u t ) } } }
30. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 30/42 Testing: running tests The go tool runs tests. $ g o t e s t P A S S o k s t r i n g s 0 . 6 8 2 s $ g o t e s t - v = = = R U N T e s t I n d e x - - - P A S S : T e s t I n d e x ( 0 . 0 0 s e c o n d s ) = = = R U N T e s t R e p l a c e - - - P A S S : T e s t R e p l a c e ( 0 . 0 0 s e c o n d s ) = = = R U N T e s t C o n t a i n s - - - P A S S : T e s t C o n t a i n s ( 0 . 0 0 s e c o n d s ) . . . m a n y l i n e s o m i t t e d . . . P A S S o k s t r i n g s 0 . 6 5 9 s
31. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 31/42 Testing: coverage statistics New in Go 1.2: the g o tool can report test coverage statistics. $ g o t e s t - c o v e r P A S S c o v e r a g e : 9 6 . 4 % o f s t a t e m e n t s o k s t r i n g s 0 . 6 9 2 s The g o tool can generate coverage profiles that may be intepreted by the c o v e r tool. $ g o t e s t - c o v e r p r o f i l e = c o v e r . o u t $ g o t o o l c o v e r - f u n c = c o v e r . o u t s t r i n g s / r e a d e r . g o : L e n 6 6 . 7 % s t r i n g s / s t r i n g s . g o : T r i m S u f f i x 1 0 0 . 0 % . . . m a n y l i n e s o m i t t e d . . . s t r i n g s / s t r i n g s . g o : R e p l a c e 1 0 0 . 0 % s t r i n g s / s t r i n g s . g o : E q u a l F o l d 1 0 0 . 0 % t o t a l : ( s t a t e m e n t s ) 9 6 . 4 %
32. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 32/42 Testing: coverage visualization $ g o t o o l c o v e r - h t m l = c o v e r . o u t
33. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 33/42 Testing: isTagged To test the i s T a g g e d function we'll use the n e t / h t t p / h t t p t e s t package. f u n c T e s t I s T a g g e d ( t * t e s t i n g . T ) { v a r s t a t u s S t a t u s H a n d l e r = h t t p . S t a t u s N o t F o u n d s : = h t t p t e s t . N e w S e r v e r ( & s t a t u s ) d e f e r s . C l o s e ( ) i f i s T a g g e d ( s . U R L ) { t . F a t a l ( " i s T a g g e d = = t r u e , w a n t f a l s e " ) } s t a t u s = h t t p . S t a t u s O K i f ! i s T a g g e d ( s . U R L ) { t . F a t a l ( " i s T a g g e d = = f a l s e , w a n t t r u e " ) } } The S t a t u s H a n d l e r type is an h t t p . H a n d l e r that sends an HTTP status of itself. t y p e S t a t u s H a n d l e r i n t f u n c ( h * S t a t u s H a n d l e r ) S e r v e H T T P ( w h t t p . R e s p o n s e W r i t e r , r * h t t p . R e q u e s t ) { w . W r i t e H e a d e r ( i n t ( * h ) ) }
34. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 34/42 Testing: an integration test (1/2) f u n c T e s t I n t e g r a t i o n ( t * t e s t i n g . T ) { v a r s t a t u s S t a t u s H a n d l e r = h t t p . S t a t u s N o t F o u n d t s : = h t t p t e s t . N e w S e r v e r ( & s t a t u s ) d e f e r t s . C l o s e ( ) s : = N e w S e r v e r ( t s . U R L , 1 0 * t i m e . M i l l i s e c o n d ) t i m e . S l e e p ( 2 0 * t i m e . M i l l i s e c o n d ) r , _ : = h t t p . N e w R e q u e s t ( " G E T " , " / " , n i l ) w : = h t t p t e s t . N e w R e c o r d e r ( ) s . S e r v e H T T P ( w , r ) i f b : = w . B o d y . S t r i n g ( ) ; ! s t r i n g s . C o n t a i n s ( b , " N o . " ) { t . F a t a l f ( " b o d y = % q , w a n t n o " , b ) } s t a t u s = h t t p . S t a t u s O K t i m e . S l e e p ( 2 0 * t i m e . M i l l i s e c o n d ) w = h t t p t e s t . N e w R e c o r d e r ( ) s . S e r v e H T T P ( w , r ) i f b : = w . B o d y . S t r i n g ( ) ; ! s t r i n g s . C o n t a i n s ( b , " Y E S ! " ) { t . F a t a l f ( " b o d y = % q , w a n t y e s " , b ) } }
35. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 35/42 One more thing
36. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 36/42 expvar (1/2) The e x p v a r package allows you to export variables via an HTTP handler registered at / d e b u g / v a r s (http://localhost:8080/debug/vars) . p a c k a g e m a i n i m p o r t ( " e x p v a r " " l o g " " n e t / h t t p " " t i m e " ) f u n c m a i n ( ) { c o u n t : = e x p v a r . N e w I n t ( " c o u n t " ) g o f u n c ( ) { f o r { c o u n t . A d d ( 1 ) t i m e . S l e e p ( t i m e . S e c o n d ) } } ( ) l o g . F a t a l ( h t t p . L i s t e n A n d S e r v e ( " l o c a l h o s t : 8 0 8 0 " , n i l ) ) } Run
37. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 37/42 expvar (2/2) v a r ( h i t C o u n t = e x p v a r . N e w I n t ( " h i t C o u n t " ) p o l l C o u n t = e x p v a r . N e w I n t ( " p o l l C o u n t " ) p o l l E r r o r = e x p v a r . N e w S t r i n g ( " p o l l E r r o r " ) p o l l E r r o r C o u n t = e x p v a r . N e w I n t ( " p o l l E r r o r C o u n t " ) ) f u n c i s T a g g e d ( u r l s t r i n g ) b o o l { p o l l C o u n t . A d d ( 1 ) r , e r r : = h t t p . H e a d ( u r l ) i f e r r ! = n i l { l o g . P r i n t ( e r r ) p o l l E r r o r . S e t ( e r r . E r r o r ( ) ) p o l l E r r o r C o u n t . A d d ( 1 ) r e t u r n f a l s e } r e t u r n r . S t a t u s C o d e = = h t t p . S t a t u s O K } f u n c ( s * S e r v e r ) S e r v e H T T P ( w h t t p . R e s p o n s e W r i t e r , r * h t t p . R e q u e s t ) { h i t C o u n t . A d d ( 1 )
38. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 38/42 Demo
39. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 39/42 Conclusion Simplicity revisited: Small Readable Consistent Orthogonal Predictable Robust Useful by default
40. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 40/42 Stuff I didn't talk about Types and interfaces Concurrency in depth Packages and the build system Code formatting Benchmarking Debugging Documentation
41. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 41/42 Learn more about Go The Go web site golang.org (http://golang.org) The Go blog blog.golang.org (http://blog.golang.org) Go talks talks.golang.org (http://talks.golang.org) A Tour of Go tour.golang.org (http://tour.golang.org)
42. 11/13/13 Go: a simple programming environment 127.0.0.1:3999/talk.slide#19 42/42 Thank you Andrew Gerrand Google @enneff (http://twitter.com/enneff) adg@golang.org (mailto:adg@golang.org) http://golang.org (http://golang.org)