My Favorite Talks at GopherCon 2019

Transcript

1. 1 My Favorite Talks at GopherCon 2019 Yuki ITO
2. Table of Contents !2 Observability in Go Socket to Me: Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2:
3. Table of Contents !3 Observability in Go Socket to Me: Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2:
4. Observability in Go !4 https://github.com/freeformz/go-observability-workshop/ Repository
5. Observability in Go !5 Log Trace Metric 3 Pillars for Observability
6. Observability in Go !6 Log Trace Metric 3 Pillars for Observability
7. Logs !7 log package func handler(w http.ResponseWriter, r *http.Request) { status := http.StatusOK // net/http returns 200 by default defer func(t time.Time) { log.Printf( "%s %q => %d (%2.3fs)\n", r.Method, r.URL.String(), status, time.Since(t).Seconds(), ) }(time.Now()) // … }
8. Logs !8 log package func main() { log.SetFlags(log.Flags() | log.Lshortfile) log.Println("Listening at: http://localhost:" + port) http.ListenAndServe(":"+port, nil) // ... } 2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080
9. Logs !9 Structured Logs github.com/sirupsen/logrus 2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080 { “level”:"info", "msg":"Listening at: http://localhost:8080”, “time":"2019-07-17T21:26:08-07:00" }
10. !10 Log Trace Metric How should we log? If the only observability tool we have is logs, then we should log everything.
11. !11 Log Trace Metric How should we log? If we have metrics and traces, then reduce logging to important events meant largely for human consumption.
12. Observability in Go !12 Log Trace Metric 3 Pillars for Observability
13. Metrics !13 expvar package import ( "expvar" // ... ) func main() { // ... // Expose the port value ep := expvar.NewString("Port") ep.Set(port) http.ListenAndServe(":"+port, nil) //... }
14. Metrics !14 expvar package package expvar // ... func init() { http.HandleFunc("/debug/vars", expvarHandler) Publish("cmdline", Func(cmdline)) Publish("memstats", Func(memstats)) }
15. Metrics !15 expvar package $ curl -s http://localhost:8080/debug/vars | jq .Port "8080"
16. Metrics !16 expvar makes us be able to expose metrics We can recognize only `current status` of system. Hoever…
17. Metrics !17 Prometheus Time Series Database
18. Metrics !18 import ( // ... "github.com/prometheus/client_golang/prometheus/promhttp" ) func main() { // ... http.Handle("/metrics", promhttp.Handler()) // … http.ListenAndServe(":"+port, nil) // ... }
19. Metrics !19 Pull Model Prometheus Scrape `/metrics` Application server
20. Metrics !20
21. Observability in Go !21 Log Trace Metric 3 Pillars for Observability
22. Traces !22 Microservices ServiceB ServiceA
23. Traces !23 Distributed Tracing Jaeger
24. Traces !24 import ( // ... "contrib.go.opencensus.io/exporter/jaeger" "go.opencensus.io/trace" ) func main() { je, err := jaeger.NewExporter(jaeger.Options{ CollectorEndpoint: "http://localhost:14268/api/traces", Process: jaeger.Process{ ServiceName: "servicea", Tags: []jaeger.Tag{ jaeger.StringTag("server", "1"), jaeger.StringTag("port", port), jaeger.BoolTag("demo", true), }, }, }) trace.RegisterExporter(je) }
25. Traces !25 func queryServiceBHandler(c *http.Client, url string) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { ctx, span := trace.StartSpan( r.Context(), "queryServiceBHandler", ) defer span.End() // ... } }
26. Traces !26
27. Traces !27
28. Observability in Go !28 Log Trace Metric 3 Pillars for Observability
29. Table of Contents !29 Observability in Go Socket to Me: Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2:
30. !30 System calls under net.Listen & Accept https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
31. !31 System calls under net.Listen & Accept https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
32. !32 func (srv *Server) Serve(l net.Listener) error { // ... for { rw, e := l.Accept() // ... c := srv.newConn(rw) go c.serve(ctx) } } Socket to me net/http/server.go
33. !33 System calls under net.Listen & Accept https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
34. Socket Options in Go !34 ListenConfig.Listen() ListenConfig.Control RawConn.Control() Callback to set socket options
35. Socket Options in Go !35 https://github.com/110y/sockoptgo/ Example Implementation
36. Socket Options in Go !36 ListenConfig.Listen() ListenConfig.Control RawConn.Control() Callback to set socket options
37. !37 func Listen(network, address string) (Listener, error) { var lc ListenConfig return lc.Listen(context.Background(), network, address) } net.Listen() Socket Options in Go
38. !38 Socket Options in Go lc := net.ListenConfig{ Control: listenControler, } lis, err := lc.Listen(ctx, "tcp4", addr)
39. Socket Options in Go !39 ListenConfig.Listen() ListenConfig.Control RawConn.Control() Callback to set socket options
40. !40 func listenControler(network, address string, c syscall.RawConn) error { return c.Control(func(fd uintptr) { syscall.SetsockoptInt( int(fd), syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1, ) }) } Socket Options in Go
41. !41 func listenControler(network, address string, c syscall.RawConn) error { return c.Control(func(fd uintptr) { syscall.SetsockoptInt( int(fd), syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1, ) }) } Socket Options in Go
42. Table of Contents !42 Observability in Go Socket to Me: Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2: