Upgrade to Pro — share decks privately, control downloads, hide ads and more …

My Favorite Talks at GopherCon 2019

Yuki Ito
August 20, 2019

My Favorite Talks at GopherCon 2019

Yuki Ito

August 20, 2019
Tweet

More Decks by Yuki Ito

Other Decks in Technology

Transcript

  1. 1
    My Favorite Talks
    at
    GopherCon 2019
    Yuki ITO

    View full-size slide

  2. Table of Contents
    !2
    Observability in Go
    Socket to Me:
    Where do Sockets Live in Go?
    Day 0 (Pre-Conference Workshop):
    Day 2:

    View full-size slide

  3. Table of Contents
    !3
    Observability in Go
    Socket to Me:
    Where do Sockets Live in Go?
    Day 0 (Pre-Conference Workshop):
    Day 2:

    View full-size slide

  4. Observability in Go
    !4
    https://github.com/freeformz/go-observability-workshop/
    Repository

    View full-size slide

  5. Observability in Go
    !5
    Log
    Trace
    Metric
    3 Pillars for Observability

    View full-size slide

  6. Observability in Go
    !6
    Log
    Trace
    Metric
    3 Pillars for Observability

    View full-size slide

  7. Logs
    !7
    log package
    func handler(w http.ResponseWriter, r *http.Request) {
    status := http.StatusOK // net/http returns 200 by default
    defer func(t time.Time) {
    log.Printf(
    "%s %q => %d (%2.3fs)\n",
    r.Method,
    r.URL.String(),
    status,
    time.Since(t).Seconds(),
    )
    }(time.Now())
    // …
    }

    View full-size slide

  8. Logs
    !8
    log package
    func main() {
    log.SetFlags(log.Flags() | log.Lshortfile)
    log.Println("Listening at: http://localhost:" + port)
    http.ListenAndServe(":"+port, nil)
    // ...
    }
    2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080

    View full-size slide

  9. Logs
    !9
    Structured Logs
    github.com/sirupsen/logrus
    2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080
    {
    “level”:"info",
    "msg":"Listening at: http://localhost:8080”,
    “time":"2019-07-17T21:26:08-07:00"
    }

    View full-size slide

  10. !10
    Log
    Trace
    Metric
    How should we log?
    If the only observability tool we have is logs,
    then we should log everything.

    View full-size slide

  11. !11
    Log
    Trace
    Metric
    How should we log?
    If we have metrics and traces,
    then reduce logging to important events meant
    largely for human consumption.

    View full-size slide

  12. Observability in Go
    !12
    Log
    Trace
    Metric
    3 Pillars for Observability

    View full-size slide

  13. Metrics
    !13
    expvar package
    import (
    "expvar"
    // ...
    )
    func main() {
    // ...
    // Expose the port value
    ep := expvar.NewString("Port")
    ep.Set(port)
    http.ListenAndServe(":"+port, nil)
    //...
    }

    View full-size slide

  14. Metrics
    !14
    expvar package
    package expvar
    // ...
    func init() {
    http.HandleFunc("/debug/vars", expvarHandler)
    Publish("cmdline", Func(cmdline))
    Publish("memstats", Func(memstats))
    }

    View full-size slide

  15. Metrics
    !15
    expvar package
    $ curl -s http://localhost:8080/debug/vars | jq .Port
    "8080"

    View full-size slide

  16. Metrics
    !16
    expvar makes us be able to expose metrics
    We can recognize only `current status` of system.


    Hoever…

    View full-size slide

  17. Metrics
    !17
    Prometheus
    Time Series Database

    View full-size slide

  18. Metrics
    !18
    import (
    // ...
    "github.com/prometheus/client_golang/prometheus/promhttp"
    )
    func main() {
    // ...
    http.Handle("/metrics", promhttp.Handler())
    // …
    http.ListenAndServe(":"+port, nil)
    // ...
    }

    View full-size slide

  19. Metrics
    !19
    Pull Model
    Prometheus
    Scrape `/metrics`
    Application server

    View full-size slide

  20. Observability in Go
    !21
    Log
    Trace
    Metric
    3 Pillars for Observability

    View full-size slide

  21. Traces
    !22
    Microservices
    ServiceB
    ServiceA

    View full-size slide

  22. Traces
    !23
    Distributed Tracing
    Jaeger

    View full-size slide

  23. Traces
    !24
    import (
    // ...
    "contrib.go.opencensus.io/exporter/jaeger"
    "go.opencensus.io/trace"
    )
    func main() {
    je, err := jaeger.NewExporter(jaeger.Options{
    CollectorEndpoint: "http://localhost:14268/api/traces",
    Process: jaeger.Process{
    ServiceName: "servicea",
    Tags: []jaeger.Tag{
    jaeger.StringTag("server", "1"),
    jaeger.StringTag("port", port),
    jaeger.BoolTag("demo", true),
    },
    },
    })
    trace.RegisterExporter(je)
    }

    View full-size slide

  24. Traces
    !25
    func queryServiceBHandler(c *http.Client, url string)
    http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
    ctx, span := trace.StartSpan(
    r.Context(),
    "queryServiceBHandler",
    )
    defer span.End()
    // ...
    }
    }

    View full-size slide

  25. Observability in Go
    !28
    Log
    Trace
    Metric
    3 Pillars for Observability

    View full-size slide

  26. Table of Contents
    !29
    Observability in Go
    Socket to Me:
    Where do Sockets Live in Go?
    Day 0 (Pre-Conference Workshop):
    Day 2:

    View full-size slide

  27. !30
    System calls under net.Listen & Accept
    https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go

    View full-size slide

  28. !31
    System calls under net.Listen & Accept
    https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go

    View full-size slide

  29. !32
    func (srv *Server) Serve(l net.Listener) error {
    // ...
    for {
    rw, e := l.Accept()
    // ...
    c := srv.newConn(rw)
    go c.serve(ctx)
    }
    }
    Socket to me
    net/http/server.go

    View full-size slide

  30. !33
    System calls under net.Listen & Accept
    https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go

    View full-size slide

  31. Socket Options in Go
    !34
    ListenConfig.Listen()
    ListenConfig.Control
    RawConn.Control()
    Callback to set socket options

    View full-size slide

  32. Socket Options in Go
    !35
    https://github.com/110y/sockoptgo/
    Example Implementation

    View full-size slide

  33. Socket Options in Go
    !36
    ListenConfig.Listen()
    ListenConfig.Control
    RawConn.Control()
    Callback to set socket options

    View full-size slide

  34. !37
    func Listen(network, address string) (Listener, error) {
    var lc ListenConfig
    return lc.Listen(context.Background(), network, address)
    }
    net.Listen()
    Socket Options in Go

    View full-size slide

  35. !38
    Socket Options in Go
    lc := net.ListenConfig{
    Control: listenControler,
    }
    lis, err := lc.Listen(ctx, "tcp4", addr)

    View full-size slide

  36. Socket Options in Go
    !39
    ListenConfig.Listen()
    ListenConfig.Control
    RawConn.Control()
    Callback to set socket options

    View full-size slide

  37. !40
    func listenControler(network, address string, c syscall.RawConn) error {
    return c.Control(func(fd uintptr) {
    syscall.SetsockoptInt(
    int(fd),
    syscall.SOL_SOCKET,
    syscall.SO_REUSEADDR,
    1,
    )
    })
    }
    Socket Options in Go

    View full-size slide

  38. !41
    func listenControler(network, address string, c syscall.RawConn) error {
    return c.Control(func(fd uintptr) {
    syscall.SetsockoptInt(
    int(fd),
    syscall.SOL_SOCKET,
    syscall.SO_REUSEADDR,
    1,
    )
    })
    }
    Socket Options in Go

    View full-size slide

  39. Table of Contents
    !42
    Observability in Go
    Socket to Me:
    Where do Sockets Live in Go?
    Day 0 (Pre-Conference Workshop):
    Day 2:

    View full-size slide