Slide 1
Slide 1 text
1
My Favorite Talks
at
GopherCon 2019
Yuki ITO
Slide 2
Slide 2 text
Table of Contents
!2
Observability in Go
Socket to Me:
Where do Sockets Live in Go?
Day 0 (Pre-Conference Workshop):
Day 2:
Slide 3
Slide 3 text
Table of Contents
!3
Observability in Go
Socket to Me:
Where do Sockets Live in Go?
Day 0 (Pre-Conference Workshop):
Day 2:
Slide 4
Slide 4 text
Observability in Go
!4
https://github.com/freeformz/go-observability-workshop/
Repository
Slide 5
Slide 5 text
Observability in Go
!5
Log
Trace
Metric
3 Pillars for Observability
Slide 6
Slide 6 text
Observability in Go
!6
Log
Trace
Metric
3 Pillars for Observability
Slide 7
Slide 7 text
Logs
!7
log package
func handler(w http.ResponseWriter, r *http.Request) {
status := http.StatusOK // net/http returns 200 by default
defer func(t time.Time) {
log.Printf(
"%s %q => %d (%2.3fs)\n",
r.Method,
r.URL.String(),
status,
time.Since(t).Seconds(),
)
}(time.Now())
// …
}
Slide 8
Slide 8 text
Logs
!8
log package
func main() {
log.SetFlags(log.Flags() | log.Lshortfile)
log.Println("Listening at: http://localhost:" + port)
http.ListenAndServe(":"+port, nil)
// ...
}
2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080
Slide 9
Slide 9 text
Logs
!9
Structured Logs
github.com/sirupsen/logrus
2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080
{
“level”:"info",
"msg":"Listening at: http://localhost:8080”,
“time":"2019-07-17T21:26:08-07:00"
}
Slide 10
Slide 10 text
!10
Log
Trace
Metric
How should we log?
If the only observability tool we have is logs,
then we should log everything.
Slide 11
Slide 11 text
!11
Log
Trace
Metric
How should we log?
If we have metrics and traces,
then reduce logging to important events meant
largely for human consumption.
Slide 12
Slide 12 text
Observability in Go
!12
Log
Trace
Metric
3 Pillars for Observability
Slide 13
Slide 13 text
Metrics
!13
expvar package
import (
"expvar"
// ...
)
func main() {
// ...
// Expose the port value
ep := expvar.NewString("Port")
ep.Set(port)
http.ListenAndServe(":"+port, nil)
//...
}
Slide 14
Slide 14 text
Metrics
!14
expvar package
package expvar
// ...
func init() {
http.HandleFunc("/debug/vars", expvarHandler)
Publish("cmdline", Func(cmdline))
Publish("memstats", Func(memstats))
}
Slide 15
Slide 15 text
Metrics
!15
expvar package
$ curl -s http://localhost:8080/debug/vars | jq .Port
"8080"
Slide 16
Slide 16 text
Metrics
!16
expvar makes us be able to expose metrics
We can recognize only `current status` of system.
Hoever…
Slide 17
Slide 17 text
Metrics
!17
Prometheus
Time Series Database
Slide 18
Slide 18 text
Metrics
!18
import (
// ...
"github.com/prometheus/client_golang/prometheus/promhttp"
)
func main() {
// ...
http.Handle("/metrics", promhttp.Handler())
// …
http.ListenAndServe(":"+port, nil)
// ...
}
Slide 19
Slide 19 text
Metrics
!19
Pull Model
Prometheus
Scrape `/metrics`
Application server
Slide 20
Slide 20 text
Metrics
!20
Slide 21
Slide 21 text
Observability in Go
!21
Log
Trace
Metric
3 Pillars for Observability
Slide 22
Slide 22 text
Traces
!22
Microservices
ServiceB
ServiceA
Slide 23
Slide 23 text
Traces
!23
Distributed Tracing
Jaeger
Slide 24
Slide 24 text
Traces
!24
import (
// ...
"contrib.go.opencensus.io/exporter/jaeger"
"go.opencensus.io/trace"
)
func main() {
je, err := jaeger.NewExporter(jaeger.Options{
CollectorEndpoint: "http://localhost:14268/api/traces",
Process: jaeger.Process{
ServiceName: "servicea",
Tags: []jaeger.Tag{
jaeger.StringTag("server", "1"),
jaeger.StringTag("port", port),
jaeger.BoolTag("demo", true),
},
},
})
trace.RegisterExporter(je)
}
Slide 25
Slide 25 text
Traces
!25
func queryServiceBHandler(c *http.Client, url string)
http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx, span := trace.StartSpan(
r.Context(),
"queryServiceBHandler",
)
defer span.End()
// ...
}
}
Slide 26
Slide 26 text
Traces
!26
Slide 27
Slide 27 text
Traces
!27
Slide 28
Slide 28 text
Observability in Go
!28
Log
Trace
Metric
3 Pillars for Observability
Slide 29
Slide 29 text
Table of Contents
!29
Observability in Go
Socket to Me:
Where do Sockets Live in Go?
Day 0 (Pre-Conference Workshop):
Day 2:
Slide 30
Slide 30 text
!30
System calls under net.Listen & Accept
https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
Slide 31
Slide 31 text
!31
System calls under net.Listen & Accept
https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
Slide 32
Slide 32 text
!32
func (srv *Server) Serve(l net.Listener) error {
// ...
for {
rw, e := l.Accept()
// ...
c := srv.newConn(rw)
go c.serve(ctx)
}
}
Socket to me
net/http/server.go
Slide 33
Slide 33 text
!33
System calls under net.Listen & Accept
https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
Slide 34
Slide 34 text
Socket Options in Go
!34
ListenConfig.Listen()
ListenConfig.Control
RawConn.Control()
Callback to set socket options
Slide 35
Slide 35 text
Socket Options in Go
!35
https://github.com/110y/sockoptgo/
Example Implementation
Slide 36
Slide 36 text
Socket Options in Go
!36
ListenConfig.Listen()
ListenConfig.Control
RawConn.Control()
Callback to set socket options
Slide 37
Slide 37 text
!37
func Listen(network, address string) (Listener, error) {
var lc ListenConfig
return lc.Listen(context.Background(), network, address)
}
net.Listen()
Socket Options in Go
Slide 38
Slide 38 text
!38
Socket Options in Go
lc := net.ListenConfig{
Control: listenControler,
}
lis, err := lc.Listen(ctx, "tcp4", addr)
Slide 39
Slide 39 text
Socket Options in Go
!39
ListenConfig.Listen()
ListenConfig.Control
RawConn.Control()
Callback to set socket options
Slide 40
Slide 40 text
!40
func listenControler(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
syscall.SetsockoptInt(
int(fd),
syscall.SOL_SOCKET,
syscall.SO_REUSEADDR,
1,
)
})
}
Socket Options in Go
Slide 41
Slide 41 text
!41
func listenControler(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
syscall.SetsockoptInt(
int(fd),
syscall.SOL_SOCKET,
syscall.SO_REUSEADDR,
1,
)
})
}
Socket Options in Go
Slide 42
Slide 42 text
Table of Contents
!42
Observability in Go
Socket to Me:
Where do Sockets Live in Go?
Day 0 (Pre-Conference Workshop):
Day 2: