Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
My Favorite Talks at GopherCon 2019
Yuki Ito
August 20, 2019
Technology
3
1.5k
My Favorite Talks at GopherCon 2019
Yuki Ito
August 20, 2019
Tweet
Share
More Decks by Yuki Ito
See All by Yuki Ito
Custom Kubernetes Controllers at Mercari
110y
1
320
What Is the Go Workspace Mode
110y
4
890
What Are We Doing as Merpay Architect
110y
0
1k
Kubernetes Casual Talk
110y
0
36
How to Make Kubernetes Controller Development Happier: mercari.go #17
110y
1
460
メルカリのマイクロサービスをPCで動かそう!
110y
5
1.4k
Tour of Istio
110y
0
430
Cloud Run + Observability
110y
3
12k
Proxyless Service Mesh with gRPC
110y
3
1.3k
Other Decks in Technology
See All in Technology
新規ゲームのリリース(開発)前からのSRE活動
tmkoikee
1
630
Puny to Powerful PostgreSQL Rails Apps
andyatkinson
PRO
0
410
CAMのサービス開発の歴史と共通基盤を使った 開発スタイルへの変遷について
ishikawa_pro
0
110
ソフトウェアテストで参考にしている67のモノ #scrumniigata / 67 things for software testing
kyonmm
PRO
1
840
NestJS + Prisma2 で歩む RLS の世界
ynaka81
1
100
Apache Kafka and the World of Streams
hashitokyo
0
130
プロダクトの理想と現実はなぜ乖離しがち?プロダクト作りに潜む問題を考える
suzukentaro
0
270
Who owns the Service Level?
chaspy
5
1.5k
プルリク作ったらデプロイされる仕組み on ECS / SRE NEXT 2022
carta_engineering
1
660
プロダクション環境の信頼性を損ねず観測する技術
egmc
4
910
Scrum Fest Niigata 2022 開発エンジニアに聞いてみよう!
moritamasami
1
680
LINE API Update 20220525
line_developers_tw
PRO
0
100
Featured
See All Featured
KATA
mclloyd
7
8.6k
Making Projects Easy
brettharned
98
4.3k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
119
28k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
224
49k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
15k
Six Lessons from altMBA
skipperchong
14
1.3k
Embracing the Ebb and Flow
colly
73
3.3k
What the flash - Photography Introduction
edds
61
10k
Put a Button on it: Removing Barriers to Going Fast.
kastner
56
2.3k
Art Directing for the Web. Five minutes with CSS Template Areas
malarkey
196
9.4k
Docker and Python
trallard
27
1.5k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
349
27k
Transcript
1 My Favorite Talks at GopherCon 2019 Yuki ITO
Table of Contents !2 Observability in Go Socket to Me:
Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2:
Table of Contents !3 Observability in Go Socket to Me:
Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2:
Observability in Go !4 https://github.com/freeformz/go-observability-workshop/ Repository
Observability in Go !5 Log Trace Metric 3 Pillars for
Observability
Observability in Go !6 Log Trace Metric 3 Pillars for
Observability
Logs !7 log package func handler(w http.ResponseWriter, r *http.Request) {
status := http.StatusOK // net/http returns 200 by default defer func(t time.Time) { log.Printf( "%s %q => %d (%2.3fs)\n", r.Method, r.URL.String(), status, time.Since(t).Seconds(), ) }(time.Now()) // … }
Logs !8 log package func main() { log.SetFlags(log.Flags() | log.Lshortfile)
log.Println("Listening at: http://localhost:" + port) http.ListenAndServe(":"+port, nil) // ... } 2019/08/20 19:00:00 server.go:54: Listening at: http://localhost:8080
Logs !9 Structured Logs github.com/sirupsen/logrus 2019/08/20 19:00:00 server.go:54: Listening at:
http://localhost:8080 { “level”:"info", "msg":"Listening at: http://localhost:8080”, “time":"2019-07-17T21:26:08-07:00" }
!10 Log Trace Metric How should we log? If the
only observability tool we have is logs, then we should log everything.
!11 Log Trace Metric How should we log? If we
have metrics and traces, then reduce logging to important events meant largely for human consumption.
Observability in Go !12 Log Trace Metric 3 Pillars for
Observability
Metrics !13 expvar package import ( "expvar" // ... )
func main() { // ... // Expose the port value ep := expvar.NewString("Port") ep.Set(port) http.ListenAndServe(":"+port, nil) //... }
Metrics !14 expvar package package expvar // ... func init()
{ http.HandleFunc("/debug/vars", expvarHandler) Publish("cmdline", Func(cmdline)) Publish("memstats", Func(memstats)) }
Metrics !15 expvar package $ curl -s http://localhost:8080/debug/vars | jq
.Port "8080"
Metrics !16 expvar makes us be able to expose metrics
We can recognize only `current status` of system. Hoever…
Metrics !17 Prometheus Time Series Database
Metrics !18 import ( // ... "github.com/prometheus/client_golang/prometheus/promhttp" ) func main()
{ // ... http.Handle("/metrics", promhttp.Handler()) // … http.ListenAndServe(":"+port, nil) // ... }
Metrics !19 Pull Model Prometheus Scrape `/metrics` Application server
Metrics !20
Observability in Go !21 Log Trace Metric 3 Pillars for
Observability
Traces !22 Microservices ServiceB ServiceA
Traces !23 Distributed Tracing Jaeger
Traces !24 import ( // ... "contrib.go.opencensus.io/exporter/jaeger" "go.opencensus.io/trace" ) func
main() { je, err := jaeger.NewExporter(jaeger.Options{ CollectorEndpoint: "http://localhost:14268/api/traces", Process: jaeger.Process{ ServiceName: "servicea", Tags: []jaeger.Tag{ jaeger.StringTag("server", "1"), jaeger.StringTag("port", port), jaeger.BoolTag("demo", true), }, }, }) trace.RegisterExporter(je) }
Traces !25 func queryServiceBHandler(c *http.Client, url string) http.HandlerFunc { return
func(w http.ResponseWriter, r *http.Request) { ctx, span := trace.StartSpan( r.Context(), "queryServiceBHandler", ) defer span.End() // ... } }
Traces !26
Traces !27
Observability in Go !28 Log Trace Metric 3 Pillars for
Observability
Table of Contents !29 Observability in Go Socket to Me:
Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2:
!30 System calls under net.Listen & Accept https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
!31 System calls under net.Listen & Accept https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
!32 func (srv *Server) Serve(l net.Listener) error { // ...
for { rw, e := l.Accept() // ... c := srv.newConn(rw) go c.serve(ctx) } } Socket to me net/http/server.go
!33 System calls under net.Listen & Accept https://about.sourcegraph.com/go/gophercon-2019-socket-to-me-where-do-sockets-live-in-go
Socket Options in Go !34 ListenConfig.Listen() ListenConfig.Control RawConn.Control() Callback to
set socket options
Socket Options in Go !35 https://github.com/110y/sockoptgo/ Example Implementation
Socket Options in Go !36 ListenConfig.Listen() ListenConfig.Control RawConn.Control() Callback to
set socket options
!37 func Listen(network, address string) (Listener, error) { var lc
ListenConfig return lc.Listen(context.Background(), network, address) } net.Listen() Socket Options in Go
!38 Socket Options in Go lc := net.ListenConfig{ Control: listenControler,
} lis, err := lc.Listen(ctx, "tcp4", addr)
Socket Options in Go !39 ListenConfig.Listen() ListenConfig.Control RawConn.Control() Callback to
set socket options
!40 func listenControler(network, address string, c syscall.RawConn) error { return
c.Control(func(fd uintptr) { syscall.SetsockoptInt( int(fd), syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1, ) }) } Socket Options in Go
!41 func listenControler(network, address string, c syscall.RawConn) error { return
c.Control(func(fd uintptr) { syscall.SetsockoptInt( int(fd), syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1, ) }) } Socket Options in Go
Table of Contents !42 Observability in Go Socket to Me:
Where do Sockets Live in Go? Day 0 (Pre-Conference Workshop): Day 2: