Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
You, latency and profiling @ GopherCon India 2017
Search
Filippo Valsorda
February 25, 2017
Programming
13
3.6k
You, latency and profiling @ GopherCon India 2017
Filippo Valsorda
February 25, 2017
Tweet
Share
More Decks by Filippo Valsorda
See All by Filippo Valsorda
Asynchronous networking @ GopherCon 2018
filosottile
3
1.8k
Le note cifrate di Antonio Marzi
filosottile
1
290
Why cgo is slow @ CapitalGo 2018
filosottile
2
3.9k
Squeezing a key through a carry bit @ 34c3
filosottile
0
1.4k
Calling Rust from Go, without cgo @ GothamGo 2017
filosottile
1
2.3k
You, latency and profiling @ GolangUK 2017
filosottile
0
960
Encrypting the Internet with Go @ GopherCon 2017
filosottile
9
2.5k
TLS 1.3 @ 33c3
filosottile
4
6.4k
Stealing Bitcoin With Math - HOPE XI
filosottile
13
29k
Other Decks in Programming
See All in Programming
Cloud RunとCloud PubSubでサーバレスなデータ基盤2024 with Terraform / Cloud Run and PubSub with Terraform
shinyorke
7
1.9k
AppDeveloperCon 2024 EU: Building polyglot developer experiences in 2024
salaboy
0
380
object-oriented-conference-2024
fuwasegu
6
1.8k
Honoとhtmx
yusukebe
6
1.2k
受託開発でGitLab CI を活用していく
xiombatsg
1
110
MySQL のインデックスの種類をおさらいしよう! / overviewing indexes in MySQL
okashoi
0
170
Building a Smaller App Binary
kateinoigakukun
2
200
とにかくHTTP3をライトニングに話す / Anyway, I'll talk to Lightning about HTTP3.
seike460
PRO
0
120
ファイル先頭の use の意味、説明できますか? 〜PHP の namespace と autoloading の関係を正しく理解しよう〜 / namespace and autoloading in php
okashoi
2
470
Material 3で Material 2ぽい見た目にする
numeroanddev
2
250
Enhancing Applications with Accessibility API
kishikawakatsumi
3
890
Understanding Ast By Looking
inouehi
0
120
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
355
22k
A better future with KSS
kneath
230
16k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
272
12k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
16k
What's in a price? How to price your products and services
michaelherold
236
11k
Writing Fast Ruby
sferik
619
59k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
19
1.6k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
352
28k
In The Pink: A Labor of Love
frogandcode
137
21k
jQuery: Nuts, Bolts and Bling
dougneiner
57
7.1k
Docker and Python
trallard
33
2.6k
How To Stay Up To Date on Web Technology
chriscoyier
781
250k
Transcript
You, latency and profiling Filippo Valsorda — @FiloSottile
What is fast?
A fast hash… … can hash many MB per second
A fast regex… … can process many MB/s
A fast database… … can store many GB per second
A fast database… … can store many GB per second
… or replies to queries in a few milliseconds
A fast website… … can handle many requests at the
same time … or loads in a few milliseconds
A fast API… … can handle many clients at the
same time … or answers in a few milliseconds
Fast is… Throughput and Latency
CPU profiling -cpuprofile /debug/pprof/profile
CPU profiling SIGPROF
runtime/proc.go
CPU profiling http.Handler 1 Running on CPU http.Handler 2 I/O
I/O http.Handler 3 I/O http.Handler 4 I/O
CPU profiling Running on CPU I/O I/O I/O I/O http.Handler
1 http.Handler 2 http.Handler 3 http.Handler 4
CPU profiling I/O Observed latency http.Handler 1
func Write(data []byte) { for i := 0; i <
50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total go tool pprof -web
slowserver cpu.pprof
None
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total $ time
curl http://127.0.0.1:12345/write-no-hash 7.692 total
Latency is not all about CPU
CPU profiling is for throughput The tracer is for latency
$ time curl http://127.0.0.1:12345/write-no-hash 7.692 total go tool trace -pprof=syscall
trace.out > syscall.pprof go tool pprof -web syscall.pprof
None
The tracer Detailed nanosecond-level log of execution events: • goroutine
scheduling, blocking • syscalls, network, I/O • garbage collection
The tracer • log files are BIG • 25% overhead
(after Go 1.7) • symbols are embedded (Go 1.7)
Trace vs. profile • Full event context • Goroutine metadata
and history • Discrete events, not samples
The tracer import "runtime/trace" import "net/http/pprof" /debug/pprof/trace?seconds=5
None
go tool trace -pprof=TYPE trace.out Supported profile types are: -
net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
go tool trace -pprof=syscall trace.out > syscall.pprof go tool pprof
-web syscall.pprof
go tool trace -pprof=syscall trace.out > syscall.pprof go-torch -b syscall.pprof
&& open torch.svg
None
None
-pprof=syscall func Write(data []byte) { for i := 0; i
< 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
-pprof=sync // Block waits on a channel func Block() {
<-time.After(1 * time.Second) }
-pprof=sync
-pprof=sync
-pprof=net func Download() []byte { r, err := http.Get("https://golang.org/doc/go1.8") body,
err := ioutil.ReadAll(r.Body) r.Body.Close() return body }
-pprof=net
-pprof=net
None
Writing our own profile prof := make(map[uint64]pprof.Record) for _, ev
:= range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
var childG = make(map[uint64]struct{}) var lastGLen int for { for
_, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) {
continue }
tracefocus tracefocus -filter=ServeHTTP trace.out > net.pprof github.com/FiloSottile/tracetools
None
None
Build more tools! • Focus on goroutine number • Aggregate
all blocking types • Visualizations • …
tracehist tracehist -filter=Close trace.out github.com/FiloSottile/tracetools
None
Build more tools! Profile latency!
Questions? Filippo Valsorda @FiloSottile filippo@cloudflare.com