Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
You, latency and profiling @ GolangUK 2017
Search
Filippo Valsorda
August 17, 2017
Programming
0
1k
You, latency and profiling @ GolangUK 2017
Filippo Valsorda
August 17, 2017
Tweet
Share
More Decks by Filippo Valsorda
See All by Filippo Valsorda
Asynchronous networking @ GopherCon 2018
filosottile
3
1.9k
Le note cifrate di Antonio Marzi
filosottile
1
330
Why cgo is slow @ CapitalGo 2018
filosottile
2
4.1k
Squeezing a key through a carry bit @ 34c3
filosottile
0
1.5k
Calling Rust from Go, without cgo @ GothamGo 2017
filosottile
1
2.4k
Encrypting the Internet with Go @ GopherCon 2017
filosottile
9
2.5k
You, latency and profiling @ GopherCon India 2017
filosottile
13
3.8k
TLS 1.3 @ 33c3
filosottile
4
6.5k
Stealing Bitcoin With Math - HOPE XI
filosottile
13
29k
Other Decks in Programming
See All in Programming
CSC307 Lecture 11
javiergs
PRO
0
240
コード生成を伴うLLMエージェント - 2024.07.18 Tokyo AI
smiyawaki0820
11
4.1k
AWS CDKにおける「再利用性」を考える / aws-cdk-reusability
gotok365
6
1.3k
CSC307 Lecture 14
javiergs
PRO
0
220
「2024年版 Kotlin サーバーサイドプログラミング実践開発」の補講 〜O/Rマッパー編〜
n_takehata
2
260
日付と正規化
megmogmog1965
0
140
Clean Architecture by TypeScript & NestJS
ryounasso
0
150
Microservices rules (July 2024) : what good looks like
cer
PRO
0
1.6k
Harnessing Large Language Models for Training-free Video Anomaly Detection
tereka114
1
1.3k
大規模マルチテナントを解決するYugabyteDBという選択肢
nnaka2992
1
250
Android開発者のための Kotlin Multiplatform入門
ntaro
0
190
20240706_CDKConf
takuyay0ne
0
1.2k
Featured
See All Featured
The MySQL Ecosystem @ GitHub 2015
samlambert
248
12k
Documentation Writing (for coders)
carmenintech
63
4.2k
The Power of CSS Pseudo Elements
geoffreycrofte
64
5.2k
Rails Girls Zürich Keynote
gr2m
93
13k
We Have a Design System, Now What?
morganepeng
46
7k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
224
21k
Code Reviewing Like a Champion
maltzj
517
39k
Faster Mobile Websites
deanohume
303
30k
Mobile First: as difficult as doing things right
swwweet
219
8.8k
Code Review Best Practice
trishagee
58
16k
Designing with Data
zakiwarfel
96
5k
4 Signs Your Business is Dying
shpigford
178
21k
Transcript
You, latency and profiling Filippo Valsorda — @FiloSo/le
What is fast?
A fast hash… … can hash many MB per second
A fast regex… … can process many MB/s
A fast database… … can store many GB per second
A fast database… … can store many GB per second
… or replies to queries in a few ms
A fast website… … can handle many requests … or
loads in a few milliseconds
A fast API… … can handle many clients … or
answers in a few milliseconds
Fast is… Throughput and Latency
CPU profiling -cpuprofile /debug/pprof/profile
SIGPROF CPU profiling
runtime/proc.go
h>p.Handler 1 Running on CPU h>p.Handler 2 I/O I/O h>p.Handler
3 I/O h>p.Handler 4 I/O CPU profiling
Running on CPU I/O I/O I/O I/O h>p.Handler 1 h>p.Handler
2 h>p.Handler 3 h>p.Handler 4 CPU profiling
I/O Observed latency h>p.Handler 1 CPU profiling
func Write(data []byte) { for i := 0; i <
50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total go tool pprof -web
slowserver cpu.pprof
None
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total $ time
curl http://127.0.0.1:12345/write-no-hash 7.692 total
Latency is not all about CPU
CPU profiling is for throughput The tracer is for latency
$ time curl http://127.0.0.1:12345/write-no-hash 7.692 total go tool trace -pprof=syscall
trace.out > syscall.pprof go tool pprof -web syscall.pprof
None
The tracer Detailed nanosecond-level log of execuOon events: • gorouOne
scheduling, blocking • syscalls, network, I/O • garbage collecOon
•log files are BIG •25% overhead (aWer Go 1.7) •symbols
are embedded (Go 1.7) The tracer
•Full event context •GorouOne metadata and history •Discrete events, not
samples Trace vs. profile
CPU profiling Total: 20 samples
Tracing events
import "runtime/trace" import "net/http/pprof" /debug/pprof/trace?seconds=5 The tracer
None
go tool trace -pprof=TYPE trace.out Supported profile types are: -
net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
go tool trace -pprof=syscall trace.out > syscall.pprof go tool pprof
-web syscall.pprof
go tool trace -pprof=syscall trace.out > syscall.pprof go-torch -b syscall.pprof
&& open torch.svg
None
None
-pprof=syscall func Write(data []byte) { for i := 0; i
< 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
-pprof=sync // Block waits on a channel func Block() {
<-time.After(1 * time.Second) }
-pprof=sync
-pprof=sync
-pprof=net func Download() []byte { r, err := http.Get("https://golang.org/doc/go1.8") body,
err := ioutil.ReadAll(r.Body) r.Body.Close() return body }
-pprof=net
-pprof=net
-pprof=net
None
WriIng our own profile prof := make(map[uint64]pprof.Record) for _, ev
:= range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
var childG = make(map[uint64]struct{}) var lastGLen int for { for
_, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
tracefocus tracefocus -filter=ServeHTTP trace.out > net.pprof github.com/FiloSottile/tracetools
None
None
Build more tools! • Focus on gorouOne number • Aggregate
all blocking types • VisualizaOons • …
tracehist tracehist -filter=Close trace.out github.com/FiloSottile/tracetools
None
Build more tools! Profile latency!
QuesIons? Filippo Valsorda @FiloSo/le hi@filippo.io