Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
You, latency and profiling @ GopherCon India 2017
Search
Filippo Valsorda
February 25, 2017
Programming
13
4.1k
You, latency and profiling @ GopherCon India 2017
Filippo Valsorda
February 25, 2017
Tweet
Share
More Decks by Filippo Valsorda
See All by Filippo Valsorda
Asynchronous networking @ GopherCon 2018
filosottile
3
2.1k
Le note cifrate di Antonio Marzi
filosottile
1
410
Why cgo is slow @ CapitalGo 2018
filosottile
2
4.6k
Squeezing a key through a carry bit @ 34c3
filosottile
0
1.7k
Calling Rust from Go, without cgo @ GothamGo 2017
filosottile
1
2.7k
You, latency and profiling @ GolangUK 2017
filosottile
0
1.2k
Encrypting the Internet with Go @ GopherCon 2017
filosottile
9
2.7k
TLS 1.3 @ 33c3
filosottile
4
6.9k
Stealing Bitcoin With Math - HOPE XI
filosottile
13
30k
Other Decks in Programming
See All in Programming
LLMとPlaywright/reg-suitを活用した jQueryリファクタリングの実際
kinocoboy2
4
670
私はどうやって技術力を上げたのか
yusukebe
43
17k
overlayPreferenceValue で実現する ピュア SwiftUI な AdMob ネイティブ広告
uhucream
0
110
メモリ不足との戦い〜大量データを扱うアプリでの実践例〜
kwzr
1
850
アメ車でサンノゼを走ってきたよ!
s_shimotori
0
140
iOSアプリの信頼性を向上させる取り組み/ios-app-improve-reliability
shino8rayu9
0
150
uniqueパッケージの内部実装を支えるweak pointerの話
magavel
0
920
Web フロントエンドエンジニアに開かれる AI Agent プロダクト開発 - Vercel AI SDK を観察して AI Agent と仲良くなろう! #FEC余熱NIGHT
izumin5210
3
390
CSC305 Lecture 01
javiergs
PRO
1
400
デミカツ切り抜きで面倒くさいことはPythonにやらせよう
aokswork3
0
180
大規模アプリのDIフレームワーク刷新戦略 ~過去最大規模の並行開発を止めずにアプリ全体に導入するまで~
mot_techtalk
0
380
ポスターセッション: 「まっすぐ行って、右!」って言ってラズパイカーを動かしたい 〜生成AI × Raspberry Pi Pico × Gradioの試作メモ〜
komofr
0
940
Featured
See All Featured
A better future with KSS
kneath
239
17k
Designing Experiences People Love
moore
142
24k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.5k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
32
2.2k
Why Our Code Smells
bkeepers
PRO
339
57k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
45
2.5k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
610
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Raft: Consensus for Rubyists
vanstee
139
7.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
The Cult of Friendly URLs
andyhume
79
6.6k
Transcript
You, latency and profiling Filippo Valsorda — @FiloSottile
What is fast?
A fast hash… … can hash many MB per second
A fast regex… … can process many MB/s
A fast database… … can store many GB per second
A fast database… … can store many GB per second
… or replies to queries in a few milliseconds
A fast website… … can handle many requests at the
same time … or loads in a few milliseconds
A fast API… … can handle many clients at the
same time … or answers in a few milliseconds
Fast is… Throughput and Latency
CPU profiling -cpuprofile /debug/pprof/profile
CPU profiling SIGPROF
runtime/proc.go
CPU profiling http.Handler 1 Running on CPU http.Handler 2 I/O
I/O http.Handler 3 I/O http.Handler 4 I/O
CPU profiling Running on CPU I/O I/O I/O I/O http.Handler
1 http.Handler 2 http.Handler 3 http.Handler 4
CPU profiling I/O Observed latency http.Handler 1
func Write(data []byte) { for i := 0; i <
50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total go tool pprof -web
slowserver cpu.pprof
None
$ time curl http://127.0.0.1:12345/hash-and-write 9.831 total $ time
curl http://127.0.0.1:12345/write-no-hash 7.692 total
Latency is not all about CPU
CPU profiling is for throughput The tracer is for latency
$ time curl http://127.0.0.1:12345/write-no-hash 7.692 total go tool trace -pprof=syscall
trace.out > syscall.pprof go tool pprof -web syscall.pprof
None
The tracer Detailed nanosecond-level log of execution events: • goroutine
scheduling, blocking • syscalls, network, I/O • garbage collection
The tracer • log files are BIG • 25% overhead
(after Go 1.7) • symbols are embedded (Go 1.7)
Trace vs. profile • Full event context • Goroutine metadata
and history • Discrete events, not samples
The tracer import "runtime/trace" import "net/http/pprof" /debug/pprof/trace?seconds=5
None
go tool trace -pprof=TYPE trace.out Supported profile types are: -
net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
go tool trace -pprof=syscall trace.out > syscall.pprof go tool pprof
-web syscall.pprof
go tool trace -pprof=syscall trace.out > syscall.pprof go-torch -b syscall.pprof
&& open torch.svg
None
None
-pprof=syscall func Write(data []byte) { for i := 0; i
< 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
-pprof=sync // Block waits on a channel func Block() {
<-time.After(1 * time.Second) }
-pprof=sync
-pprof=sync
-pprof=net func Download() []byte { r, err := http.Get("https://golang.org/doc/go1.8") body,
err := ioutil.ReadAll(r.Body) r.Body.Close() return body }
-pprof=net
-pprof=net
None
Writing our own profile prof := make(map[uint64]pprof.Record) for _, ev
:= range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
var childG = make(map[uint64]struct{}) var lastGLen int for { for
_, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) {
continue }
tracefocus tracefocus -filter=ServeHTTP trace.out > net.pprof github.com/FiloSottile/tracetools
None
None
Build more tools! • Focus on goroutine number • Aggregate
all blocking types • Visualizations • …
tracehist tracehist -filter=Close trace.out github.com/FiloSottile/tracetools
None
Build more tools! Profile latency!
Questions? Filippo Valsorda @FiloSottile filippo@cloudflare.com