Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
OSS Performance Tuning Tips
Search
orisano
October 28, 2019
Programming
8
5.9k
OSS Performance Tuning Tips
orisano
October 28, 2019
Tweet
Share
More Decks by orisano
See All by orisano
Docker-Compose & BuildKit
orisano
4
1k
Container Build Talk
orisano
3
2.5k
dockerignore talk
orisano
2
7.1k
Better docker image+
orisano
6
6.1k
Socket.IO Introduction
orisano
0
3.2k
Profiling Go Application
orisano
11
7.8k
Multi-stage Builds Patterns & Practice
orisano
6
5.1k
better docker image
orisano
22
30k
the world of characters
orisano
8
1.4k
Other Decks in Programming
See All in Programming
型付きアクターモデルがもたらす分散シミュレーションの未来
piyo7
0
810
Webの外へ飛び出せ NativePHPが切り拓くPHPの未来
takuyakatsusa
2
450
CursorはMCPを使った方が良いぞ
taigakono
1
200
LT 2025-06-30: プロダクトエンジニアの役割
yamamotok
0
600
git worktree × Claude Code × MCP ~生成AI時代の並列開発フロー~
hisuzuya
1
510
Blazing Fast UI Development with Compose Hot Reload (droidcon New York 2025)
zsmb
1
260
なぜ適用するか、移行して理解するClean Architecture 〜構造を超えて設計を継承する〜 / Why Apply, Migrate and Understand Clean Architecture - Inherit Design Beyond Structure
seike460
PRO
1
710
Team operations that are not burdened by SRE
kazatohiei
1
270
Create a website using Spatial Web
akkeylab
0
310
DroidKnights 2025 - 다양한 스크롤 뷰에서의 영상 재생
gaeun5744
3
330
Benchmark
sysong
0
280
来たるべき 8.0 に備えて React 19 新機能と React Router 固有機能の取捨選択とすり合わせを考える
oukayuka
2
870
Featured
See All Featured
Automating Front-end Workflow
addyosmani
1370
200k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Fireside Chat
paigeccino
37
3.5k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Become a Pro
speakerdeck
PRO
28
5.4k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.4k
It's Worth the Effort
3n
185
28k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Code Reviewing Like a Champion
maltzj
524
40k
Art, The Web, and Tiny UX
lynnandtonic
299
21k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Transcript
OSS Performance Tuning Tips #gocon #gocon_hall GoCon 2019 Autumn @orisano
త ̎ͭͷ͜ͱΛڞ༗͍ͨ͠
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ Ҿ͖ग़͠Λ૿͢
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ Ҿ͖ग़͠Λ૿͢
Ͳ͏ਐΊ͍ͯΔͷ͔
0. ෆຬͷϋʔυϧΛԼ͛Δ
ෆຬ͕ վળͷ͖͔͚ͬʹͳΓ·͢
ීஈ͍ͬͯΔ ϥΠϒϥϦɺιϑτΣΞ͕ ͍ͷͩͱࢥͬͯΈΔ
͖͔͚ͬͱ ϞνϕʔγϣϯΛ ࣗΒ࡞Γग़͠·͠ΐ͏
1. ܭଌͷϋʔυϧΛԼ͛Δ
͍ݪҼΛ ؾܰʹ ௐΒΕΔೳྗΛʹ͚ͭΔ
ख͕͔͔ؒΔߦҝΛ ਓؒ͋·Γ͠ͳ͍
ͨ͘͞Μܦݧ͢Δ
ؾܰʹ ௐࠪͰ͖ΔΑ͏ͳΈΛ ೖΕΔ
͜ΕOSSΛެ։͢Δଆͷ ڥมͰࢦఆͨ͠Γ σϑΥϧτͰprofile͕औΕΔ ΈΛఏڙ͢Δͱ վળ͍͢͠/͞Ε͍͢
ʹૺ۰ͨ͠ਓ͕ ݪҼௐ͍ࠪ͢͠ͱخ͍͠
2. ՕॴΛಛఆ͢Δ
͍ͱײͯ͡ Օॴ͕ؔϨϕϧͰ Θ͔Βͳ͍ͷ cliΛ࣮ߦ͍ͯ͠Δͱ͖
mainʹ github.com/pkg/profile ΛՃͯ͠ΈΔ
None
͜Ε͚ͩ
cliͷ࣮ߦ͕1ʹ 1࣌ؒҎ্͔͔Δ߹ͳͲ net/http/pprofͷ΄͏͕ ྑ͍߹
profileΛݟͯ ؔϨϕϧͰಛఆ͢Δ
pprofͷweb൛ʹ׳ΕΔ ͪΌΜͱҙຯ͕Θ͔Δ
͍ΖΜͳݟํͰݟͯΈΔ top, graph, flame-graph source, disasm
3. BenchmarkΛॻ͘
͢Ͱʹॻ͍ͯ͋Δ߹ ϥοΩʔ ͘ͳ͍ͬͯͳ͍߹ ͘ͳΔέʔεΛՃ͢Δ
BenchmarkΛॻ࣌͘ ࠷దԽͰফ͍͑ͯͳ͍͔֬ೝ
BenchmarkΛॻ͘ͱ ؆୯ʹprofile͕औΕΔ
go test -cpuprofile go test -memprofile Λ֮͑Α͏ (go help testflagΛಡ͏)
4. ղܾࡦΛߟ͑Δ
Ͳ͏ղܾ͢Δ͔ʹ͍ͭͯ “Ҿ͖ग़͠Λ૿͢”Ͱ
OSSͷ ύϑΥʔϚϯενϡʔχϯά Ϛʔδ͞Εͯྃ
Ͳ͏ΕϚʔδ͞ΕΔ ύονʹͳΔ͔
Ϛʔδ͢Δͱ͍͏͜ͱ ૬ख͕ϝϯςφϯε͢Δ ͱ͍͏͜ͱ
มߋ͕গͳ͍ ૬खͷίετΛ૿͞ͳ͍
ഁյతมߋ͕ͳ͍ ͘ΘΕ͍ͯΔͷ΄Ͳ ॏཁͳͱ͜Ζ
ҟৗͳίετ͕ͳ͍ มߋ, อक͕͘͠ͳΔ࣮ (ΞηϯϒϥΛͬͨΓ)
ґଘؔΛՃ͢Δ߹ ৻ॏʹબ͢Δ ΞΫςΟϒʹϝϯςφϯε͞ Ε͍ͯΔͷ͔ ຊʹඞཁ͔?
͘͢͝ύϑΥʔϚϯε͕ վળ͢Δ ܶతͳվળड͚ೖΕΒΕΔ
ςετ͕ॻ͍ͯ͋Δ ͢Ͱʹॻ͍ͯ͋Δ߹डཧ ͞Ε͍͢
5. ࢼߦࡨޡ͢Δ
·ͣॳظঢ়ଶͰ ेͳճ timeoutʹͳΒͳ͍Α͏ʹ Benchmark݁ՌΛऔ͓ͬͯ͘
go test -bench . -count=10 -timeout=30000s | tee old.txt
ύονΛૹΔͱ͖ʹ benchstatͷ݁ՌΛૹΔͷͰ ϕʔεϥΠϯेͳճͰ
timeout σϑΥϧτͩͱ10mͰ ҙ֎ͱΦʔόʔ͢Δ
ղܾࡦΛࢥ͍͍ͭͨΒ ·ͣ1Benchmark
ے͕ྑͦ͞͏ͳΒ ेͳճBenchmark
ଟ͘ࢼߦࡨޡ͍ͯ͠Δͱ profileͱsourceͷ͕ؔ Θ͔Βͳ͘ͳΔ
pprofʹ sourceΛݟΔػೳ͕͋Δ͕ อ͍࣋ͯ͠ΔΘ͚Ͱͳ͍ pathͷΈ
git add . git commit -m “$2” REV=$(git rev-parse HEAD)
go test -bench $1 -benchmem -cpuprofile cpu.${REV}.pb.gz -memprofile mem.$ {REV}.pb.gz | tee ${REV}.txt rm ./${REV}.* && git reset HEAD^
sourcegitʹཧͯ͠Β͏ ͪΌΜͱඥ͚ΒΕΔ
ଟ͘ͷprofile͕͋Δͱ ୯ମͷޮՌ͕Θ͔Βͳ͍ ͜ͱ͕͋Δ
go tool pprof -diff_base Ͱprofileؒͷ͕ࠩݟΕΔ
6. ύονΛૹΔ
࠷ऴͷBenchmarkΛ ेͳճߦ͏
ઐ༻ͷΠϯελϯεͰܭଌ ͍͠߹ ۃྗ֎෦ϓϩηεΛఀࢭ͢Δ
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ Ҿ͖ग़͠Λ૿͢
࣮ྫϕʔεͰ ͲͷΑ͏ʹղܾ͔ͨ͠ ࣗͳΒͲ͏ղܾ͢Δ͔ ߟ͑ͯΒ͍͍ͨ
1. ࣮ࡍʹ͛ͨύον 2. ͍ϥΠϒϥϦ͕ॻ͖͍ͨ
1. ࣮ࡍʹ͛ͨύον 2. ͍ϥΠϒϥϦ͕ॻ͖͍ͨ
src-d/go-git
pure-goͰ gitͷૢ࡞Λ͢ΔͨΊͷ ϥΠϒϥϦ
ύονΛૹͬͨഎܠ
aquasecurity/trivy Ͱॳ͍͕ͬͯͨ େ͖ͳrepositoryͷcloneʹ ҟৗͳ࣌ؒ(10min~)͕͔͔Δ ͕͋ͬͨ
ݪҼ gitͷIndexͷߏஙΛߦ͏Օॴ ͩͬͨ
Indexͷ෦දݱ(public)͕ sliceʹͳ͓ͬͯΓ nameͰҰҙʹ͢ΔͨΊʹ deleteͱappend͕ ϑΝΠϧճ࣮ߦ͞Ε͍ͯͨ
deletenameΛࢦఆͯ͠ আ͢ΔͷͰ sliceͷཁૉΛͯࠪ͢͢Δ
ॳ ෦දݱΛmapʹมߋ͢Ε deleteͷܭࢉྔΛݮΒͤΔ ͷͰͦ͏͠Α͏ͱࢥͬͨ
͔͠͠ publicͳϑΟʔϧυ internalͰͳ͍ύοέʔδ 4500 star͑ͷ໊
౸ఈड͚ೖΕΒΕͳ͍ มߋͩͱࢥ͍ఘΊͨ
ޙ
ιʔείʔυΛ ݟ͍ͯ͠Δͱ
໌ه͞Ε͍ͯΔ ϑΟʔϧυͷ༷ (sliceͷॱংอূ͠ͳ͍) େྔͷIndexΛߏங͢Δ λΠϛϯά(clone࣌)
ݟͨ݁͠Ռ ഁյతมߋແ͠Ͱ मਖ਼͢Δํ๏Λࢥ͍͍ͭͨ
ଟ͘ݺͼग़͞ΕΔՕॴ ͷΈʹ͓͍ͯ mapͰอ࣋͠return͢Δࡍʹ sliceʹม͢Δ
mapΛprivateͳؔͰ Ҿ͖ճ͢Α͏ʹ͢Δ͜ͱͰ ഁյతมߋΛճආͰ͖ͨ
None
ͦ͜Λղܾ͢Δͱ mallocgc͕ॏ͘ͳͬͨ
ϝϞϦ༻ྔͷݪҼ io.Copyʹ͋ΔΑ͏ͩͬͨ
ϑΝΠϧ͝ͱʹio.Copy͕ ݺΕ͍ͯͨ
େྔʹݺΕΔՄೳੑ͕͋Δ ՕॴͰio.CopyͰͳ͘ io.CopyBufferΛ͏ͱ ϝϞϦ༻ྔΛ੍ޚͰ͖Δ
͏bufferΛͲ͏ࢦఆ͢Δ͔ ֎෦͔Β༩͑ΒΕͳ͍ͱ ݁ہϝϞϦ༻ྔมΘΒͳ͍
privateͳϝιουͱ͍͑ ҾʹՃ͢Δͱ มߋൣғ͕େ͖͘ͳΔ
ղܾࡦͱͯ͠ globalʹsync.PoolΛ ஔ͘͜ͱʹͨ͠
None
https://github.com/src-d/ go-git/pull/1179
None
605ඵ -> 249ඵ
͔͠͠ ·ͩ·ͩϝϞϦ༻ྔ͕ଟ͍ 56 GB/op
ϑΝΠϧʹࠩΛద༻͢Δ ॲཧ͕ϝϞϦΛେྔʹ༻
ؔ෦Ͱ publicͷؔͰ buffer͕֎෦͔Βͤͳ͍ ͷ͕ΘΕ͍ͯͨ
গͳ͘ͱ෦͔Βͷ༻ ͰbufferΛࢦఆ͍ͨ͠
None
෦͚ʹbufferࢦఆͰ͖Δ Α͏ʹͯ͠ղܾ
https://github.com/src-d/ go-git/pull/1180
56.1 GB -> 29.8 GB
None
None
image/png
ύονΛૹͬͨഎܠ
ࣾISUCONͰ QRίʔυΛߴʹੜ͢Δ ඞཁ͕͋ͬͨͨΊ
ࣾISUCONͷৼΓฦΓͰ ύϑΥʔϚϯενϡʔχϯά ΛҰਓͰָ͠ΜͰ͍ͨ
ऄ
ߴʹେྔͷpngΛ ग़ྗ͢ΔͨΊʹ
1.9͔Βೖͬͨ png.EncoderBufferPool
CompressionLevelͷઃఆ
ಠࣗimage.Image࣮Λ Θͳ͍
നࠇը૾Ͱ͋Ε image.GrayΛ͏ (Opaque͕bypassͰ͖Δ)
ऄऴྃ
image/png͕ bottleneckʹͳ͖ͬͯͨ
͢ͰʹBenchmark͕ ॻ͔Ε͍ͯͨͷͰ࣮ߦ
ࠓճ͍ͬͯΔ ՕॴͰͳ͍͕࿐ࠎʹ͍ ෦͕͋ͬͨ
https://go- review.googlesource.com/ c/go/+/187417
None
࣮ࡍॏ͔ͬͨՕॴ compress/deflate
pprof͕ॏ͍ͱࣔͨ͠ͷ for͕ॻ͔Ε͍ͯΔߦͩͬͨ
None
֘forจͷasmΛݟͯΈΔͱ ແବͳϝϞϦΞΫηε͕
compilerʹregisterΛ ͬͯΒ͏ͨΊʹ ϩʔΧϧมΛఆٛ
None
https://go- review.googlesource.com/ c/go/+/187837
None
None
GoogleContainerTools/ kaniko
ύονΛૹͬͨഎܠ
ΞΠσΟΞ͕εΩͩͬͨ
kanikoΛCIͰͬͯ Կʹ͕͔͔͍࣌ؒͬͯΔͷ͔ ؾʹͳͬͨ
kanikoϝϞϦ্ʹ filesystemͷsnapshotΛ࣋ͭ
ίϚϯυΛ࣮ߦ͢Δͨͼʹ ͕ࠩͳ͍͔ൺֱ͢Δ
md5Ͱൺֱ͢Δ
ͦͷmd5͕ॏ͔ͬͨ
ϑΝΠϧ͕ ಉҰ͔Ͳ͏͔͚ͩͰྑ͍ͷͰ md5Ͱ͋Δඞཁͳ͍
minio/HighwayHashʹมߋ (ຊΑ͘ͳ͍)
None
None
mount͞Ε͍ͯΔ σΟϨΫτϦ snapshotର֎
ϑΝΠϧ͕whitelistʹ ؚ·ΕΔఆ͢ΔՕॴͰ strings.SplitΛ༻
ϑΝΠϧ͕ଟ͔ͬͨΓ directory͕ਂ͍ͱ ແବʹϝϞϦΛ༻͢Δ
ಛੑΛߟ͑ͯ strings.SplitNΛ༻
None
None
https://github.com/ GoogleContainerTools/ kaniko/pull/694
129.54s -> 88.29s
None
1. ࣮ࡍʹ͛ͨύον 2. ͍ϥΠϒϥϦ͕ॻ͖͍ͨ
orisano/wyhash
kanikoͷύονΛ ॻ͍͍ͯΔͱ͖ʹ Կ͕ྑ͍hashͳͷͩΖ͏
Q. ͍hash?
Q. ͍hash? A. ܭଌ͠·͠ΐ͏
dgryski/trifles/hashbench खݩͰΒͤͯΈ·͠ΐ͏
(ݟ͔ͭΒͳ͍package͕ ͋ΔͷͰಈ͖·ͤΜ)
wyhash͕ GitHubͷTrendingͰ ্͕͖ͬͯͨ
ͯ͘ϙʔλϒϧͰڧ͍ Β͍͠
ඇৗʹ୯७ͳͷͰ GoʹҠ২ͯ͠ΈΑ͏ͱࢥͬͨ
2ҐͰҠ২͕ऴྃ
hashbenchʹՃ ֬ೝ͢ΔͱϘϩෛ͚͍ͯ͠Δ
͜Μͳܭࢉ͔͠ͳ͍ॲཧΛ Ͳ͏ͬͯߴԽ͢Δͷ͔
ྨࣅϥΠϒϥϦͷௐࠪ
҉߸ܥhashܥ جຊతʹasm͕ΘΕ͍ͯΔ
asmΛ͏ͱ͍?
ॻ͍ͯΈΑ͏
Go asmಠಛͳײ͡ ॻ͍͍ͯΔຊਓ͕ ΄ͱΜͲ͍ͳ͍? ͋·Γࢿྉ͕ͳ͍
ؤுͬͯ AVXΛͬͯॏ͍ॲཧΛॻ͘
lldbΛͬͯ bug(SEGV)Λमਖ਼͢Δ
Benchmark݁Ռ ͘ͳ͍ͬͯΔ
None
Կނ͔
asmͰॻ͍ͨؔ inlineԽ͞Εͳ͍
math/bits encoding/binary ίϯύΠϥ͕ݡ͘࠷దԽ͢Δ https://dave.cheney.net/ 2019/08/20/go-compiler- intrinsics
inlineԽ͞ΕΔΑ͏ͳ খ͞ͳؔ asmͷޮՌ͕ಘΒΕͳ͍
ࠓճͷΑ͏ͳ߹ͩͱ loop·ͰؚΊͯasmԽ͖͢
ෆ׳ΕͳasmͰ ଟ͘ͷίʔυΛॻ͖ͨ͘ͳ͍
mmcloughlin/avo Λ͓͏
None
GoͰasmΛੜ͢Δ ϓϩάϥϜΛॻ͘Ξϓϩʔν
Կ͕ྑ͍͔?
Go asmͷ͓࡞๏Λ avo͕ͬͯ͘ΕΔ
Go IDEͰͷิ͕ޮ͘
avoΛͬͯؤுͬͨ 5 GB/s -> 11 GB/s
ߴͳasmΛॻ͘ͷ͕͍͠
asmϨϕϧͰͳ͍ͥͷ͔ pprofͰΘ͔Βͳ͍
ύΠϓϥΠχϯάΛҙࣝ͢Δ 11 GB/s -> 14 GB/s
·ͱΊ
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ • 0. ෆຬͷϋʔυϧΛԼ͛Δ • 1. ܭଌͷϋʔυϧΛԼ͛Δ • 2. ՕॴΛಛఆ͢Δ
• 3. BenchmarkΛॻ͘
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ • 4. ղܾࡦΛߟ͑Δ • Ϛʔδ͞Ε͍͢ղܾࡦΛࢦ͢ • ϝϯςφϯείετΛ্͛ͳ͍ͷ • 5.
ࢼߦࡨޡ͢Δ • 6. ύονΛૹΔ
Ҿ͖ग़͠Λ૿͢ • ϝϞϦ༻ྔʹͳΓ͕ͪ • ֎෦͔ΒBuffer͕ड͚औΕΔAPIΛߟྀ͢Δ • มߋ͕༰қͰͳ͍߹sync.PoolΛߟྀ͢Δ
Ҿ͖ग़͠Λ૿͢ • దͳhashΛબ͢Δ • Ξηϯϒϥϝϯςφϯείετ͕ߴ͍ͷͰ ۃྗආ͚Δ • ॏ͍ॲཧͷ෦͚ͩσʔλͷ࣋ͪํΛม͑ͯ ΈΔ