Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
OSS Performance Tuning Tips
Search
orisano
October 28, 2019
Programming
8
5.8k
OSS Performance Tuning Tips
orisano
October 28, 2019
Tweet
Share
More Decks by orisano
See All by orisano
Docker-Compose & BuildKit
orisano
4
1k
Container Build Talk
orisano
3
2.5k
dockerignore talk
orisano
2
7.1k
Better docker image+
orisano
6
6.1k
Socket.IO Introduction
orisano
0
3.2k
Profiling Go Application
orisano
11
7.8k
Multi-stage Builds Patterns & Practice
orisano
6
5.1k
better docker image
orisano
22
30k
the world of characters
orisano
8
1.4k
Other Decks in Programming
See All in Programming
Deep Dive into ~/.claude/projects
hiragram
8
1.5k
Bytecode Manipulation 으로 생산성 높이기
bigstark
2
380
第9回 情シス転職ミートアップ 株式会社IVRy(アイブリー)の紹介
ivry_presentationmaterials
1
240
A2A プロトコルを試してみる
azukiazusa1
2
1.1k
AIエージェントはこう育てる - GitHub Copilot Agentとチームの共進化サイクル
koboriakira
0
380
Java on Azure で LangGraph!
kohei3110
0
170
なんとなくわかった気になるブロックテーマ入門/contents.nagoya 2025 6.28
chiilog
1
210
「Cursor/Devin全社導入の理想と現実」のその後
saitoryc
0
150
5つのアンチパターンから学ぶLT設計
narihara
1
110
PHPでWebSocketサーバーを実装しよう2025
kubotak
0
160
Result型で“失敗”を型にするPHPコードの書き方
kajitack
4
380
データの民主化を支える、透明性のあるデータ利活用への挑戦 2025-06-25 Database Engineering Meetup#7
y_ken
0
320
Featured
See All Featured
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.8k
Why You Should Never Use an ORM
jnunemaker
PRO
57
9.4k
Stop Working from a Prison Cell
hatefulcrawdad
270
20k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
The Language of Interfaces
destraynor
158
25k
Embracing the Ebb and Flow
colly
86
4.7k
Why Our Code Smells
bkeepers
PRO
337
57k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Building Applications with DynamoDB
mza
95
6.5k
Docker and Python
trallard
44
3.4k
Making Projects Easy
brettharned
116
6.3k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Transcript
OSS Performance Tuning Tips #gocon #gocon_hall GoCon 2019 Autumn @orisano
త ̎ͭͷ͜ͱΛڞ༗͍ͨ͠
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ Ҿ͖ग़͠Λ૿͢
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ Ҿ͖ग़͠Λ૿͢
Ͳ͏ਐΊ͍ͯΔͷ͔
0. ෆຬͷϋʔυϧΛԼ͛Δ
ෆຬ͕ վળͷ͖͔͚ͬʹͳΓ·͢
ීஈ͍ͬͯΔ ϥΠϒϥϦɺιϑτΣΞ͕ ͍ͷͩͱࢥͬͯΈΔ
͖͔͚ͬͱ ϞνϕʔγϣϯΛ ࣗΒ࡞Γग़͠·͠ΐ͏
1. ܭଌͷϋʔυϧΛԼ͛Δ
͍ݪҼΛ ؾܰʹ ௐΒΕΔೳྗΛʹ͚ͭΔ
ख͕͔͔ؒΔߦҝΛ ਓؒ͋·Γ͠ͳ͍
ͨ͘͞Μܦݧ͢Δ
ؾܰʹ ௐࠪͰ͖ΔΑ͏ͳΈΛ ೖΕΔ
͜ΕOSSΛެ։͢Δଆͷ ڥมͰࢦఆͨ͠Γ σϑΥϧτͰprofile͕औΕΔ ΈΛఏڙ͢Δͱ վળ͍͢͠/͞Ε͍͢
ʹૺ۰ͨ͠ਓ͕ ݪҼௐ͍ࠪ͢͠ͱخ͍͠
2. ՕॴΛಛఆ͢Δ
͍ͱײͯ͡ Օॴ͕ؔϨϕϧͰ Θ͔Βͳ͍ͷ cliΛ࣮ߦ͍ͯ͠Δͱ͖
mainʹ github.com/pkg/profile ΛՃͯ͠ΈΔ
None
͜Ε͚ͩ
cliͷ࣮ߦ͕1ʹ 1࣌ؒҎ্͔͔Δ߹ͳͲ net/http/pprofͷ΄͏͕ ྑ͍߹
profileΛݟͯ ؔϨϕϧͰಛఆ͢Δ
pprofͷweb൛ʹ׳ΕΔ ͪΌΜͱҙຯ͕Θ͔Δ
͍ΖΜͳݟํͰݟͯΈΔ top, graph, flame-graph source, disasm
3. BenchmarkΛॻ͘
͢Ͱʹॻ͍ͯ͋Δ߹ ϥοΩʔ ͘ͳ͍ͬͯͳ͍߹ ͘ͳΔέʔεΛՃ͢Δ
BenchmarkΛॻ࣌͘ ࠷దԽͰফ͍͑ͯͳ͍͔֬ೝ
BenchmarkΛॻ͘ͱ ؆୯ʹprofile͕औΕΔ
go test -cpuprofile go test -memprofile Λ֮͑Α͏ (go help testflagΛಡ͏)
4. ղܾࡦΛߟ͑Δ
Ͳ͏ղܾ͢Δ͔ʹ͍ͭͯ “Ҿ͖ग़͠Λ૿͢”Ͱ
OSSͷ ύϑΥʔϚϯενϡʔχϯά Ϛʔδ͞Εͯྃ
Ͳ͏ΕϚʔδ͞ΕΔ ύονʹͳΔ͔
Ϛʔδ͢Δͱ͍͏͜ͱ ૬ख͕ϝϯςφϯε͢Δ ͱ͍͏͜ͱ
มߋ͕গͳ͍ ૬खͷίετΛ૿͞ͳ͍
ഁյతมߋ͕ͳ͍ ͘ΘΕ͍ͯΔͷ΄Ͳ ॏཁͳͱ͜Ζ
ҟৗͳίετ͕ͳ͍ มߋ, อक͕͘͠ͳΔ࣮ (ΞηϯϒϥΛͬͨΓ)
ґଘؔΛՃ͢Δ߹ ৻ॏʹબ͢Δ ΞΫςΟϒʹϝϯςφϯε͞ Ε͍ͯΔͷ͔ ຊʹඞཁ͔?
͘͢͝ύϑΥʔϚϯε͕ վળ͢Δ ܶతͳվળड͚ೖΕΒΕΔ
ςετ͕ॻ͍ͯ͋Δ ͢Ͱʹॻ͍ͯ͋Δ߹डཧ ͞Ε͍͢
5. ࢼߦࡨޡ͢Δ
·ͣॳظঢ়ଶͰ ेͳճ timeoutʹͳΒͳ͍Α͏ʹ Benchmark݁ՌΛऔ͓ͬͯ͘
go test -bench . -count=10 -timeout=30000s | tee old.txt
ύονΛૹΔͱ͖ʹ benchstatͷ݁ՌΛૹΔͷͰ ϕʔεϥΠϯेͳճͰ
timeout σϑΥϧτͩͱ10mͰ ҙ֎ͱΦʔόʔ͢Δ
ղܾࡦΛࢥ͍͍ͭͨΒ ·ͣ1Benchmark
ے͕ྑͦ͞͏ͳΒ ेͳճBenchmark
ଟ͘ࢼߦࡨޡ͍ͯ͠Δͱ profileͱsourceͷ͕ؔ Θ͔Βͳ͘ͳΔ
pprofʹ sourceΛݟΔػೳ͕͋Δ͕ อ͍࣋ͯ͠ΔΘ͚Ͱͳ͍ pathͷΈ
git add . git commit -m “$2” REV=$(git rev-parse HEAD)
go test -bench $1 -benchmem -cpuprofile cpu.${REV}.pb.gz -memprofile mem.$ {REV}.pb.gz | tee ${REV}.txt rm ./${REV}.* && git reset HEAD^
sourcegitʹཧͯ͠Β͏ ͪΌΜͱඥ͚ΒΕΔ
ଟ͘ͷprofile͕͋Δͱ ୯ମͷޮՌ͕Θ͔Βͳ͍ ͜ͱ͕͋Δ
go tool pprof -diff_base Ͱprofileؒͷ͕ࠩݟΕΔ
6. ύονΛૹΔ
࠷ऴͷBenchmarkΛ ेͳճߦ͏
ઐ༻ͷΠϯελϯεͰܭଌ ͍͠߹ ۃྗ֎෦ϓϩηεΛఀࢭ͢Δ
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ Ҿ͖ग़͠Λ૿͢
࣮ྫϕʔεͰ ͲͷΑ͏ʹղܾ͔ͨ͠ ࣗͳΒͲ͏ղܾ͢Δ͔ ߟ͑ͯΒ͍͍ͨ
1. ࣮ࡍʹ͛ͨύον 2. ͍ϥΠϒϥϦ͕ॻ͖͍ͨ
1. ࣮ࡍʹ͛ͨύον 2. ͍ϥΠϒϥϦ͕ॻ͖͍ͨ
src-d/go-git
pure-goͰ gitͷૢ࡞Λ͢ΔͨΊͷ ϥΠϒϥϦ
ύονΛૹͬͨഎܠ
aquasecurity/trivy Ͱॳ͍͕ͬͯͨ େ͖ͳrepositoryͷcloneʹ ҟৗͳ࣌ؒ(10min~)͕͔͔Δ ͕͋ͬͨ
ݪҼ gitͷIndexͷߏஙΛߦ͏Օॴ ͩͬͨ
Indexͷ෦දݱ(public)͕ sliceʹͳ͓ͬͯΓ nameͰҰҙʹ͢ΔͨΊʹ deleteͱappend͕ ϑΝΠϧճ࣮ߦ͞Ε͍ͯͨ
deletenameΛࢦఆͯ͠ আ͢ΔͷͰ sliceͷཁૉΛͯࠪ͢͢Δ
ॳ ෦දݱΛmapʹมߋ͢Ε deleteͷܭࢉྔΛݮΒͤΔ ͷͰͦ͏͠Α͏ͱࢥͬͨ
͔͠͠ publicͳϑΟʔϧυ internalͰͳ͍ύοέʔδ 4500 star͑ͷ໊
౸ఈड͚ೖΕΒΕͳ͍ มߋͩͱࢥ͍ఘΊͨ
ޙ
ιʔείʔυΛ ݟ͍ͯ͠Δͱ
໌ه͞Ε͍ͯΔ ϑΟʔϧυͷ༷ (sliceͷॱংอূ͠ͳ͍) େྔͷIndexΛߏங͢Δ λΠϛϯά(clone࣌)
ݟͨ݁͠Ռ ഁյతมߋແ͠Ͱ मਖ਼͢Δํ๏Λࢥ͍͍ͭͨ
ଟ͘ݺͼग़͞ΕΔՕॴ ͷΈʹ͓͍ͯ mapͰอ࣋͠return͢Δࡍʹ sliceʹม͢Δ
mapΛprivateͳؔͰ Ҿ͖ճ͢Α͏ʹ͢Δ͜ͱͰ ഁյతมߋΛճආͰ͖ͨ
None
ͦ͜Λղܾ͢Δͱ mallocgc͕ॏ͘ͳͬͨ
ϝϞϦ༻ྔͷݪҼ io.Copyʹ͋ΔΑ͏ͩͬͨ
ϑΝΠϧ͝ͱʹio.Copy͕ ݺΕ͍ͯͨ
େྔʹݺΕΔՄೳੑ͕͋Δ ՕॴͰio.CopyͰͳ͘ io.CopyBufferΛ͏ͱ ϝϞϦ༻ྔΛ੍ޚͰ͖Δ
͏bufferΛͲ͏ࢦఆ͢Δ͔ ֎෦͔Β༩͑ΒΕͳ͍ͱ ݁ہϝϞϦ༻ྔมΘΒͳ͍
privateͳϝιουͱ͍͑ ҾʹՃ͢Δͱ มߋൣғ͕େ͖͘ͳΔ
ղܾࡦͱͯ͠ globalʹsync.PoolΛ ஔ͘͜ͱʹͨ͠
None
https://github.com/src-d/ go-git/pull/1179
None
605ඵ -> 249ඵ
͔͠͠ ·ͩ·ͩϝϞϦ༻ྔ͕ଟ͍ 56 GB/op
ϑΝΠϧʹࠩΛద༻͢Δ ॲཧ͕ϝϞϦΛେྔʹ༻
ؔ෦Ͱ publicͷؔͰ buffer͕֎෦͔Βͤͳ͍ ͷ͕ΘΕ͍ͯͨ
গͳ͘ͱ෦͔Βͷ༻ ͰbufferΛࢦఆ͍ͨ͠
None
෦͚ʹbufferࢦఆͰ͖Δ Α͏ʹͯ͠ղܾ
https://github.com/src-d/ go-git/pull/1180
56.1 GB -> 29.8 GB
None
None
image/png
ύονΛૹͬͨഎܠ
ࣾISUCONͰ QRίʔυΛߴʹੜ͢Δ ඞཁ͕͋ͬͨͨΊ
ࣾISUCONͷৼΓฦΓͰ ύϑΥʔϚϯενϡʔχϯά ΛҰਓͰָ͠ΜͰ͍ͨ
ऄ
ߴʹେྔͷpngΛ ग़ྗ͢ΔͨΊʹ
1.9͔Βೖͬͨ png.EncoderBufferPool
CompressionLevelͷઃఆ
ಠࣗimage.Image࣮Λ Θͳ͍
നࠇը૾Ͱ͋Ε image.GrayΛ͏ (Opaque͕bypassͰ͖Δ)
ऄऴྃ
image/png͕ bottleneckʹͳ͖ͬͯͨ
͢ͰʹBenchmark͕ ॻ͔Ε͍ͯͨͷͰ࣮ߦ
ࠓճ͍ͬͯΔ ՕॴͰͳ͍͕࿐ࠎʹ͍ ෦͕͋ͬͨ
https://go- review.googlesource.com/ c/go/+/187417
None
࣮ࡍॏ͔ͬͨՕॴ compress/deflate
pprof͕ॏ͍ͱࣔͨ͠ͷ for͕ॻ͔Ε͍ͯΔߦͩͬͨ
None
֘forจͷasmΛݟͯΈΔͱ ແବͳϝϞϦΞΫηε͕
compilerʹregisterΛ ͬͯΒ͏ͨΊʹ ϩʔΧϧมΛఆٛ
None
https://go- review.googlesource.com/ c/go/+/187837
None
None
GoogleContainerTools/ kaniko
ύονΛૹͬͨഎܠ
ΞΠσΟΞ͕εΩͩͬͨ
kanikoΛCIͰͬͯ Կʹ͕͔͔͍࣌ؒͬͯΔͷ͔ ؾʹͳͬͨ
kanikoϝϞϦ্ʹ filesystemͷsnapshotΛ࣋ͭ
ίϚϯυΛ࣮ߦ͢Δͨͼʹ ͕ࠩͳ͍͔ൺֱ͢Δ
md5Ͱൺֱ͢Δ
ͦͷmd5͕ॏ͔ͬͨ
ϑΝΠϧ͕ ಉҰ͔Ͳ͏͔͚ͩͰྑ͍ͷͰ md5Ͱ͋Δඞཁͳ͍
minio/HighwayHashʹมߋ (ຊΑ͘ͳ͍)
None
None
mount͞Ε͍ͯΔ σΟϨΫτϦ snapshotର֎
ϑΝΠϧ͕whitelistʹ ؚ·ΕΔఆ͢ΔՕॴͰ strings.SplitΛ༻
ϑΝΠϧ͕ଟ͔ͬͨΓ directory͕ਂ͍ͱ ແବʹϝϞϦΛ༻͢Δ
ಛੑΛߟ͑ͯ strings.SplitNΛ༻
None
None
https://github.com/ GoogleContainerTools/ kaniko/pull/694
129.54s -> 88.29s
None
1. ࣮ࡍʹ͛ͨύον 2. ͍ϥΠϒϥϦ͕ॻ͖͍ͨ
orisano/wyhash
kanikoͷύονΛ ॻ͍͍ͯΔͱ͖ʹ Կ͕ྑ͍hashͳͷͩΖ͏
Q. ͍hash?
Q. ͍hash? A. ܭଌ͠·͠ΐ͏
dgryski/trifles/hashbench खݩͰΒͤͯΈ·͠ΐ͏
(ݟ͔ͭΒͳ͍package͕ ͋ΔͷͰಈ͖·ͤΜ)
wyhash͕ GitHubͷTrendingͰ ্͕͖ͬͯͨ
ͯ͘ϙʔλϒϧͰڧ͍ Β͍͠
ඇৗʹ୯७ͳͷͰ GoʹҠ২ͯ͠ΈΑ͏ͱࢥͬͨ
2ҐͰҠ২͕ऴྃ
hashbenchʹՃ ֬ೝ͢ΔͱϘϩෛ͚͍ͯ͠Δ
͜Μͳܭࢉ͔͠ͳ͍ॲཧΛ Ͳ͏ͬͯߴԽ͢Δͷ͔
ྨࣅϥΠϒϥϦͷௐࠪ
҉߸ܥhashܥ جຊతʹasm͕ΘΕ͍ͯΔ
asmΛ͏ͱ͍?
ॻ͍ͯΈΑ͏
Go asmಠಛͳײ͡ ॻ͍͍ͯΔຊਓ͕ ΄ͱΜͲ͍ͳ͍? ͋·Γࢿྉ͕ͳ͍
ؤுͬͯ AVXΛͬͯॏ͍ॲཧΛॻ͘
lldbΛͬͯ bug(SEGV)Λमਖ਼͢Δ
Benchmark݁Ռ ͘ͳ͍ͬͯΔ
None
Կނ͔
asmͰॻ͍ͨؔ inlineԽ͞Εͳ͍
math/bits encoding/binary ίϯύΠϥ͕ݡ͘࠷దԽ͢Δ https://dave.cheney.net/ 2019/08/20/go-compiler- intrinsics
inlineԽ͞ΕΔΑ͏ͳ খ͞ͳؔ asmͷޮՌ͕ಘΒΕͳ͍
ࠓճͷΑ͏ͳ߹ͩͱ loop·ͰؚΊͯasmԽ͖͢
ෆ׳ΕͳasmͰ ଟ͘ͷίʔυΛॻ͖ͨ͘ͳ͍
mmcloughlin/avo Λ͓͏
None
GoͰasmΛੜ͢Δ ϓϩάϥϜΛॻ͘Ξϓϩʔν
Կ͕ྑ͍͔?
Go asmͷ͓࡞๏Λ avo͕ͬͯ͘ΕΔ
Go IDEͰͷิ͕ޮ͘
avoΛͬͯؤுͬͨ 5 GB/s -> 11 GB/s
ߴͳasmΛॻ͘ͷ͕͍͠
asmϨϕϧͰͳ͍ͥͷ͔ pprofͰΘ͔Βͳ͍
ύΠϓϥΠχϯάΛҙࣝ͢Δ 11 GB/s -> 14 GB/s
·ͱΊ
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ • 0. ෆຬͷϋʔυϧΛԼ͛Δ • 1. ܭଌͷϋʔυϧΛԼ͛Δ • 2. ՕॴΛಛఆ͢Δ
• 3. BenchmarkΛॻ͘
νϡʔχϯάΛͲ͏ਐΊΔͷ͔ • 4. ղܾࡦΛߟ͑Δ • Ϛʔδ͞Ε͍͢ղܾࡦΛࢦ͢ • ϝϯςφϯείετΛ্͛ͳ͍ͷ • 5.
ࢼߦࡨޡ͢Δ • 6. ύονΛૹΔ
Ҿ͖ग़͠Λ૿͢ • ϝϞϦ༻ྔʹͳΓ͕ͪ • ֎෦͔ΒBuffer͕ड͚औΕΔAPIΛߟྀ͢Δ • มߋ͕༰қͰͳ͍߹sync.PoolΛߟྀ͢Δ
Ҿ͖ग़͠Λ૿͢ • దͳhashΛબ͢Δ • Ξηϯϒϥϝϯςφϯείετ͕ߴ͍ͷͰ ۃྗආ͚Δ • ॏ͍ॲཧͷ෦͚ͩσʔλͷ࣋ͪํΛม͑ͯ ΈΔ