Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Associatio...
Search
Takashi Kokubun
July 07, 2018
Programming
10
14k
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant
Rubyアソシエーション開発助成成果報告会
https://rubyassociation.doorkeeper.jp/events/74355
Takashi Kokubun
July 07, 2018
Tweet
Share
More Decks by Takashi Kokubun
See All by Takashi Kokubun
ZJIT: Building a New JIT Compiler for Ruby / REBASE 2025
k0kubun
0
18
Deoptimization: How YJIT Speeds Up Ruby by Slowing Down / RubyKaigi 2025
k0kubun
2
3.4k
YJIT Makes Rails 1.7x faster / RubyKaigi 2024
k0kubun
7
14k
Ruby JIT Hacking Guide / RubyKaigi 2023
k0kubun
2
10k
YJIT: Dive into Ruby's JIT compiler written in Rust / Rust.Tokyo 2022
k0kubun
2
2.2k
Towards Ruby 4 JIT / RubyKaigi 2022
k0kubun
3
11k
Optimizing Production Performance with MRI JIT / RubyConf 2021
k0kubun
1
480
Why Ruby's JIT was slow / RubyKaigi Takeout 2021
k0kubun
3
2k
数時間かかる週一リリースを毎日何度も爆速でできるようにするまで / CI/CD Conference 2021
k0kubun
21
14k
Other Decks in Programming
See All in Programming
Software Architecture
hschwentner
6
2.3k
CSC305 Lecture 08
javiergs
PRO
0
280
Go言語はstack overflowの夢を見るか?
logica0419
0
610
なんでRustの環境構築してないのにRust製のツールが動くの? / Why Do Rust-Based Tools Run Without a Rust Environment?
ssssota
14
46k
iOSでSVG画像を扱う
kishikawakatsumi
0
170
CSC305 Lecture 09
javiergs
PRO
0
310
AI Agent 時代的開發者生存指南
eddie
4
2.1k
コードとあなたと私の距離 / The Distance Between Code, You, and I
hiro_y
0
190
釣り地図SNSにおける有料機能の実装
nokonoko1203
0
200
SwiftDataを使って10万件のデータを読み書きする
akidon0000
0
240
Building, Deploying, and Monitoring Ruby Web Applications with Falcon (Kaigi on Rails 2025)
ioquatix
4
2.5k
CSC509 Lecture 08
javiergs
PRO
0
250
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
80
6k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
127
54k
Agile that works and the tools we love
rasmusluckow
331
21k
What’s in a name? Adding method to the madness
productmarketing
PRO
24
3.7k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Reflections from 52 weeks, 52 projects
jeffersonlam
353
21k
Building an army of robots
kneath
305
46k
It's Worth the Effort
3n
187
28k
Scaling GitHub
holman
463
140k
Build The Right Thing And Hit Your Dates
maggiecrowley
38
2.9k
Speed Design
sergeychernyshev
32
1.2k
4 Signs Your Business is Dying
shpigford
185
22k
Transcript
Ruby 3x3 ʹ͚ͨ RubyͷϕϯνϚʔΫڥͱՌ RubyΞιγΤʔγϣϯ։ൃॿ Ռใࠂձ @k0kubun
ࣗݾհ GitHub, Twitter: k0kubun Rubyίϛολ: JITίϯύΠϥͷ։ൃ, ERBͷϝϯςφϯε ࡞ͬͨ: Hamlit, MItamae,
md2key, xremap, LLRB
ࠓ͢͜ͱ 1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠ 2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά 3. ϓϥάΠϯΞʔΩςΫνϟ 4. ϓϩδΣΫτऴྃޙͷ׆ಈ
1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠
͜Ε·ͰͷRubyͷϕϯνϚʔΫڥ
ϒϩοΫݺͼग़͠ͷΦʔόʔϔου ඪ४ͷbenchmarkϥΠϒϥϦbenchmark-ipsɺܭଌ୯Ґ͕ϒ ϩοΫʹͳ͍ͬͯΔ ϒϩοΫݺͼग़͠ී௨ͷϝιουݺͼग़͠૬ͷΦʔόʔ ϔου͕͋Δ ͦΕΑΓ࠷దԽ͞Ε໋ͨྩͷܭଌ?
ෳͷRuby࣮ؒͰͷൺֱͷ͠͞ ϒϩοΫͰಈతʹܭଌํ๏͕ࢦఆՄೳͩͱɺͦͷϒϩοΫΛ ผͷRubyΠϯλϓϦλʹͯ͠ܭଌ͢Δͷࠔ ͔͠͠Ruby 3x3ͷͨΊʹɺॲཧܥࣗମͷมߋͷޮՌΛධՁ ͍ͨ͠
ཚཱ͢ΔϕϯνϚʔΧʔ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
ͰͲ͏͢Δ͔?
࠷͜ͷͷղܾʹ͍ۙπʔϧ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
benchmark/driver.rb ͱ ෳRubyόΠφϦؒͰൺֱ͕Ͱ͖ɺ࣮ߦ࣌ؒͷଞʹϝϞϦ༻ྔΛܭଌͰ ͖Δ ϒϩοΫͰͳ͘whileϧʔϓΛ͏ͨΊɺܭଌͷΦʔόʔϔου͕গͳ͍ ͨͩ͠RubyϦϙδτϦͷεΫϦϓτʹಛԽͨ͠ॲཧ͕ೖ͍ͬͯΔ benchmark/driver.rb Λ͍ͯ͘͢͠օͰ͍͍͑ͷͰ? ͦΕ͕ benchmark_driver.gem
࠾͞ΕͨϓϩδΣΫτ “ΑΓ౷ҰతͰ؆୯ͳRubyͷϕϯνϚʔΫڥͷ࣮ݱ" benchmark_driver.gem ͷػೳ֦ॆ Ruby 3x3 ʹ͚ͨϕϯνϚʔΫηοτͱCIͷඋ
benchmark_driver.gem ͷಛ
1. ͋ΒΏΔϝτϦΫεΛ͋ΒΏΔํ๏Ͱग़ྗͰ͖Δ ޙड़͢ΔϓϥάΠϯػߏʹΑΓɺ༷ʑͳܭଌํ๏Λ༷ʑͳํ ๏ͰूܭՄೳ σϑΥϧτͰbenchmark-ips෩ͷܭଌɾग़ྗ ಉ͡ग़ྗͷ··ϝϞϦ༻ྔΛܭଌͨ͠ΓɺOptcarrotͷfps Λܭଌͨ͠ΓͰ͖Δ
2. ਫ਼ͷྑ͍ܭଌεΫϦϓτΛࣗಈͰੜ whileϧʔϓΛ͢ΔεΫϦϓτΛಈతʹੜ͠ɺ͔ͭwhileϧʔϓ ʹ͔͔Δ࣌ؒܭଌͯࠩ͠͠Ҿ͘ ͦͷͨΊɺܭଌରϒϩοΫͰͳ͘ඞͣจࣈྻͰ༩͑ΒΕΔ Rubyͷىಈ࣌ؒܭଌͯ͠͠·͏ΦϦδφϧͷbenchmark/ driver.rbͱҧ͍ɺੜͨ͠εΫϦϓτͰ࣌ؒͷܭଌΛ͢Δ ىಈ࣌ؒͷϒϨͷӨڹΛड͚ʹ͍͘
3. RubyʹՃ͑ɺYAMLͰ؆͔ܿͭॊೈʹܭଌ͕ߦͳ͑Δ ಈతͳϒϩοΫͰͳ͘จࣈྻͰ༩͑ΔҎ্ɺΘ͟Θ͟RubyͰϕ ϯνϚʔΫΛఆٛͤ͞Δඞཁ͕ͳ͍ YAMLͷγϯλοΫεͰϕϯνϚʔΫΛఆٛ͢Δͱ͘ॻ͚Δ .travis.yml .circleci/config.yml ͘Β͍ͷؾܰ͞Ͱஔ͍ͯΒ͍͍ͨ OptcarrotʹطʹYAML͕ஔ͞Ε͍ͯΔ
4. ෳͷRuby࣮ؒɺgemͷผόʔδϣϯؒͰൺֱͰ͖Δ RubyόΠφϦΛىಈͯ͠ܭଌ͢ΔͨΊɺෳͷRuby࣮ؒͰ ൺֱͰ͖ɺRuby 3x3ʹ࠷ద ҰRubyΛىಈͨ͠Βͦ͜Ͱಉ͡gemͷෳόʔδϣϯΛڞ ଘͤ͞Δ͜ͱෆՄೳ͕ͩɺbenchmark_driver.gemͰͦͷൺ ֱ͕Ͱ͖Δ
࣮ࡍͷ༻ྫ
Haml 4 vs Haml 5
ERB Ruby 2.4 vs Ruby 2.5
Optcarrot
VS JRuby
VS TruffleRuby
Docker Time.utc
Docker Time.utc w/ $TZ
Ruby 2.6 block.call
benchmark_driver.gemͷ։ൃ Ҏ֎ͷՌ
ϕϯνϚʔΫηοτͷ༻ҙ FluentdϕϯνϚʔΫͷࣗಈԽɾbenchmark_driverରԠ benchmark-driver/fluentd-benchmark MJIT-benchmarksͷbenchmark_driverରԠ benchmark-driver/mjit-benchmarks Watson1978͞ΜͷϕϯνϚʔΫηοτͷม benchmark-driver/ruby-method-benchmarks
benchmark-driver.github.io
benchmark-driver.github.io GitHub pages ϕϯνϚʔΫαʔόʔ cronͰruby/ruby ΛϙʔϦϯά + ϕϯνϚʔΫ middleman build
git push Travis ϕϯνϚʔΫ݁Ռͷ YAMLΛอଘ͢Δ GitHubϦϙδτϦ Πϯλʔωοτ ݁ՌͷYAMLΛ git push webhook ݟΕΔ आΓͯΔαʔόʔͷ༷Ͱ ֎෦͔ΒΞΫηεͰ͖ͳ͍
benchmark-driver.github.io benchmark-driver/ruby-builder, skybench, sky2-infra… RubyBenchͩͱϓϩδΣΫτظؒதʹ͍͡Γ͢ͷ͕͍͠ ͱߟ͑ͨͨΊɺࢼݧతʹผϓϩδΣΫτͱͯ࢝͠ಈ ࣮ࡍʹϓϩδΣΫτऴྃޙΛ͔͚ͨΒ͙͢৭ʑͳݖݶ ΛΒ͑ͨ ৄࡉޙͰ͠·͢
2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά
มଟ͗͢ ܭଌରͷεΫϦϓτୡ ࣄલ࣮ߦεΫϦϓτͷ߹ͤ ܭଌ͢ΔϝτϦΫεɺ͋Δ͍ܭଌํ๏ Rubyͷόʔδϣϯ gemͷόʔδϣϯ
ෳϝτϦΫεͷಉ࣌ܭଌ ܭଌ͢ΔεΫϦϓτͷҰཡͱRubyͷόʔδϣϯҰཡ͚ͩͰදʹͳ͍ͬͯ ͯɺෳͷϝτϦΫεΛࠞͥΔͱਏ͍ Ͱuser, systemͷCPU࣌ؒͱ࣮࣌ؒΛಉ࣌ʹग़͢Έ͍ͨͳͷΓ͍ͨ (Ϟσϧ্ରԠ͚ͨ͠Ͳ࣮·ͩ͜Ε࣮ͯ͠ͳ͍) શͯͷϢʔεέʔεΛΧόʔͭͭ͠ഁ͍ͯ͠ͳ͍ϞσϧΛͲ͏ʹ͔ ࡞Εͳ͍͔?
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
ݕ౼ͨ͠ࣄ߲ HashͷΩʔඞͣΠϛϡʔλϒϧͳͷʹ͢Δ ͡Όͳ͍ͱOutputϓϥάΠϯ͕Hash͔ΒΛࢀর͢Δ࣌ʹࠔΔ Outputͷ͢͠͞ΑΓRunnerͷ࣮͕ഁ͠ͳ͍ͷΛ༏ઌͨ͠ߏ ಉ࣌ʹऔΒΕΔϝτϦΫε͕ͳΔ͍ۙ͘ॴʹདྷΔߏ Output࠷ѱ࠷ޙʹ·ͱΊͯͲ͏ʹ͔Ͱ͖ΔͨΊ
3. ϓϥάΠϯΞʔΩςΫνϟ
աڈͷϓϥάΠϯࣄྫ ϓϥάΠϯαϙʔτͷ͋Δͷ͜Ε·ͰͷϓϩδΣΫτ cookpad/barbeque itamae-kitchen/mitamae benchmark-driver/benchmark-driver
ڞ௨: ͔ࣗ͠ϓϥάΠϯॻ͍ͯͳ͍ ͔ࣗ͠ݾධՁͰbenchmark-driverͷΠϯλʔϑΣʔε͕Ұ൪ ͍ ͦͦ͋·ΓநԽʹ͍ͯͳ͍ཁ݅ͳؾ͍ͯ͠Δ ͳͷͰʹฉ͍͓͍͍ͯͯͩ͘͞
ϓϥάΠϯγεςϜͷ֓ཁ
2छྨͷϓϥάΠϯαϙʔτ RunnerϓϥάΠϯ: ܭଌํ๏ͱܭଌ͍ͨ͠ϝτϦΫεΛܾΊΔ OutputϓϥάΠϯ: ܭଌ݁ՌͷूܭΛߦͳ͏
RunnerϓϥάΠϯ Ruby API͔Β࣮ߦ͞ΕΔ͕ɺYAMLͷύʔε͜ͷRunnerϓ ϥάΠϯ͕ߦͳ͏ RunnerͷTypebenchmark-driverίϚϯυͰ༩͑ΒΕΔ͕ɺ YAMLͷதͰॻ͚ΔͷͰɺYAMLͰTypeΛݻఆͭͭ͠ಠ ࣗͷYAMLύʔαΛॻ͘ͱ͖์YAMLͷจ๏͕֦ுͰ͖ Δ
RunnerϓϥάΠϯ σϑΥϧτͷYAMLఆٛͷͷ: ips: ࣮ߦ࣌ؒ/࣮ߦճΛܭଌ͢Δ once: ςετ༻ʹ1ճ͚ͩΒͤΔ time: ࣮ߦ࣌ؒ memory: Max
Resident Set Size (ࠓLinuxͷΈαϙʔτ)
RunnerϓϥάΠϯ ಠࣗͷYAMLఆٛͷͷ: recorded: record outputΛ͏ͱੜ͞ΕΔYAMLͰɺಉ͡ܭ ଌ݁ՌΛԿग़ྗͰ͖Δ ruby_stdout: RubyίϚϯυͷstdoutΛύʔεͯ͠ϝτϦΫε ΛऔΔ
memory
memory
Optcarrot
Optcarrot
Runnerʹ͍ͭͯҙਤతʹαϙʔτΛΊͨػೳ Ruby͔Β࣮ߦ࣌ɺจࣈྻ͡Όͳͯ͘ϒϩοΫͰܭଌରΛ༩͑Δͷα ϙʔτ͍ͯͨ͠ Ҡߦύε + ෳόʔδϣϯ͕ࢦఆ͞Εͳ͚ΕevalͰΒͤͨํ͕͍ ࣮࠷ॳ͜ͷػೳ͕Runnerͱͯ͠ϓϥάΠϯαϙʔτ͞Ε͍ͯͨ(3छ ྨͷϓϥάΠϯ͕͋ͬͨ) υϥΠόͱಉ໊͡લۭؒͰΒͤΔ࣮ߦ͕ΫϦʔϯ͡Όͳ͘ɺಉ͡εΫ ϦϓτΛ࣮ߦ͢Δʹαϙʔτ͕ਏ͔ͬͨ͠ϝϦοτബ͍ͷͰΊͨ
v0.11~12ͷRunnerϓϥάΠϯͷॻ͖ํ YAMLΛநԽ͢ΔJobͱɺYAML͔ΒͦΕΛ࡞ΔJobParserΛఆٛ͢Δ #initializeʹɺCLIΦϓγϣϯΛ࣋ͭconfig, OutputΛநԽ͢Δ output, δϣϒͷϚτϦΫεΛ࡞Δcontexts͕༩͑ΒΕΔ #run ͖์࣮͍͍͕ͯ͠ɺoutputʹରͯ͠ϝιου܊Λܾ·ͬ ͨॱ൪ʹݺͳ͚ΕͳΒͳ͍ #with_warmup,
#with_benchmark, #with_job, #with_context, #report
OutputϓϥάΠϯ લड़ͨ͠ϞσϧΛड͚औͬͯूܭ͢Δׂ 1ͭܭଌ͢Δ͝ͱʹoutputʹͤΔΠϯλʔϑΣʔεͳͷͰɺ ݁Ռ͕ετϦʔϛϯάग़ྗͰ͖Δ શ෦ऴΘͬͨޙʹόϧΫͰग़ྗ͢ΔͨΊͷ؆୯ͳΠϯλʔ ϑΣʔε༻ҙ͍ͯ͠Δ
OutputϓϥάΠϯ ϏϧτΠϯͷϓϥάΠϯ compare: benchmark-ips෩ͷൺֱग़ྗ simple: benchmark/driver.rb ෩ग़ྗ markdown: GFMͷςʔϒϧग़ྗ record:
ܭଌ݁ՌΛอଘͯ͠ผͷoutputʹྲྀ͢༻
OutputϓϥάΠϯ ඇެࣜϓϥάΠϯ gruff: gruff.gemͰάϥϑ͕ग़ྗͰ͖Δ rubybench: RubyBenchʹ݁ՌΛૹ৴
Markdown
gruff
ϓϥάΠϯʹؔ͢Δҙࣄ߲ v0ͷؒԿϓϥάΠϯΠϯλʔϑΣʔε͕มΘ͍ͬͯͯɺv1ʹ͢Δ·Ͱ ֬ఆ͡Όͳ͍Ͱ͢ Runner Typeݻ༗ͷprelude͕άϩʔόϧͳcontexʹೖͬͯΔͷҾ͖ണ͕ ͢༧ఆ ϓϥάΠϯΛ࡞ͬͯͩ͘͞ΔํґଘΛ "benchmark_driver", "~> v0.X.Y"
ʹ ໘͍͚ͯͨͩ͠Δͱॿ͔Γ·͢ ಉ͡ v0.X ͷؒյ͞ͳ͍Α͏ʹ͍ͯ͠·͢
4. ϓϩδΣΫτऴྃޙͷ׆ಈ
RubyBench
RubyBenchͱ RubyͷύϑΥʔϚϯεΛࢹ͢ΔCI benchmark-driver.github.io ΑΓେલ͔Β͋Δ
RubyBenchͷΞʔΩςΫνϟ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
͜͜Λஔ͖͍͑ͨ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
RubyBenchͷ౷߹ JITͱͷൺֱάϥϑޙ͔ΒRubyBenchʹ࣮͞Εͨ όάͬͯͨͷͰͨ͠Γͨ͠ benchmark_driver͚ͷϕϯνϚʔΫηοτਫ਼ͷྑ͍ܭଌػೳ͕ࠓޙ׆༻͠ ͍͢Α͏ɺϕϯνϚʔΧʔͷbenchmark_driverͷஔ͖͑Λ͍ͬͯΔ طʹOptcarrotbenchmark_driverͰܭଌ͞Ε͍ͯΔ ॏෳ͕ͪ͠ͳ࣮͕ϓϥάΠϯͰநԽͰ͖͓ͯΓbenchmark_driverͷϓϥά Πϯػߏ͕׆༂͍ͯ͠Δ
RubyBench͚ʹɺϕϯνϚʔΫͷ֦ॆ͍͖ͬͯ·͢ VladͷMJIT͚ϕϯνϚʔΫूΛՃ༧ఆ Watson1978͞Μͷ֤ϝιου͚ϕϯνϚʔΫूΛมͨ͠ ͷΘ͍͔ͤͯͨͩ͘͞? https://github.com/benchmark-driver/ruby-method-benchmarks
·ͱΊ benchmark_driver ศརͰ͢ͷͰɺੋඇ͝ར༻͍ͩ͘͞ RubyͷߴԽRubyBenchʹطʹຊՌ͕׆༻͞Ε͍ͯ·͢
ँࣙ ຊϓϩδΣΫτɺాߞҰ༷Λ࢝Ίͱͨ͠RubyΞιγΤʔ γϣϯ༷ͷ͝ڠྗʹΑ࣮ͬͯݱ͢Δ͜ͱ͕Ͱ͖·ͨ͠ɻ͜ͷ ΛआΓͯޚྱਃ্͛͠·͢ɻ