Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Associatio...
Search
Takashi Kokubun
July 07, 2018
Programming
10
14k
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant
Rubyアソシエーション開発助成成果報告会
https://rubyassociation.doorkeeper.jp/events/74355
Takashi Kokubun
July 07, 2018
Tweet
Share
More Decks by Takashi Kokubun
See All by Takashi Kokubun
Deoptimization: How YJIT Speeds Up Ruby by Slowing Down / RubyKaigi 2025
k0kubun
2
2.9k
YJIT Makes Rails 1.7x faster / RubyKaigi 2024
k0kubun
7
14k
Ruby JIT Hacking Guide / RubyKaigi 2023
k0kubun
2
10k
YJIT: Dive into Ruby's JIT compiler written in Rust / Rust.Tokyo 2022
k0kubun
1
2.1k
Towards Ruby 4 JIT / RubyKaigi 2022
k0kubun
3
11k
Optimizing Production Performance with MRI JIT / RubyConf 2021
k0kubun
1
460
Why Ruby's JIT was slow / RubyKaigi Takeout 2021
k0kubun
3
1.9k
数時間かかる週一リリースを毎日何度も爆速でできるようにするまで / CI/CD Conference 2021
k0kubun
21
14k
Ruby 3 JIT's roadmap / RubyConf China 2020
k0kubun
0
820
Other Decks in Programming
See All in Programming
Go Modules: From Basics to Beyond / Go Modulesの基本とその先へ
kuro_kurorrr
0
100
機械学習って何? 5分で解説頑張ってみる
kuroneko2828
0
190
The Evolution of Enterprise Java with Jakarta EE 11 and Beyond
ivargrimstad
1
500
FormFlow - Build Stunning Multistep Forms
yceruto
1
140
List Unfolding - 'unfold' as the Computational Dual of 'fold', and how 'unfold' relates to 'iterate'"
philipschwarz
PRO
0
180
REST API設計の実践 – ベストプラクティスとその落とし穴
kentaroutakeda
2
360
Zennの運営完全に理解した #完全に理解したTalk
wadayusuke
1
180
try-catchを使わないエラーハンドリング!? PHPでResult型の考え方を取り入れてみよう
kajitack
3
470
Rails産でないDBを Railsに引っ越すHACK - Omotesando.rb #110
lnit
1
160
FastMCPでMCPサーバー/クライアントを構築してみる
ttnyt8701
2
130
OpenNext + Hono on Cloudflare でイマドキWeb開発スタックを実現する
rokuosan
0
120
ワンバイナリWebサービスのススメ
mackee
10
7.7k
Featured
See All Featured
The Language of Interfaces
destraynor
158
25k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Statistics for Hackers
jakevdp
799
220k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.6k
Designing Experiences People Love
moore
142
24k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
14
1.5k
Optimizing for Happiness
mojombo
379
70k
Fireside Chat
paigeccino
37
3.5k
How to Ace a Technical Interview
jacobian
276
23k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Embracing the Ebb and Flow
colly
85
4.7k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Transcript
Ruby 3x3 ʹ͚ͨ RubyͷϕϯνϚʔΫڥͱՌ RubyΞιγΤʔγϣϯ։ൃॿ Ռใࠂձ @k0kubun
ࣗݾհ GitHub, Twitter: k0kubun Rubyίϛολ: JITίϯύΠϥͷ։ൃ, ERBͷϝϯςφϯε ࡞ͬͨ: Hamlit, MItamae,
md2key, xremap, LLRB
ࠓ͢͜ͱ 1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠ 2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά 3. ϓϥάΠϯΞʔΩςΫνϟ 4. ϓϩδΣΫτऴྃޙͷ׆ಈ
1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠
͜Ε·ͰͷRubyͷϕϯνϚʔΫڥ
ϒϩοΫݺͼग़͠ͷΦʔόʔϔου ඪ४ͷbenchmarkϥΠϒϥϦbenchmark-ipsɺܭଌ୯Ґ͕ϒ ϩοΫʹͳ͍ͬͯΔ ϒϩοΫݺͼग़͠ී௨ͷϝιουݺͼग़͠૬ͷΦʔόʔ ϔου͕͋Δ ͦΕΑΓ࠷దԽ͞Ε໋ͨྩͷܭଌ?
ෳͷRuby࣮ؒͰͷൺֱͷ͠͞ ϒϩοΫͰಈతʹܭଌํ๏͕ࢦఆՄೳͩͱɺͦͷϒϩοΫΛ ผͷRubyΠϯλϓϦλʹͯ͠ܭଌ͢Δͷࠔ ͔͠͠Ruby 3x3ͷͨΊʹɺॲཧܥࣗମͷมߋͷޮՌΛධՁ ͍ͨ͠
ཚཱ͢ΔϕϯνϚʔΧʔ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
ͰͲ͏͢Δ͔?
࠷͜ͷͷղܾʹ͍ۙπʔϧ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
benchmark/driver.rb ͱ ෳRubyόΠφϦؒͰൺֱ͕Ͱ͖ɺ࣮ߦ࣌ؒͷଞʹϝϞϦ༻ྔΛܭଌͰ ͖Δ ϒϩοΫͰͳ͘whileϧʔϓΛ͏ͨΊɺܭଌͷΦʔόʔϔου͕গͳ͍ ͨͩ͠RubyϦϙδτϦͷεΫϦϓτʹಛԽͨ͠ॲཧ͕ೖ͍ͬͯΔ benchmark/driver.rb Λ͍ͯ͘͢͠օͰ͍͍͑ͷͰ? ͦΕ͕ benchmark_driver.gem
࠾͞ΕͨϓϩδΣΫτ “ΑΓ౷ҰతͰ؆୯ͳRubyͷϕϯνϚʔΫڥͷ࣮ݱ" benchmark_driver.gem ͷػೳ֦ॆ Ruby 3x3 ʹ͚ͨϕϯνϚʔΫηοτͱCIͷඋ
benchmark_driver.gem ͷಛ
1. ͋ΒΏΔϝτϦΫεΛ͋ΒΏΔํ๏Ͱग़ྗͰ͖Δ ޙड़͢ΔϓϥάΠϯػߏʹΑΓɺ༷ʑͳܭଌํ๏Λ༷ʑͳํ ๏ͰूܭՄೳ σϑΥϧτͰbenchmark-ips෩ͷܭଌɾग़ྗ ಉ͡ग़ྗͷ··ϝϞϦ༻ྔΛܭଌͨ͠ΓɺOptcarrotͷfps Λܭଌͨ͠ΓͰ͖Δ
2. ਫ਼ͷྑ͍ܭଌεΫϦϓτΛࣗಈͰੜ whileϧʔϓΛ͢ΔεΫϦϓτΛಈతʹੜ͠ɺ͔ͭwhileϧʔϓ ʹ͔͔Δ࣌ؒܭଌͯࠩ͠͠Ҿ͘ ͦͷͨΊɺܭଌରϒϩοΫͰͳ͘ඞͣจࣈྻͰ༩͑ΒΕΔ Rubyͷىಈ࣌ؒܭଌͯ͠͠·͏ΦϦδφϧͷbenchmark/ driver.rbͱҧ͍ɺੜͨ͠εΫϦϓτͰ࣌ؒͷܭଌΛ͢Δ ىಈ࣌ؒͷϒϨͷӨڹΛड͚ʹ͍͘
3. RubyʹՃ͑ɺYAMLͰ؆͔ܿͭॊೈʹܭଌ͕ߦͳ͑Δ ಈతͳϒϩοΫͰͳ͘จࣈྻͰ༩͑ΔҎ্ɺΘ͟Θ͟RubyͰϕ ϯνϚʔΫΛఆٛͤ͞Δඞཁ͕ͳ͍ YAMLͷγϯλοΫεͰϕϯνϚʔΫΛఆٛ͢Δͱ͘ॻ͚Δ .travis.yml .circleci/config.yml ͘Β͍ͷؾܰ͞Ͱஔ͍ͯΒ͍͍ͨ OptcarrotʹطʹYAML͕ஔ͞Ε͍ͯΔ
4. ෳͷRuby࣮ؒɺgemͷผόʔδϣϯؒͰൺֱͰ͖Δ RubyόΠφϦΛىಈͯ͠ܭଌ͢ΔͨΊɺෳͷRuby࣮ؒͰ ൺֱͰ͖ɺRuby 3x3ʹ࠷ద ҰRubyΛىಈͨ͠Βͦ͜Ͱಉ͡gemͷෳόʔδϣϯΛڞ ଘͤ͞Δ͜ͱෆՄೳ͕ͩɺbenchmark_driver.gemͰͦͷൺ ֱ͕Ͱ͖Δ
࣮ࡍͷ༻ྫ
Haml 4 vs Haml 5
ERB Ruby 2.4 vs Ruby 2.5
Optcarrot
VS JRuby
VS TruffleRuby
Docker Time.utc
Docker Time.utc w/ $TZ
Ruby 2.6 block.call
benchmark_driver.gemͷ։ൃ Ҏ֎ͷՌ
ϕϯνϚʔΫηοτͷ༻ҙ FluentdϕϯνϚʔΫͷࣗಈԽɾbenchmark_driverରԠ benchmark-driver/fluentd-benchmark MJIT-benchmarksͷbenchmark_driverରԠ benchmark-driver/mjit-benchmarks Watson1978͞ΜͷϕϯνϚʔΫηοτͷม benchmark-driver/ruby-method-benchmarks
benchmark-driver.github.io
benchmark-driver.github.io GitHub pages ϕϯνϚʔΫαʔόʔ cronͰruby/ruby ΛϙʔϦϯά + ϕϯνϚʔΫ middleman build
git push Travis ϕϯνϚʔΫ݁Ռͷ YAMLΛอଘ͢Δ GitHubϦϙδτϦ Πϯλʔωοτ ݁ՌͷYAMLΛ git push webhook ݟΕΔ आΓͯΔαʔόʔͷ༷Ͱ ֎෦͔ΒΞΫηεͰ͖ͳ͍
benchmark-driver.github.io benchmark-driver/ruby-builder, skybench, sky2-infra… RubyBenchͩͱϓϩδΣΫτظؒதʹ͍͡Γ͢ͷ͕͍͠ ͱߟ͑ͨͨΊɺࢼݧతʹผϓϩδΣΫτͱͯ࢝͠ಈ ࣮ࡍʹϓϩδΣΫτऴྃޙΛ͔͚ͨΒ͙͢৭ʑͳݖݶ ΛΒ͑ͨ ৄࡉޙͰ͠·͢
2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά
มଟ͗͢ ܭଌରͷεΫϦϓτୡ ࣄલ࣮ߦεΫϦϓτͷ߹ͤ ܭଌ͢ΔϝτϦΫεɺ͋Δ͍ܭଌํ๏ Rubyͷόʔδϣϯ gemͷόʔδϣϯ
ෳϝτϦΫεͷಉ࣌ܭଌ ܭଌ͢ΔεΫϦϓτͷҰཡͱRubyͷόʔδϣϯҰཡ͚ͩͰදʹͳ͍ͬͯ ͯɺෳͷϝτϦΫεΛࠞͥΔͱਏ͍ Ͱuser, systemͷCPU࣌ؒͱ࣮࣌ؒΛಉ࣌ʹग़͢Έ͍ͨͳͷΓ͍ͨ (Ϟσϧ্ରԠ͚ͨ͠Ͳ࣮·ͩ͜Ε࣮ͯ͠ͳ͍) શͯͷϢʔεέʔεΛΧόʔͭͭ͠ഁ͍ͯ͠ͳ͍ϞσϧΛͲ͏ʹ͔ ࡞Εͳ͍͔?
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
ݕ౼ͨ͠ࣄ߲ HashͷΩʔඞͣΠϛϡʔλϒϧͳͷʹ͢Δ ͡Όͳ͍ͱOutputϓϥάΠϯ͕Hash͔ΒΛࢀর͢Δ࣌ʹࠔΔ Outputͷ͢͠͞ΑΓRunnerͷ࣮͕ഁ͠ͳ͍ͷΛ༏ઌͨ͠ߏ ಉ࣌ʹऔΒΕΔϝτϦΫε͕ͳΔ͍ۙ͘ॴʹདྷΔߏ Output࠷ѱ࠷ޙʹ·ͱΊͯͲ͏ʹ͔Ͱ͖ΔͨΊ
3. ϓϥάΠϯΞʔΩςΫνϟ
աڈͷϓϥάΠϯࣄྫ ϓϥάΠϯαϙʔτͷ͋Δͷ͜Ε·ͰͷϓϩδΣΫτ cookpad/barbeque itamae-kitchen/mitamae benchmark-driver/benchmark-driver
ڞ௨: ͔ࣗ͠ϓϥάΠϯॻ͍ͯͳ͍ ͔ࣗ͠ݾධՁͰbenchmark-driverͷΠϯλʔϑΣʔε͕Ұ൪ ͍ ͦͦ͋·ΓநԽʹ͍ͯͳ͍ཁ݅ͳؾ͍ͯ͠Δ ͳͷͰʹฉ͍͓͍͍ͯͯͩ͘͞
ϓϥάΠϯγεςϜͷ֓ཁ
2छྨͷϓϥάΠϯαϙʔτ RunnerϓϥάΠϯ: ܭଌํ๏ͱܭଌ͍ͨ͠ϝτϦΫεΛܾΊΔ OutputϓϥάΠϯ: ܭଌ݁ՌͷूܭΛߦͳ͏
RunnerϓϥάΠϯ Ruby API͔Β࣮ߦ͞ΕΔ͕ɺYAMLͷύʔε͜ͷRunnerϓ ϥάΠϯ͕ߦͳ͏ RunnerͷTypebenchmark-driverίϚϯυͰ༩͑ΒΕΔ͕ɺ YAMLͷதͰॻ͚ΔͷͰɺYAMLͰTypeΛݻఆͭͭ͠ಠ ࣗͷYAMLύʔαΛॻ͘ͱ͖์YAMLͷจ๏͕֦ுͰ͖ Δ
RunnerϓϥάΠϯ σϑΥϧτͷYAMLఆٛͷͷ: ips: ࣮ߦ࣌ؒ/࣮ߦճΛܭଌ͢Δ once: ςετ༻ʹ1ճ͚ͩΒͤΔ time: ࣮ߦ࣌ؒ memory: Max
Resident Set Size (ࠓLinuxͷΈαϙʔτ)
RunnerϓϥάΠϯ ಠࣗͷYAMLఆٛͷͷ: recorded: record outputΛ͏ͱੜ͞ΕΔYAMLͰɺಉ͡ܭ ଌ݁ՌΛԿग़ྗͰ͖Δ ruby_stdout: RubyίϚϯυͷstdoutΛύʔεͯ͠ϝτϦΫε ΛऔΔ
memory
memory
Optcarrot
Optcarrot
Runnerʹ͍ͭͯҙਤతʹαϙʔτΛΊͨػೳ Ruby͔Β࣮ߦ࣌ɺจࣈྻ͡Όͳͯ͘ϒϩοΫͰܭଌରΛ༩͑Δͷα ϙʔτ͍ͯͨ͠ Ҡߦύε + ෳόʔδϣϯ͕ࢦఆ͞Εͳ͚ΕevalͰΒͤͨํ͕͍ ࣮࠷ॳ͜ͷػೳ͕Runnerͱͯ͠ϓϥάΠϯαϙʔτ͞Ε͍ͯͨ(3छ ྨͷϓϥάΠϯ͕͋ͬͨ) υϥΠόͱಉ໊͡લۭؒͰΒͤΔ࣮ߦ͕ΫϦʔϯ͡Όͳ͘ɺಉ͡εΫ ϦϓτΛ࣮ߦ͢Δʹαϙʔτ͕ਏ͔ͬͨ͠ϝϦοτബ͍ͷͰΊͨ
v0.11~12ͷRunnerϓϥάΠϯͷॻ͖ํ YAMLΛநԽ͢ΔJobͱɺYAML͔ΒͦΕΛ࡞ΔJobParserΛఆٛ͢Δ #initializeʹɺCLIΦϓγϣϯΛ࣋ͭconfig, OutputΛநԽ͢Δ output, δϣϒͷϚτϦΫεΛ࡞Δcontexts͕༩͑ΒΕΔ #run ͖์࣮͍͍͕ͯ͠ɺoutputʹରͯ͠ϝιου܊Λܾ·ͬ ͨॱ൪ʹݺͳ͚ΕͳΒͳ͍ #with_warmup,
#with_benchmark, #with_job, #with_context, #report
OutputϓϥάΠϯ લड़ͨ͠ϞσϧΛड͚औͬͯूܭ͢Δׂ 1ͭܭଌ͢Δ͝ͱʹoutputʹͤΔΠϯλʔϑΣʔεͳͷͰɺ ݁Ռ͕ετϦʔϛϯάग़ྗͰ͖Δ શ෦ऴΘͬͨޙʹόϧΫͰग़ྗ͢ΔͨΊͷ؆୯ͳΠϯλʔ ϑΣʔε༻ҙ͍ͯ͠Δ
OutputϓϥάΠϯ ϏϧτΠϯͷϓϥάΠϯ compare: benchmark-ips෩ͷൺֱग़ྗ simple: benchmark/driver.rb ෩ग़ྗ markdown: GFMͷςʔϒϧग़ྗ record:
ܭଌ݁ՌΛอଘͯ͠ผͷoutputʹྲྀ͢༻
OutputϓϥάΠϯ ඇެࣜϓϥάΠϯ gruff: gruff.gemͰάϥϑ͕ग़ྗͰ͖Δ rubybench: RubyBenchʹ݁ՌΛૹ৴
Markdown
gruff
ϓϥάΠϯʹؔ͢Δҙࣄ߲ v0ͷؒԿϓϥάΠϯΠϯλʔϑΣʔε͕มΘ͍ͬͯͯɺv1ʹ͢Δ·Ͱ ֬ఆ͡Όͳ͍Ͱ͢ Runner Typeݻ༗ͷprelude͕άϩʔόϧͳcontexʹೖͬͯΔͷҾ͖ണ͕ ͢༧ఆ ϓϥάΠϯΛ࡞ͬͯͩ͘͞ΔํґଘΛ "benchmark_driver", "~> v0.X.Y"
ʹ ໘͍͚ͯͨͩ͠Δͱॿ͔Γ·͢ ಉ͡ v0.X ͷؒյ͞ͳ͍Α͏ʹ͍ͯ͠·͢
4. ϓϩδΣΫτऴྃޙͷ׆ಈ
RubyBench
RubyBenchͱ RubyͷύϑΥʔϚϯεΛࢹ͢ΔCI benchmark-driver.github.io ΑΓେલ͔Β͋Δ
RubyBenchͷΞʔΩςΫνϟ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
͜͜Λஔ͖͍͑ͨ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
RubyBenchͷ౷߹ JITͱͷൺֱάϥϑޙ͔ΒRubyBenchʹ࣮͞Εͨ όάͬͯͨͷͰͨ͠Γͨ͠ benchmark_driver͚ͷϕϯνϚʔΫηοτਫ਼ͷྑ͍ܭଌػೳ͕ࠓޙ׆༻͠ ͍͢Α͏ɺϕϯνϚʔΧʔͷbenchmark_driverͷஔ͖͑Λ͍ͬͯΔ طʹOptcarrotbenchmark_driverͰܭଌ͞Ε͍ͯΔ ॏෳ͕ͪ͠ͳ࣮͕ϓϥάΠϯͰநԽͰ͖͓ͯΓbenchmark_driverͷϓϥά Πϯػߏ͕׆༂͍ͯ͠Δ
RubyBench͚ʹɺϕϯνϚʔΫͷ֦ॆ͍͖ͬͯ·͢ VladͷMJIT͚ϕϯνϚʔΫूΛՃ༧ఆ Watson1978͞Μͷ֤ϝιου͚ϕϯνϚʔΫूΛมͨ͠ ͷΘ͍͔ͤͯͨͩ͘͞? https://github.com/benchmark-driver/ruby-method-benchmarks
·ͱΊ benchmark_driver ศརͰ͢ͷͰɺੋඇ͝ར༻͍ͩ͘͞ RubyͷߴԽRubyBenchʹطʹຊՌ͕׆༻͞Ε͍ͯ·͢
ँࣙ ຊϓϩδΣΫτɺాߞҰ༷Λ࢝Ίͱͨ͠RubyΞιγΤʔ γϣϯ༷ͷ͝ڠྗʹΑ࣮ͬͯݱ͢Δ͜ͱ͕Ͱ͖·ͨ͠ɻ͜ͷ ΛआΓͯޚྱਃ্͛͠·͢ɻ