Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant
Search
Takashi Kokubun
July 07, 2018
Programming
10
13k
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant
Rubyアソシエーション開発助成成果報告会
https://rubyassociation.doorkeeper.jp/events/74355
Takashi Kokubun
July 07, 2018
Tweet
Share
More Decks by Takashi Kokubun
See All by Takashi Kokubun
Ruby JIT Hacking Guide / RubyKaigi 2023
k0kubun
1
8.9k
YJIT: Dive into Ruby's JIT compiler written in Rust / Rust.Tokyo 2022
k0kubun
1
1.8k
Towards Ruby 4 JIT / RubyKaigi 2022
k0kubun
3
10k
Optimizing Production Performance with MRI JIT / RubyConf 2021
k0kubun
1
340
Why Ruby's JIT was slow / RubyKaigi Takeout 2021
k0kubun
3
1.7k
数時間かかる週一リリースを毎日何度も爆速でできるようにするまで / CI/CD Conference 2021
k0kubun
21
14k
Ruby 3 JIT's roadmap / RubyConf China 2020
k0kubun
0
660
Ruby 3.0 JIT on Rails
k0kubun
9
8.8k
JIT ロードマップ / Ruby 3 さみっと
k0kubun
2
1.2k
Other Decks in Programming
See All in Programming
try!Swift Tokyo 2024 参加報告 LT
akidon0000
1
220
PHPはいつから死んでいるかの調査
chiroruxx
1
400
AWS CDKコントリビュートTIPS / aws-cdk-contribution-tips
gotok365
2
230
Kotlin Multiplatform at Stable and Beyond (Android Makers 2024)
zsmb
0
310
Amazon SQSコンシューマー疎結合への旅 - 出張! #DevelopersIO IT技術ブログの中の人が語る勉強会 #3
quiver
0
280
二郎系ラーメンのコールで学ぶ AST 解析
memory1994
PRO
7
1.7k
Rubyでたのしむクリエイティブコーディング/Enjoy Creative coding with Ruby
chobishiba
1
180
#phpcon_odawara オープン・クローズドなテストフィクスチャを求めて / open closed test fixtures
77web
3
230
Ruby GitHub Packages
bkuhlmann
0
630
デフォルトにして至高、RubyMineの大好きな所
ruzia
0
440
VS Code をプロダクトにどう取り込むか
onomax
1
430
Netty Chicago Java User Group 2024-04-17
sullis
0
180
Featured
See All Featured
Building Your Own Lightsaber
phodgson
99
5.7k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
2
1.3k
[RailsConf 2023] Rails as a piece of cake
palkan
23
4k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
21
1.6k
Adopting Sorbet at Scale
ufuk
68
8.6k
Robots, Beer and Maslow
schacon
PRO
155
7.9k
No one is an island. Learnings from fostering a developers community.
thoeni
16
2.1k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
155
14k
Building Effective Engineering Teams - LeadDev
addyosmani
28
1.9k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
2
3.4k
4 Signs Your Business is Dying
shpigford
175
21k
KATA
mclloyd
15
12k
Transcript
Ruby 3x3 ʹ͚ͨ RubyͷϕϯνϚʔΫڥͱՌ RubyΞιγΤʔγϣϯ։ൃॿ Ռใࠂձ @k0kubun
ࣗݾհ GitHub, Twitter: k0kubun Rubyίϛολ: JITίϯύΠϥͷ։ൃ, ERBͷϝϯςφϯε ࡞ͬͨ: Hamlit, MItamae,
md2key, xremap, LLRB
ࠓ͢͜ͱ 1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠ 2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά 3. ϓϥάΠϯΞʔΩςΫνϟ 4. ϓϩδΣΫτऴྃޙͷ׆ಈ
1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠
͜Ε·ͰͷRubyͷϕϯνϚʔΫڥ
ϒϩοΫݺͼग़͠ͷΦʔόʔϔου ඪ४ͷbenchmarkϥΠϒϥϦbenchmark-ipsɺܭଌ୯Ґ͕ϒ ϩοΫʹͳ͍ͬͯΔ ϒϩοΫݺͼग़͠ී௨ͷϝιουݺͼग़͠૬ͷΦʔόʔ ϔου͕͋Δ ͦΕΑΓ࠷దԽ͞Ε໋ͨྩͷܭଌ?
ෳͷRuby࣮ؒͰͷൺֱͷ͠͞ ϒϩοΫͰಈతʹܭଌํ๏͕ࢦఆՄೳͩͱɺͦͷϒϩοΫΛ ผͷRubyΠϯλϓϦλʹͯ͠ܭଌ͢Δͷࠔ ͔͠͠Ruby 3x3ͷͨΊʹɺॲཧܥࣗମͷมߋͷޮՌΛධՁ ͍ͨ͠
ཚཱ͢ΔϕϯνϚʔΧʔ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
ͰͲ͏͢Δ͔?
࠷͜ͷͷղܾʹ͍ۙπʔϧ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
benchmark/driver.rb ͱ ෳRubyόΠφϦؒͰൺֱ͕Ͱ͖ɺ࣮ߦ࣌ؒͷଞʹϝϞϦ༻ྔΛܭଌͰ ͖Δ ϒϩοΫͰͳ͘whileϧʔϓΛ͏ͨΊɺܭଌͷΦʔόʔϔου͕গͳ͍ ͨͩ͠RubyϦϙδτϦͷεΫϦϓτʹಛԽͨ͠ॲཧ͕ೖ͍ͬͯΔ benchmark/driver.rb Λ͍ͯ͘͢͠օͰ͍͍͑ͷͰ? ͦΕ͕ benchmark_driver.gem
࠾͞ΕͨϓϩδΣΫτ “ΑΓ౷ҰతͰ؆୯ͳRubyͷϕϯνϚʔΫڥͷ࣮ݱ" benchmark_driver.gem ͷػೳ֦ॆ Ruby 3x3 ʹ͚ͨϕϯνϚʔΫηοτͱCIͷඋ
benchmark_driver.gem ͷಛ
1. ͋ΒΏΔϝτϦΫεΛ͋ΒΏΔํ๏Ͱग़ྗͰ͖Δ ޙड़͢ΔϓϥάΠϯػߏʹΑΓɺ༷ʑͳܭଌํ๏Λ༷ʑͳํ ๏ͰूܭՄೳ σϑΥϧτͰbenchmark-ips෩ͷܭଌɾग़ྗ ಉ͡ग़ྗͷ··ϝϞϦ༻ྔΛܭଌͨ͠ΓɺOptcarrotͷfps Λܭଌͨ͠ΓͰ͖Δ
2. ਫ਼ͷྑ͍ܭଌεΫϦϓτΛࣗಈͰੜ whileϧʔϓΛ͢ΔεΫϦϓτΛಈతʹੜ͠ɺ͔ͭwhileϧʔϓ ʹ͔͔Δ࣌ؒܭଌͯࠩ͠͠Ҿ͘ ͦͷͨΊɺܭଌରϒϩοΫͰͳ͘ඞͣจࣈྻͰ༩͑ΒΕΔ Rubyͷىಈ࣌ؒܭଌͯ͠͠·͏ΦϦδφϧͷbenchmark/ driver.rbͱҧ͍ɺੜͨ͠εΫϦϓτͰ࣌ؒͷܭଌΛ͢Δ ىಈ࣌ؒͷϒϨͷӨڹΛड͚ʹ͍͘
3. RubyʹՃ͑ɺYAMLͰ؆͔ܿͭॊೈʹܭଌ͕ߦͳ͑Δ ಈతͳϒϩοΫͰͳ͘จࣈྻͰ༩͑ΔҎ্ɺΘ͟Θ͟RubyͰϕ ϯνϚʔΫΛఆٛͤ͞Δඞཁ͕ͳ͍ YAMLͷγϯλοΫεͰϕϯνϚʔΫΛఆٛ͢Δͱ͘ॻ͚Δ .travis.yml .circleci/config.yml ͘Β͍ͷؾܰ͞Ͱஔ͍ͯΒ͍͍ͨ OptcarrotʹطʹYAML͕ஔ͞Ε͍ͯΔ
4. ෳͷRuby࣮ؒɺgemͷผόʔδϣϯؒͰൺֱͰ͖Δ RubyόΠφϦΛىಈͯ͠ܭଌ͢ΔͨΊɺෳͷRuby࣮ؒͰ ൺֱͰ͖ɺRuby 3x3ʹ࠷ద ҰRubyΛىಈͨ͠Βͦ͜Ͱಉ͡gemͷෳόʔδϣϯΛڞ ଘͤ͞Δ͜ͱෆՄೳ͕ͩɺbenchmark_driver.gemͰͦͷൺ ֱ͕Ͱ͖Δ
࣮ࡍͷ༻ྫ
Haml 4 vs Haml 5
ERB Ruby 2.4 vs Ruby 2.5
Optcarrot
VS JRuby
VS TruffleRuby
Docker Time.utc
Docker Time.utc w/ $TZ
Ruby 2.6 block.call
benchmark_driver.gemͷ։ൃ Ҏ֎ͷՌ
ϕϯνϚʔΫηοτͷ༻ҙ FluentdϕϯνϚʔΫͷࣗಈԽɾbenchmark_driverରԠ benchmark-driver/fluentd-benchmark MJIT-benchmarksͷbenchmark_driverରԠ benchmark-driver/mjit-benchmarks Watson1978͞ΜͷϕϯνϚʔΫηοτͷม benchmark-driver/ruby-method-benchmarks
benchmark-driver.github.io
benchmark-driver.github.io GitHub pages ϕϯνϚʔΫαʔόʔ cronͰruby/ruby ΛϙʔϦϯά + ϕϯνϚʔΫ middleman build
git push Travis ϕϯνϚʔΫ݁Ռͷ YAMLΛอଘ͢Δ GitHubϦϙδτϦ Πϯλʔωοτ ݁ՌͷYAMLΛ git push webhook ݟΕΔ आΓͯΔαʔόʔͷ༷Ͱ ֎෦͔ΒΞΫηεͰ͖ͳ͍
benchmark-driver.github.io benchmark-driver/ruby-builder, skybench, sky2-infra… RubyBenchͩͱϓϩδΣΫτظؒதʹ͍͡Γ͢ͷ͕͍͠ ͱߟ͑ͨͨΊɺࢼݧతʹผϓϩδΣΫτͱͯ࢝͠ಈ ࣮ࡍʹϓϩδΣΫτऴྃޙΛ͔͚ͨΒ͙͢৭ʑͳݖݶ ΛΒ͑ͨ ৄࡉޙͰ͠·͢
2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά
มଟ͗͢ ܭଌରͷεΫϦϓτୡ ࣄલ࣮ߦεΫϦϓτͷ߹ͤ ܭଌ͢ΔϝτϦΫεɺ͋Δ͍ܭଌํ๏ Rubyͷόʔδϣϯ gemͷόʔδϣϯ
ෳϝτϦΫεͷಉ࣌ܭଌ ܭଌ͢ΔεΫϦϓτͷҰཡͱRubyͷόʔδϣϯҰཡ͚ͩͰදʹͳ͍ͬͯ ͯɺෳͷϝτϦΫεΛࠞͥΔͱਏ͍ Ͱuser, systemͷCPU࣌ؒͱ࣮࣌ؒΛಉ࣌ʹग़͢Έ͍ͨͳͷΓ͍ͨ (Ϟσϧ্ରԠ͚ͨ͠Ͳ࣮·ͩ͜Ε࣮ͯ͠ͳ͍) શͯͷϢʔεέʔεΛΧόʔͭͭ͠ഁ͍ͯ͠ͳ͍ϞσϧΛͲ͏ʹ͔ ࡞Εͳ͍͔?
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
ݕ౼ͨ͠ࣄ߲ HashͷΩʔඞͣΠϛϡʔλϒϧͳͷʹ͢Δ ͡Όͳ͍ͱOutputϓϥάΠϯ͕Hash͔ΒΛࢀর͢Δ࣌ʹࠔΔ Outputͷ͢͠͞ΑΓRunnerͷ࣮͕ഁ͠ͳ͍ͷΛ༏ઌͨ͠ߏ ಉ࣌ʹऔΒΕΔϝτϦΫε͕ͳΔ͍ۙ͘ॴʹདྷΔߏ Output࠷ѱ࠷ޙʹ·ͱΊͯͲ͏ʹ͔Ͱ͖ΔͨΊ
3. ϓϥάΠϯΞʔΩςΫνϟ
աڈͷϓϥάΠϯࣄྫ ϓϥάΠϯαϙʔτͷ͋Δͷ͜Ε·ͰͷϓϩδΣΫτ cookpad/barbeque itamae-kitchen/mitamae benchmark-driver/benchmark-driver
ڞ௨: ͔ࣗ͠ϓϥάΠϯॻ͍ͯͳ͍ ͔ࣗ͠ݾධՁͰbenchmark-driverͷΠϯλʔϑΣʔε͕Ұ൪ ͍ ͦͦ͋·ΓநԽʹ͍ͯͳ͍ཁ݅ͳؾ͍ͯ͠Δ ͳͷͰʹฉ͍͓͍͍ͯͯͩ͘͞
ϓϥάΠϯγεςϜͷ֓ཁ
2छྨͷϓϥάΠϯαϙʔτ RunnerϓϥάΠϯ: ܭଌํ๏ͱܭଌ͍ͨ͠ϝτϦΫεΛܾΊΔ OutputϓϥάΠϯ: ܭଌ݁ՌͷूܭΛߦͳ͏
RunnerϓϥάΠϯ Ruby API͔Β࣮ߦ͞ΕΔ͕ɺYAMLͷύʔε͜ͷRunnerϓ ϥάΠϯ͕ߦͳ͏ RunnerͷTypebenchmark-driverίϚϯυͰ༩͑ΒΕΔ͕ɺ YAMLͷதͰॻ͚ΔͷͰɺYAMLͰTypeΛݻఆͭͭ͠ಠ ࣗͷYAMLύʔαΛॻ͘ͱ͖์YAMLͷจ๏͕֦ுͰ͖ Δ
RunnerϓϥάΠϯ σϑΥϧτͷYAMLఆٛͷͷ: ips: ࣮ߦ࣌ؒ/࣮ߦճΛܭଌ͢Δ once: ςετ༻ʹ1ճ͚ͩΒͤΔ time: ࣮ߦ࣌ؒ memory: Max
Resident Set Size (ࠓLinuxͷΈαϙʔτ)
RunnerϓϥάΠϯ ಠࣗͷYAMLఆٛͷͷ: recorded: record outputΛ͏ͱੜ͞ΕΔYAMLͰɺಉ͡ܭ ଌ݁ՌΛԿग़ྗͰ͖Δ ruby_stdout: RubyίϚϯυͷstdoutΛύʔεͯ͠ϝτϦΫε ΛऔΔ
memory
memory
Optcarrot
Optcarrot
Runnerʹ͍ͭͯҙਤతʹαϙʔτΛΊͨػೳ Ruby͔Β࣮ߦ࣌ɺจࣈྻ͡Όͳͯ͘ϒϩοΫͰܭଌରΛ༩͑Δͷα ϙʔτ͍ͯͨ͠ Ҡߦύε + ෳόʔδϣϯ͕ࢦఆ͞Εͳ͚ΕevalͰΒͤͨํ͕͍ ࣮࠷ॳ͜ͷػೳ͕Runnerͱͯ͠ϓϥάΠϯαϙʔτ͞Ε͍ͯͨ(3छ ྨͷϓϥάΠϯ͕͋ͬͨ) υϥΠόͱಉ໊͡લۭؒͰΒͤΔ࣮ߦ͕ΫϦʔϯ͡Όͳ͘ɺಉ͡εΫ ϦϓτΛ࣮ߦ͢Δʹαϙʔτ͕ਏ͔ͬͨ͠ϝϦοτബ͍ͷͰΊͨ
v0.11~12ͷRunnerϓϥάΠϯͷॻ͖ํ YAMLΛநԽ͢ΔJobͱɺYAML͔ΒͦΕΛ࡞ΔJobParserΛఆٛ͢Δ #initializeʹɺCLIΦϓγϣϯΛ࣋ͭconfig, OutputΛநԽ͢Δ output, δϣϒͷϚτϦΫεΛ࡞Δcontexts͕༩͑ΒΕΔ #run ͖์࣮͍͍͕ͯ͠ɺoutputʹରͯ͠ϝιου܊Λܾ·ͬ ͨॱ൪ʹݺͳ͚ΕͳΒͳ͍ #with_warmup,
#with_benchmark, #with_job, #with_context, #report
OutputϓϥάΠϯ લड़ͨ͠ϞσϧΛड͚औͬͯूܭ͢Δׂ 1ͭܭଌ͢Δ͝ͱʹoutputʹͤΔΠϯλʔϑΣʔεͳͷͰɺ ݁Ռ͕ετϦʔϛϯάग़ྗͰ͖Δ શ෦ऴΘͬͨޙʹόϧΫͰग़ྗ͢ΔͨΊͷ؆୯ͳΠϯλʔ ϑΣʔε༻ҙ͍ͯ͠Δ
OutputϓϥάΠϯ ϏϧτΠϯͷϓϥάΠϯ compare: benchmark-ips෩ͷൺֱग़ྗ simple: benchmark/driver.rb ෩ग़ྗ markdown: GFMͷςʔϒϧग़ྗ record:
ܭଌ݁ՌΛอଘͯ͠ผͷoutputʹྲྀ͢༻
OutputϓϥάΠϯ ඇެࣜϓϥάΠϯ gruff: gruff.gemͰάϥϑ͕ग़ྗͰ͖Δ rubybench: RubyBenchʹ݁ՌΛૹ৴
Markdown
gruff
ϓϥάΠϯʹؔ͢Δҙࣄ߲ v0ͷؒԿϓϥάΠϯΠϯλʔϑΣʔε͕มΘ͍ͬͯͯɺv1ʹ͢Δ·Ͱ ֬ఆ͡Όͳ͍Ͱ͢ Runner Typeݻ༗ͷprelude͕άϩʔόϧͳcontexʹೖͬͯΔͷҾ͖ണ͕ ͢༧ఆ ϓϥάΠϯΛ࡞ͬͯͩ͘͞ΔํґଘΛ "benchmark_driver", "~> v0.X.Y"
ʹ ໘͍͚ͯͨͩ͠Δͱॿ͔Γ·͢ ಉ͡ v0.X ͷؒյ͞ͳ͍Α͏ʹ͍ͯ͠·͢
4. ϓϩδΣΫτऴྃޙͷ׆ಈ
RubyBench
RubyBenchͱ RubyͷύϑΥʔϚϯεΛࢹ͢ΔCI benchmark-driver.github.io ΑΓେલ͔Β͋Δ
RubyBenchͷΞʔΩςΫνϟ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
͜͜Λஔ͖͍͑ͨ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
RubyBenchͷ౷߹ JITͱͷൺֱάϥϑޙ͔ΒRubyBenchʹ࣮͞Εͨ όάͬͯͨͷͰͨ͠Γͨ͠ benchmark_driver͚ͷϕϯνϚʔΫηοτਫ਼ͷྑ͍ܭଌػೳ͕ࠓޙ׆༻͠ ͍͢Α͏ɺϕϯνϚʔΧʔͷbenchmark_driverͷஔ͖͑Λ͍ͬͯΔ طʹOptcarrotbenchmark_driverͰܭଌ͞Ε͍ͯΔ ॏෳ͕ͪ͠ͳ࣮͕ϓϥάΠϯͰநԽͰ͖͓ͯΓbenchmark_driverͷϓϥά Πϯػߏ͕׆༂͍ͯ͠Δ
RubyBench͚ʹɺϕϯνϚʔΫͷ֦ॆ͍͖ͬͯ·͢ VladͷMJIT͚ϕϯνϚʔΫूΛՃ༧ఆ Watson1978͞Μͷ֤ϝιου͚ϕϯνϚʔΫूΛมͨ͠ ͷΘ͍͔ͤͯͨͩ͘͞? https://github.com/benchmark-driver/ruby-method-benchmarks
·ͱΊ benchmark_driver ศརͰ͢ͷͰɺੋඇ͝ར༻͍ͩ͘͞ RubyͷߴԽRubyBenchʹطʹຊՌ͕׆༻͞Ε͍ͯ·͢
ँࣙ ຊϓϩδΣΫτɺాߞҰ༷Λ࢝Ίͱͨ͠RubyΞιγΤʔ γϣϯ༷ͷ͝ڠྗʹΑ࣮ͬͯݱ͢Δ͜ͱ͕Ͱ͖·ͨ͠ɻ͜ͷ ΛआΓͯޚྱਃ্͛͠·͢ɻ