Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant

Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant

Rubyアソシエーション開発助成成果報告会
https://rubyassociation.doorkeeper.jp/events/74355

Takashi Kokubun

July 07, 2018
Tweet

More Decks by Takashi Kokubun

Other Decks in Programming

Transcript

  1. Ruby 3x3 ʹ޲͚ͨ
    RubyͷϕϯνϚʔΫ؀ڥͱ੒Ռ
    RubyΞιγΤʔγϣϯ։ൃॿ੒ ੒Ռใࠂձ
    @k0kubun

    View Slide

  2. ࣗݾ঺հ
    GitHub, Twitter: k0kubun
    Rubyίϛολ: JITίϯύΠϥͷ։ൃ, ERBͷϝϯςφϯε
    ࡞ͬͨ: Hamlit, MItamae, md2key, xremap, LLRB

    View Slide

  3. ࠓ೔࿩͢͜ͱ
    1. ຊϓϩδΣΫτ͸ԿΛղܾ͔ͨ͠
    2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά
    3. ϓϥάΠϯΞʔΩςΫνϟ
    4. ϓϩδΣΫτऴྃޙͷ׆ಈ

    View Slide

  4. 1. ຊϓϩδΣΫτ͸ԿΛղܾ͔ͨ͠

    View Slide

  5. ͜Ε·ͰͷRubyͷϕϯνϚʔΫ؀ڥ

    View Slide

  6. ϒϩοΫݺͼग़͠ͷΦʔόʔϔου໰୊
    ඪ४ͷbenchmarkϥΠϒϥϦ΋benchmark-ips΋ɺܭଌ୯Ґ͕ϒ
    ϩοΫʹͳ͍ͬͯΔ
    ϒϩοΫݺͼग़͠͸ී௨ͷϝιουݺͼग़͠૬౰ͷΦʔόʔ
    ϔου͕͋Δ
    ͦΕΑΓ΋࠷దԽ͞Ε໋ͨྩͷܭଌ͸?

    View Slide

  7. ෳ਺ͷRuby࣮૷ؒͰͷൺֱͷ೉͠͞
    ϒϩοΫͰಈతʹܭଌํ๏͕ࢦఆՄೳͩͱɺͦͷϒϩοΫΛ
    ผͷRubyΠϯλϓϦλʹ౉ͯ͠ܭଌ͢Δͷ͸ࠔ೉
    ͔͠͠Ruby 3x3ͷͨΊʹ͸ɺॲཧܥࣗମͷมߋͷޮՌΛධՁ
    ͍ͨ͠

    View Slide

  8. ཚཱ͢ΔϕϯνϚʔΧʔ
    ඪ४ͷbenchmarkϥΠϒϥϦ
    benchmark-ips
    RubyϦϙδτϦ಺ͷ benchmark/driver.rb
    RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ
    VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ

    View Slide

  9. Ͱ͸Ͳ͏͢Δ͔?

    View Slide

  10. ࠷΋͜ͷ໰୊ͷղܾʹ͍ۙπʔϧ
    ඪ४ͷbenchmarkϥΠϒϥϦ
    benchmark-ips
    RubyϦϙδτϦ಺ͷ benchmark/driver.rb
    RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ
    VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ

    View Slide

  11. benchmark/driver.rb ͱ͸
    ෳ਺RubyόΠφϦؒͰൺֱ͕Ͱ͖ɺ࣮ߦ࣌ؒͷଞʹϝϞϦ࢖༻ྔΛܭଌͰ
    ͖Δ
    ϒϩοΫͰ͸ͳ͘whileϧʔϓΛ࢖͏ͨΊɺܭଌͷΦʔόʔϔου͕গͳ͍
    ͨͩ͠RubyϦϙδτϦ಺ͷεΫϦϓτʹಛԽͨ͠ॲཧ͕ೖ͍ͬͯΔ
    benchmark/driver.rb Λ࢖͍΍ͯ͘͢͠օͰ࢖͑͹͍͍ͷͰ͸?
    ͦΕ͕ benchmark_driver.gem

    View Slide

  12. ࠾୒͞ΕͨϓϩδΣΫτ
    “ΑΓ౷ҰతͰ؆୯ͳRubyͷϕϯνϚʔΫ؀ڥͷ࣮ݱ"
    benchmark_driver.gem ͷػೳ֦ॆ
    Ruby 3x3 ʹ޲͚ͨϕϯνϚʔΫηοτͱCIͷ੔උ

    View Slide

  13. benchmark_driver.gem ͷಛ௃

    View Slide

  14. 1. ͋ΒΏΔϝτϦΫεΛ͋ΒΏΔํ๏Ͱग़ྗͰ͖Δ
    ޙड़͢ΔϓϥάΠϯػߏʹΑΓɺ༷ʑͳܭଌํ๏Λ༷ʑͳํ
    ๏ͰूܭՄೳ
    σϑΥϧτͰ͸benchmark-ips෩ͷܭଌɾग़ྗ
    ಉ͡ग़ྗͷ··ϝϞϦ࢖༻ྔΛܭଌͨ͠ΓɺOptcarrotͷfps
    Λܭଌͨ͠ΓͰ͖Δ

    View Slide

  15. 2. ਫ਼౓ͷྑ͍ܭଌεΫϦϓτΛࣗಈͰੜ੒
    whileϧʔϓΛ͢ΔεΫϦϓτΛಈతʹੜ੒͠ɺ͔ͭwhileϧʔϓ
    ʹ͔͔Δ࣌ؒ΋ܭଌͯࠩ͠͠Ҿ͘
    ͦͷͨΊɺܭଌର৅͸ϒϩοΫͰ͸ͳ͘ඞͣจࣈྻͰ༩͑ΒΕΔ
    Rubyͷىಈ࣌ؒ΋ܭଌͯ͠͠·͏ΦϦδφϧͷbenchmark/
    driver.rbͱҧ͍ɺੜ੒ͨ͠εΫϦϓτ಺Ͱ࣌ؒͷܭଌΛ͢Δ
    ىಈ࣌ؒͷϒϨͷӨڹΛड͚ʹ͍͘

    View Slide

  16. 3. RubyʹՃ͑ɺYAMLͰ؆͔ܿͭॊೈʹܭଌ͕ߦͳ͑Δ
    ಈతͳϒϩοΫͰ͸ͳ͘จࣈྻͰ༩͑ΔҎ্ɺΘ͟Θ͟RubyͰϕ
    ϯνϚʔΫΛఆٛͤ͞Δඞཁ͕΋͸΍ͳ͍
    YAMLͷγϯλοΫεͰϕϯνϚʔΫΛఆٛ͢Δͱ୹͘ॻ͚Δ
    .travis.yml ΍ .circleci/config.yml ͘Β͍ͷؾܰ͞Ͱஔ͍ͯ΋Β͍͍ͨ
    Optcarrotʹ͸طʹYAML͕഑ஔ͞Ε͍ͯΔ

    View Slide

  17. 4. ෳ਺ͷRuby࣮૷ؒɺgemͷผόʔδϣϯؒͰൺֱͰ͖Δ
    RubyόΠφϦΛىಈͯ͠ܭଌ͢ΔͨΊɺෳ਺ͷRuby࣮૷ؒͰ
    ൺֱͰ͖ɺRuby 3x3ʹ࠷ద
    Ұ౓RubyΛىಈͨ͠Βͦ͜Ͱಉ͡gemͷෳ਺όʔδϣϯΛڞ
    ଘͤ͞Δ͜ͱ͸ෆՄೳ͕ͩɺbenchmark_driver.gemͰ͸ͦͷൺ
    ֱ͕Ͱ͖Δ

    View Slide

  18. ࣮ࡍͷ࢖༻ྫ

    View Slide

  19. Haml 4 vs Haml 5

    View Slide

  20. ERB Ruby 2.4 vs Ruby 2.5

    View Slide

  21. Optcarrot

    View Slide

  22. VS JRuby

    View Slide

  23. VS TruffleRuby

    View Slide

  24. Docker Time.utc

    View Slide

  25. Docker Time.utc w/ $TZ

    View Slide

  26. Ruby 2.6 block.call

    View Slide

  27. benchmark_driver.gemͷ։ൃ
    Ҏ֎ͷ੒Ռ෺

    View Slide

  28. ϕϯνϚʔΫηοτͷ༻ҙ
    FluentdϕϯνϚʔΫͷࣗಈԽɾbenchmark_driverରԠ
    benchmark-driver/fluentd-benchmark
    MJIT-benchmarksͷbenchmark_driverରԠ
    benchmark-driver/mjit-benchmarks
    Watson1978͞ΜͷϕϯνϚʔΫηοτͷม׵෺
    benchmark-driver/ruby-method-benchmarks

    View Slide

  29. benchmark-driver.github.io

    View Slide

  30. benchmark-driver.github.io
    GitHub pages
    ϕϯνϚʔΫαʔόʔ
    cronͰruby/ruby
    ΛϙʔϦϯά
    + ϕϯνϚʔΫ
    middleman build
    git push
    Travis
    ϕϯνϚʔΫ݁Ռͷ
    YAMLΛอଘ͢Δ
    GitHubϦϙδτϦ
    Πϯλʔωοτ
    ݁ՌͷYAMLΛ
    git push
    webhook
    ݟΕΔ
    आΓͯΔαʔόʔͷ࢓༷Ͱ
    ֎෦͔ΒΞΫηεͰ͖ͳ͍

    View Slide

  31. benchmark-driver.github.io
    benchmark-driver/ruby-builder, skybench, sky2-infra…
    RubyBenchͩͱϓϩδΣΫτظؒதʹ͍͡Γ౗͢ͷ͕೉͍͠
    ͱߟ͑ͨͨΊɺࢼݧతʹผϓϩδΣΫτͱͯ࢝͠ಈ
    ࣮ࡍʹ͸ϓϩδΣΫτऴྃޙ੠Λ͔͚ͨΒ͙͢৭ʑͳݖݶ
    Λ΋Β͑ͨ
    ৄࡉ͸ޙ൒Ͱ࿩͠·͢

    View Slide

  32. 2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά

    View Slide

  33. ม਺ଟ͗͢໰୊
    ܭଌର৅ͷεΫϦϓτୡ
    ࣄલ࣮ߦεΫϦϓτͷ૊߹ͤ
    ܭଌ͢ΔϝτϦΫεɺ͋Δ͍͸ܭଌํ๏
    Rubyͷόʔδϣϯ
    gemͷόʔδϣϯ

    View Slide

  34. ෳ਺ϝτϦΫεͷಉ࣌ܭଌ
    ܭଌ͢ΔεΫϦϓτͷҰཡͱRubyͷόʔδϣϯҰཡ͚ͩͰදʹͳ͍ͬͯ
    ͯɺෳ਺ͷϝτϦΫεΛࠞͥΔͱਏ͍
    Ͱ΋user, systemͷCPU࣌ؒͱ࣮࣌ؒΛಉ࣌ʹग़͢Έ͍ͨͳͷ΋΍Γ͍ͨ
    (Ϟσϧ্͸ରԠ͚ͨ͠Ͳ࣮͸·ͩ͜Ε࣮૷ͯ͠ͳ͍)
    શͯͷϢʔεέʔεΛΧόʔͭͭ͠΋ഁ୼͍ͯ͠ͳ͍ϞσϧΛͲ͏ʹ͔
    ࡞Εͳ͍͔?

    View Slide

  35. v0.11~v0.12ݱࡏͷϞσϧ

    View Slide

  36. v0.11~v0.12ݱࡏͷϞσϧ

    View Slide

  37. v0.11~v0.12ݱࡏͷϞσϧ

    View Slide

  38. ݕ౼ͨ͠ࣄ߲
    HashͷΩʔ͸ඞͣΠϛϡʔλϒϧͳ΋ͷʹ͢Δ
    ͡Όͳ͍ͱOutputϓϥάΠϯ͕Hash͔Β஋Λࢀর͢Δ࣌ʹࠔΔ
    Outputͷ͠΍͢͞ΑΓRunnerͷ࣮૷͕ഁ୼͠ͳ͍ͷΛ༏ઌͨ͠ߏ଄
    ಉ࣌ʹऔΒΕΔϝτϦΫε͕ͳΔ΂͍ۙ͘৔ॴʹདྷΔߏ଄
    Output͸࠷ѱ࠷ޙʹ·ͱΊͯͲ͏ʹ͔Ͱ͖ΔͨΊ

    View Slide

  39. 3. ϓϥάΠϯΞʔΩςΫνϟ

    View Slide

  40. աڈͷϓϥάΠϯࣄྫ
    ϓϥάΠϯαϙʔτͷ͋Δ๻ͷ͜Ε·ͰͷϓϩδΣΫτ
    cookpad/barbeque
    itamae-kitchen/mitamae
    benchmark-driver/benchmark-driver

    View Slide

  41. ڞ௨఺: ࣗ෼͔͠ϓϥάΠϯॻ͍ͯͳ͍
    ͔͠΋ࣗݾධՁͰbenchmark-driverͷΠϯλʔϑΣʔε͕Ұ൪
    ΍͹͍
    ͦ΋ͦ΋͋·Γந৅Խʹ޲͍ͯͳ͍ཁ݅ͳؾ΋͍ͯ͠Δ
    ͳͷͰ࿩൒෼ʹฉ͍͓͍͍ͯͯͩ͘͞

    View Slide

  42. ϓϥάΠϯγεςϜͷ֓ཁ

    View Slide

  43. 2छྨͷϓϥάΠϯαϙʔτ
    RunnerϓϥάΠϯ: ܭଌํ๏ͱܭଌ͍ͨ͠ϝτϦΫεΛܾΊΔ
    OutputϓϥάΠϯ: ܭଌ݁ՌͷूܭΛߦͳ͏

    View Slide

  44. RunnerϓϥάΠϯ
    Ruby API͔Β࣮ߦ͞ΕΔ͕ɺYAMLͷύʔε΋͜ͷRunnerϓ
    ϥάΠϯ͕ߦͳ͏
    RunnerͷType͸benchmark-driverίϚϯυͰ༩͑ΒΕΔ͕ɺ
    YAMLͷதͰ΋ॻ͚ΔͷͰɺYAML಺ͰTypeΛݻఆͭͭ͠ಠ
    ࣗͷYAMLύʔαΛॻ͘ͱ޷͖์୊YAMLͷจ๏͕֦ுͰ͖
    Δ

    View Slide

  45. RunnerϓϥάΠϯ
    σϑΥϧτͷYAMLఆٛͷ΋ͷ:
    ips: ࣮ߦ࣌ؒ/࣮ߦճ਺Λܭଌ͢Δ
    once: ςετ༻ʹ1ճ͚ͩ૸ΒͤΔ
    time: ࣮ߦ࣌ؒ
    memory: Max Resident Set Size (ࠓ͸LinuxͷΈαϙʔτ)

    View Slide

  46. RunnerϓϥάΠϯ
    ಠࣗͷYAMLఆٛͷ΋ͷ:
    recorded: record outputΛ࢖͏ͱੜ੒͞ΕΔYAMLͰɺಉ͡ܭ
    ଌ݁ՌΛԿ౓΋ग़ྗͰ͖Δ
    ruby_stdout: RubyίϚϯυͷstdoutΛύʔεͯ͠ϝτϦΫε
    ΛऔΔ

    View Slide

  47. memory

    View Slide

  48. memory

    View Slide

  49. Optcarrot

    View Slide

  50. Optcarrot

    View Slide

  51. Runnerʹ͍ͭͯҙਤతʹαϙʔτΛ΍Ίͨػೳ
    Ruby͔Β࣮ߦ࣌ɺจࣈྻ͡Όͳͯ͘ϒϩοΫͰܭଌର৅Λ༩͑Δͷ΋α
    ϙʔτ͍ͯͨ͠
    Ҡߦύε + ෳ਺όʔδϣϯ͕ࢦఆ͞Εͳ͚Ε͹evalͰ૸Βͤͨํ͕଎͍
    ࣮͸࠷ॳ͸͜ͷػೳ͕Runnerͱͯ͠ϓϥάΠϯαϙʔτ͞Ε͍ͯͨ(3छ
    ྨͷϓϥάΠϯ͕͋ͬͨ)
    υϥΠόͱಉ໊͡લۭؒͰ૸ΒͤΔ෼࣮ߦ͕ΫϦʔϯ͡Όͳ͘ɺಉ͡εΫ
    ϦϓτΛ࣮ߦ͢Δʹ͸αϙʔτ͕ਏ͔ͬͨ͠ϝϦοτ΋ബ͍ͷͰ΍Ίͨ

    View Slide

  52. v0.11~12ͷRunnerϓϥάΠϯͷॻ͖ํ
    YAMLΛந৅Խ͢ΔJobͱɺYAML͔ΒͦΕΛ࡞ΔJobParserΛఆٛ͢Δ
    #initializeʹ͸ɺCLIΦϓγϣϯΛ࣋ͭconfig, OutputΛந৅Խ͢Δ
    output, δϣϒͷϚτϦΫεΛ࡞Δcontexts͕༩͑ΒΕΔ
    #run ͸޷͖์୊࣮૷͍͍͕ͯ͠ɺoutputʹରͯ͠ϝιου܊Λܾ·ͬ
    ͨॱ൪ʹݺ͹ͳ͚Ε͹ͳΒͳ͍
    #with_warmup, #with_benchmark, #with_job, #with_context, #report

    View Slide

  53. OutputϓϥάΠϯ
    લड़ͨ͠ϞσϧΛड͚औͬͯूܭ͢Δ໾ׂ
    1ͭܭଌ͢Δ͝ͱʹoutputʹ౉ͤΔΠϯλʔϑΣʔεͳͷͰɺ
    ݁Ռ͕ετϦʔϛϯάग़ྗͰ͖Δ
    શ෦ऴΘͬͨޙʹόϧΫͰग़ྗ͢ΔͨΊͷ؆୯ͳΠϯλʔ
    ϑΣʔε΋༻ҙ͍ͯ͠Δ

    View Slide

  54. OutputϓϥάΠϯ
    ϏϧτΠϯͷϓϥάΠϯ
    compare: benchmark-ips෩ͷൺֱग़ྗ
    simple: benchmark/driver.rb ෩ग़ྗ
    markdown: GFMͷςʔϒϧग़ྗ
    record: ܭଌ݁ՌΛอଘͯ͠ผͷoutputʹྲྀ͢༻

    View Slide

  55. OutputϓϥάΠϯ
    ඇެࣜϓϥάΠϯ
    gruff: gruff.gemͰάϥϑ͕ग़ྗͰ͖Δ
    rubybench: RubyBenchʹ݁ՌΛૹ৴

    View Slide

  56. Markdown

    View Slide

  57. gruff

    View Slide

  58. ϓϥάΠϯʹؔ͢Δ஫ҙࣄ߲
    v0ͷؒԿ౓΋ϓϥάΠϯΠϯλʔϑΣʔε͕มΘ͍ͬͯͯɺv1ʹ͢Δ·Ͱ͸
    ֬ఆ͡Όͳ͍Ͱ͢
    Runner Typeݻ༗ͷprelude͕άϩʔόϧͳcontexʹೖͬͯΔͷ͸Ҿ͖ണ͕
    ͢༧ఆ
    ϓϥάΠϯΛ࡞ͬͯͩ͘͞Δํ͸ґଘΛ "benchmark_driver", "~> v0.X.Y" ʹ
    ౰໘͸͍͚ͯͨͩ͠Δͱॿ͔Γ·͢
    ಉ͡ v0.X ͷؒ͸յ͞ͳ͍Α͏ʹ͍ͯ͠·͢

    View Slide

  59. 4. ϓϩδΣΫτऴྃޙͷ׆ಈ

    View Slide

  60. RubyBench

    View Slide

  61. RubyBenchͱ͸
    RubyͷύϑΥʔϚϯεΛ؂ࢹ͢ΔCI
    benchmark-driver.github.io ΑΓ΋େ෼લ͔Β͋Δ

    View Slide

  62. RubyBenchͷΞʔΩςΫνϟ
    RubyBench ϫʔΧʔ
    task-spooler
    → docker run ruby …
    GitHub fork for
    ruby/ruby
    RubyBench web
    Rails ΞϓϦέʔγϣϯ
    → Sidekiq
    GitHub
    ruby/ruby
    cronͰߋ৽
    Webhook
    ssh tsp …

    View Slide

  63. ͜͜Λஔ͖׵͍͑ͨ
    RubyBench ϫʔΧʔ
    task-spooler
    → docker run ruby …
    GitHub fork for
    ruby/ruby
    RubyBench web
    Rails ΞϓϦέʔγϣϯ
    → Sidekiq
    GitHub
    ruby/ruby
    cronͰߋ৽
    Webhook
    ssh tsp …

    View Slide

  64. RubyBench΁ͷ౷߹
    JITͱͷൺֱάϥϑ͸ޙ͔ΒRubyBenchʹ΋࣮૷͞Εͨ
    όάͬͯͨͷͰ௚ͨ͠Γͨ͠
    benchmark_driver޲͚ͷϕϯνϚʔΫηοτ΍ਫ਼౓ͷྑ͍ܭଌػೳ͕ࠓޙ׆༻͠
    ΍͍͢Α͏ɺϕϯνϚʔΧʔͷbenchmark_driver΁ͷஔ͖׵͑Λ΍͍ͬͯΔ
    طʹOptcarrot͸benchmark_driverͰܭଌ͞Ε͍ͯΔ
    ॏෳ͕ͪ͠ͳ࣮૷͕ϓϥάΠϯͰந৅ԽͰ͖͓ͯΓbenchmark_driverͷϓϥά
    Πϯػߏ͕׆༂͍ͯ͠Δ

    View Slide

  65. RubyBench޲͚ʹɺϕϯνϚʔΫͷ֦ॆ΋΍͍͖ͬͯ·͢
    VladͷMJIT޲͚ϕϯνϚʔΫूΛ௥Ճ༧ఆ
    Watson1978͞Μͷ֤ϝιου޲͚ϕϯνϚʔΫूΛม׵ͨ͠
    ΋ͷ΋࢖Θ͍͔ͤͯͨͩ͘͞΋?
    https://github.com/benchmark-driver/ruby-method-benchmarks

    View Slide

  66. ·ͱΊ
    benchmark_driver ศརͰ͢ͷͰɺੋඇ͝ར༻͍ͩ͘͞
    Rubyͷߴ଎Խ΍RubyBenchʹطʹຊ੒Ռ͕׆༻͞Ε͍ͯ·͢

    View Slide

  67. ँࣙ
    ຊϓϩδΣΫτ͸ɺ࡫ాߞҰ༷Λ࢝Ίͱͨ͠RubyΞιγΤʔ
    γϣϯ༷ͷ͝ڠྗʹΑ࣮ͬͯݱ͢Δ͜ͱ͕Ͱ͖·ͨ͠ɻ͜ͷ
    ৔ΛआΓͯޚྱਃ্͛͠·͢ɻ

    View Slide