Upgrade to Pro — share decks privately, control downloads, hide ads and more …

今年のRubyKaigiはProfiler Year🤘

osyoyu
April 25, 2024

今年のRubyKaigiはProfiler Year🤘

osyoyu

April 25, 2024
Tweet

More Decks by osyoyu

Other Decks in Technology

Transcript

  1. pp @osyoyu ʮ͏ͳ͹ΜʂʯϑΝʔετϥΠϒʂ ࠓि຤ 4/27 (౔) ໷ @ େԬࢁ όϯυϝϯόʔશһ

    Rubyist & RubyKaigi ࢀՃ ৄࡉ͸ "unasuke όϯυ" Ͱݕࡧʢunasuke = όϯυϝϯόʔʣ
  2. ϓϩϑΝΠϥ Stackprof, ruby-prof, rbspy, ... • ೔ຊޠͰ͸ʮੑೳղੳπʔϧʯ • ϓϩάϥϜதͷ "஗͍"

    ՕॴΛൃݟͰ͖Δ • ISUCON ͳͲͰ͓ੈ࿩ʹͳͬͨਓ΋ଟ͍ͷͰ͸ͳ͍Ͱ͠ΐ͏͔
  3. Pf2 ࠷ڧศརϓϩϑΝΠϥΛ࡞͍ͬͯ·͢ • ৽͍͠ϓϩϑΝΠϥΛ࡞ͬͯ·͢ʂ • RubyKaigi Ͱ͸ Pf2 ͷ࢓૊Έ΍ɺ࡞Δ্Ͱͷ೉͠ ͞ͷ࿩Λ͠·͢

    • Pf2 ͷ࢖͍͔ͨͷ࿩͸ࡢ೔ (4/23) ͷ Gotanda.rb #58 ͷࢿྉΛݟ͍ͯͩ͘͞ github.com/osyoyu/pf2
  4. Pf2 ͷ࢓૊Έ ΈΜͳڵຯ͋ΔΑͶ Sampling Pf2 is a sampling pro fi

    ler. This means that Pf2 collects samples of program execution periodically, instead of tracing every action (e.g. method invocations and returns). Pf2 uses the rb_pro fi le_thread_frames() API for sampling. When to do so is controlled by Schedulers, described in the following section. Schedulers Schedulers determine when to execute sample collection, based on con fi guration (time mode and interval). Pf2 has two schedulers available. SignalScheduler (Linux-only) The fi rst is the SignalScheduler, based on POSIX timers. Pf2 will use this scheduler when possible. SignalScheduler creates a POSIX timer for each Ruby Thread (the underlying pthread to be more accurate) using timer_create(3). This leaves the actual time-keeping to the OS, which is capable of tracking accurate per-thread CPU time usage. When the speci fi ed interval has arrived (the timer has expired), the OS delivers us a SIGALRM (note: Unlike setitimer(2), timer_create(3) allows us to choose which signal to be delivered, and Pf2 uses SIGALRM regardless of time mode). This is why the scheduler is named SignalScheduler. Signals are directed to Ruby Threads' underlying pthread, effectively "pausing" the Thread's activity. This routing is done using SIGEV_THREAD_ID, which is a Linux-only feature. Sample collection is done in the signal handler, which is expected to be more accurate, capturing the paused Thread's activity. This scheduler heavily relies on Ruby's 1:N Thread model (1 Ruby Threads is strongly tied to a native pthread). It will not work properly in MaNy (RUBY_MN_THREADS=1). TimerThreadScheduler Another scheduler is the TimerThreadScheduler, which maintains a time-keeping thread by itself. A new native thread (pthread on Linux/macOS) will be created, and an in fi nite loop will be run inside. After sleep(2)-ing for the speci fi ed interval time, sampling will be queued using Ruby's Postponed Job API. This scheduler is wall-time only, and does not support CPU-time based pro fi ling.
  5. 70

  6. • ࢀՃऀ਺ 1400 • RubyKaigi 2023ͷࢀՃऀ਺ • ηογϣϯ਺ 52 •

    ͏ͪɺϓϩϑΝΠϥؔ࿈ηογϣϯ͸…… ਺ࣈͰݟΔ RubyKaigi 2024 ࠜڌΛ͝આ໌͠·͠ΐ͏
  7. The depths of profiling Ruby Vernier: A next generation profiler

    for Ruby Optimizing Ruby: Building an Always-On Production Profiler ϓϩϑΝΠϥτʔΫ͕3ͭ (5%)
  8. ϓϩϑΝΠϥτʔΫ׬શղઆ • The depths of pro fi ling Ruby (osyoyu)

    • ৽ϓϩϑΝΠϥ Pf2 ͷ঺հ • Vernier: A next generation pro fi ler for Ruby (jhawthorn) • ৽ϓϩϑΝΠϥ Vernier ͷ঺հ • Optimizing Ruby: Building an Always-On Production Pro fi ler (ivoanjo) • Datadog ૊ΈࠐΈͷϓϩϑΝΠϥͷ঺հ
  9. Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ 2009 2013 TracePoint API

    ͷొ৔ (Ruby) Ruby ࣮ߦதͷ "Πϕϯτ" (ϝιουͷݺͼग़͠ɺϦλʔϯɺ...) ΛऔಘͰ͖ΔAPI ruby-prof ͸͜Εϕʔε
  10. Ruby ʹ DTrace αϙʔτ͕ೖΔ ϓϩάϥϜʹ probe ΛຒΊࠐΉ͜ͱͰɺ͋ΔߦΛ࣮ߦͨ͠ճ਺ͳͲΛऔΕΔศརͳ΍ͭ 2013 Linux 2.6

    Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ 2009 2013 2013
  11. 2014 rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε "ݱࡏ࣮ߦதͷϝιου" Λฦ͢ C

    API ͕ Ruby 2.1.0 ʹ௥Ճɺ ͦΕΛ࢖ͬͨϓϩϑΝΠϥ Stackprof ͷొ৔ Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ 2009 2013 2013 2013
  12. eBPF ͕࣮༻తʹͳͬͯ͘Δ Linux 4.1 ͋ͨΓͰ eBPF ͷػೳ͕͔ͳΓॆ࣮ͯ͘͠Δ ΧʔωϧϥϯυͰಈ࡞͢ΔϓϩϑΝΠϥΛ࡞Δ͜ͱ͕ݱ࣮తʹ ΧʔωϧϥϯυͰಈ࡞͢Δ =

    Φʔόʔϔου͕খ͍͞ɻܹ೤ʂ 2015 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε 2009 2013 2013 2013 2014
  13. Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ ࢭΊΒΕͳ͍ίϯςφԽͷྲྀΕɺ ΋͏গ͠ޙͷ࣌୅ͷ࿩͕ͩɺkubectl plugin ͱͯ͠ಈ࡞Ͱ͖Δ profiler

    धཁ΋ߴ·Δ 2016 - 2018? Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ 2009 2013 2013 2013 2014 2015
  14. rbspy ͷϦϦʔε Linux ͷ process_vm_readv(2) API Λ࢖ͬͨϓϩϑΝΠϥ ruby ϓϩηεͷ֎͔ΒϝϞϦΛͷ͖ͧݟΔ͜ͱͰಈ࡞͢Δ৽͍͠ΞΠσΞ 2018

    Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ 2009 2013 2013 2013 2014 2015 2016-2018?
  15. Continuous Profiling ֓೦ͷొ৔ ʮௐࠪͷҰ؀Ͱ profiler Λࠩ͠ࠐΉʯ͔ΒʮͣͬͱϓϩϑΝΠϧதʯͷ࣌୅΁ Datadog ΋ Continuous Profiler

    ػೳΛϦϦʔε Observability (o11y) ͷจ຺ͱͷؔ࿈Ͱ΋஫໨ 2020 - 2022? Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε 2009 2013 2013 2013 2014 2015 2016-2019? 2018
  16. Observability ֓೦ͷྲྀߦ ௨শ o11y ͳΜ͔͜͏…… ෳࡶͳγεςϜશମΛͪΌΜͱ೺ѲͰ͖ΔΑ͏ʹ͠Α͏ʂ తͳ΍ͭ 2021- 2023? Linux

    2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ 2009 2013 2013 2013 2014 2015 2016-2019? 2018 2020-2022?
  17. rb_profile_thread_frames() API ͷొ৔ rb_profile_frames() API ͷ֦ுɻελοΫΛऔಘ͢Δ Ruby Thread ΛࢦఆͰ͖Δ͜ͱͰ ΑΓਫ਼ࡉͳϓϩϑΝΠϦϯά͕Մೳʹ

    2023 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ Observability ֓೦ͷྲྀߦ 2009 2013 2013 2013 2014 2015 2016-2019? 2018 2020-2022? 2021-2023?
  18. Pf2, Vernier ͷϦϦʔε ͓ͦΒ྆͘ํ2023? ࠓ·Ͱʹͳ͔ͬͨػೳΛ࣮૷ͨ͠ϓϩϑΝΠϥ͕Ұؾʹొ৔ 2023 2009 2013 2013 2013

    2014 2015 2016-2019? 2018 2020-2022? 2021-2023? 2023 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ Observability ֓೦ͷྲྀߦ rb_profile_thread_frames() API ͷొ৔
  19. Pf2, Vernier ͷϦϦʔε ͓ͦΒ྆͘ํ2023? ࠓ·Ͱʹͳ͔ͬͨػೳΛ࣮૷ͨ͠ϓϩϑΝΠϥ͕Ұؾʹొ৔ 2023 2009 2013 2013 2013

    2014 2015 2016-2019? 2018 2020-2022? 2021-2023? 2023 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ Observability ֓೦ͷྲྀߦ rb_profile_thread_frames() API ͷొ৔ ϏδϡΞϥΠβ΋ٸ଎ʹॆ࣮͖ͯͨ͠ Speedscope, Firefox Pro fi ler, Perfetto, Chrome Pro fi ler, ...
  20. 2024೥ɺϓϩϑΝΠϥ͕೤͍ • Continuous Pro fi ling Λ͸͡Ίͱͨ͠ "धཁ" ͕ߴ·͍ͬͯΔ •

    Linux / Ruby ΁ͷ API ௥ՃΛ௨ͯ͡ "ڙڅ" ΋ߴ·͍ͬͯΔ ೾ʹ৐Γ஗ΕΔͳʂʂʂʂ