今年のRubyKaigiはProfiler Year🤘

Transcript

1. @osyoyu | 2024/4/25 RubyKaigi 2024ࣄલษڧձ ࠓ೥ͷRubyKaigi͸Profiler Year🤘
2.  p @osyoyu • osyoyu (͓͠ΐʔΏ) • VISA ϓϦΧʮB/43ʯΛεϚʔτόϯΫ (ג) Ͱ࡞ͬͯ·͢
3. pp @osyoyu ʮ͏ͳ͹ΜʂʯϑΝʔετϥΠϒʂ ࠓि຤ 4/27 (౔) ໷ @ େԬࢁ όϯυϝϯόʔશһ Rubyist & RubyKaigi ࢀՃ ৄࡉ͸ "unasuke όϯυ" Ͱݕࡧʢunasuke = όϯυϝϯόʔʣ
4. ࠓ೥ͷRubyKaigi͸Profiler Year🤘
5. ϓϩϑΝΠϥ Stackprof, ruby-prof, rbspy, ... • ೔ຊޠͰ͸ʮੑೳղੳπʔϧʯ • ϓϩάϥϜதͷ "஗͍" ՕॴΛൃݟͰ͖Δ • ISUCON ͳͲͰ͓ੈ࿩ʹͳͬͨਓ΋ଟ͍ͷͰ͸ͳ͍Ͱ͠ΐ͏͔
6. Pf2 ࠷ڧศརϓϩϑΝΠϥΛ࡞͍ͬͯ·͢ • ৽͍͠ϓϩϑΝΠϥΛ࡞ͬͯ·͢ʂ • RubyKaigi Ͱ͸ Pf2 ͷ࢓૊Έ΍ɺ࡞Δ্Ͱͷ೉͠ ͞ͷ࿩Λ͠·͢ • Pf2 ͷ࢖͍͔ͨͷ࿩͸ࡢ೔ (4/23) ͷ Gotanda.rb #58 ͷࢿྉΛݟ͍ͯͩ͘͞ github.com/osyoyu/pf2
7. ͜ͷίʔυͷ஗͍৔ॴɺΘ͔Γ·͔͢ 10,000,000,000ߦͷςΩετϑΝΠϧͷॲཧ "Roppongi;24.0" ͷΑ͏ͳ "౎ࢢ໊;ؾԹ" Λ10ԯߦಡΈࠐΈ·͢ ࡉ͔͍͜ͱ͸ The One Billion Row Challenge Ͱάά͍ͬͯͩ͘͞
8. Flamegraph ΛݟͯΈΑ͏ ԣ෯͕௕͍ϝιου͕࣮ߦ͕࣌ؒ௕͍΍ͭ
9. Flamegraph ΛݟͯΈΑ͏ ԣ෯͕௕͍ϝιου͕࣮ߦ͕࣌ؒ௕͍΍ͭ 👮
10. Pf2 ͳΒ Ruby ͷਅͷ࢟Λ๫͚Δ Ruby ͸ C Ͱ࡞ΒΕͯ·͢ rb_hash_aref = Hash#[]
11. Pf2 ͷ࢓૊Έ ΈΜͳڵຯ͋ΔΑͶ Sampling Pf2 is a sampling pro fi ler. This means that Pf2 collects samples of program execution periodically, instead of tracing every action (e.g. method invocations and returns). Pf2 uses the rb_pro fi le_thread_frames() API for sampling. When to do so is controlled by Schedulers, described in the following section. Schedulers Schedulers determine when to execute sample collection, based on con fi guration (time mode and interval). Pf2 has two schedulers available. SignalScheduler (Linux-only) The fi rst is the SignalScheduler, based on POSIX timers. Pf2 will use this scheduler when possible. SignalScheduler creates a POSIX timer for each Ruby Thread (the underlying pthread to be more accurate) using timer_create(3). This leaves the actual time-keeping to the OS, which is capable of tracking accurate per-thread CPU time usage. When the speci fi ed interval has arrived (the timer has expired), the OS delivers us a SIGALRM (note: Unlike setitimer(2), timer_create(3) allows us to choose which signal to be delivered, and Pf2 uses SIGALRM regardless of time mode). This is why the scheduler is named SignalScheduler. Signals are directed to Ruby Threads' underlying pthread, effectively "pausing" the Thread's activity. This routing is done using SIGEV_THREAD_ID, which is a Linux-only feature. Sample collection is done in the signal handler, which is expected to be more accurate, capturing the paused Thread's activity. This scheduler heavily relies on Ruby's 1:N Thread model (1 Ruby Threads is strongly tied to a native pthread). It will not work properly in MaNy (RUBY_MN_THREADS=1). TimerThreadScheduler Another scheduler is the TimerThreadScheduler, which maintains a time-keeping thread by itself. A new native thread (pthread on Linux/macOS) will be created, and an in fi nite loop will be run inside. After sleep(2)-ing for the speci fi ed interval time, sampling will be queued using Ruby's Postponed Job API. This scheduler is wall-time only, and does not support CPU-time based pro fi ling.
13. Day 1 14:50- དྷͯ͘Εʂʂʂ
14. ຊ೔
15. ࠓ೥ͷRubyKaigi͸Profiler Year✌
16. 2022, 2023: ύʔαʔ 2024: ϓϩϑΝΠϥ
17. ❌ ϓϩϑΝΠϥͷ࢖͍͔ͨ ʢΠϯλʔωοτʹॻ͍ͱ͖·ͨ͠ʣ ʢRubyKaigi Ͱฉ͖ʹདྷ͍ͯͩ͘͞ʣ ❌ Pf2 ͷৄ͍͠࿩
18. ⭕ ͍͔ʹࠓ೥͕ Profiler Year Ͱ͋Δ͔
21. 70 RubyKaigi 2024 ࢀՃऀͰɺϓϩϑΝΠϥΛ࡞͍ͬͯΔਓ
22. • ࢀՃऀ਺ 1400 • RubyKaigi 2023ͷࢀՃऀ਺ • ηογϣϯ਺ 52 • ͏ͪɺϓϩϑΝΠϥؔ࿈ηογϣϯ͸…… ਺ࣈͰݟΔ RubyKaigi 2024 ࠜڌΛ͝આ໌͠·͠ΐ͏
26. The depths of profiling Ruby Vernier: A next generation profiler for Ruby Optimizing Ruby: Building an Always-On Production Profiler ϓϩϑΝΠϥτʔΫ͕3ͭ (5%)
27. 1400×(3/52) = RubyKaigi ࢀՃऀ਺ ϓϩϑΝΠϥτʔΫ཰
28. 1400×(3/52) = RubyKaigi ࢀՃऀ਺ ϓϩϑΝΠϥτʔΫ཰ ϓϩϑΝΠϥΛ࡞ͬͯΔࢀՃऀͷ਺ 70
29. ϓϩϑΝΠϥτʔΫ׬શղઆ • The depths of pro fi ling Ruby (osyoyu) • ৽ϓϩϑΝΠϥ Pf2 ͷ঺հ • Vernier: A next generation pro fi ler for Ruby (jhawthorn) • ৽ϓϩϑΝΠϥ Vernier ͷ঺հ • Optimizing Ruby: Building an Always-On Production Pro fi ler (ivoanjo) • Datadog ૊ΈࠐΈͷϓϩϑΝΠϥͷ঺հ
33. ? ͜ΜͳʹྲྀߦΔʁ
34. धཁͱڙڅ͕Ϛονͨ݁͠Ռ ೤ڰ͕ੜ·Ε͍ͯΔ
35. धཁͱڙڅΛಡΈղ͘ ಡΈղ͘ͱݴ͑͹ྺ࢙ धཁ = ϓϩϑΝΠϥʹͰ͖ͯ΄͍͜͠ͱ ڙڅ = ϓϩϑΝΠϥ͕Ͱ͖Δ͜ͱ
37. Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ CPU ͷύϑΥʔϚϯεΧ΢ϯλ΋౷߹ͨ͠ڧྗͳϓϩϑΝΠϦϯά͕࣮ݱ ࠓͰ΋όϦόϦݱ໾ʂ ruby/ruby ͷϓϩϑΝΠϦϯάΛ͢Δͱ͖ʹ΋ศར 2009
38. Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ 2009 2013 TracePoint API ͷొ৔ (Ruby) Ruby ࣮ߦதͷ "Πϕϯτ" (ϝιουͷݺͼग़͠ɺϦλʔϯɺ...) ΛऔಘͰ͖ΔAPI ruby-prof ͸͜Εϕʔε
39. Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ ໊ஶ ύϑΥʔϚϯεքͷυϥΰϯϒοΫ େ෯Ξοϓσʔτ͞Εͨୈ2൛ΛങͬͯಡΜͰ͍ͩ͘͞ 2013 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ 2009 2013
40. Ruby ʹ DTrace αϙʔτ͕ೖΔ ϓϩάϥϜʹ probe ΛຒΊࠐΉ͜ͱͰɺ͋ΔߦΛ࣮ߦͨ͠ճ਺ͳͲΛऔΕΔศརͳ΍ͭ 2013 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ 2009 2013 2013
41. 2014 rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε "ݱࡏ࣮ߦதͷϝιου" Λฦ͢ C API ͕ Ruby 2.1.0 ʹ௥Ճɺ ͦΕΛ࢖ͬͨϓϩϑΝΠϥ Stackprof ͷొ৔ Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ 2009 2013 2013 2013
42. eBPF ͕࣮༻తʹͳͬͯ͘Δ Linux 4.1 ͋ͨΓͰ eBPF ͷػೳ͕͔ͳΓॆ࣮ͯ͘͠Δ ΧʔωϧϥϯυͰಈ࡞͢ΔϓϩϑΝΠϥΛ࡞Δ͜ͱ͕ݱ࣮తʹ ΧʔωϧϥϯυͰಈ࡞͢Δ = Φʔόʔϔου͕খ͍͞ɻܹ೤ʂ 2015 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε 2009 2013 2013 2013 2014
43. Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ ࢭΊΒΕͳ͍ίϯςφԽͷྲྀΕɺ ΋͏গ͠ޙͷ࣌୅ͷ࿩͕ͩɺkubectl plugin ͱͯ͠ಈ࡞Ͱ͖Δ profiler धཁ΋ߴ·Δ 2016 - 2018? Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ 2009 2013 2013 2013 2014 2015
44. rbspy ͷϦϦʔε Linux ͷ process_vm_readv(2) API Λ࢖ͬͨϓϩϑΝΠϥ ruby ϓϩηεͷ֎͔ΒϝϞϦΛͷ͖ͧݟΔ͜ͱͰಈ࡞͢Δ৽͍͠ΞΠσΞ 2018 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ 2009 2013 2013 2013 2014 2015 2016-2018?
45. Continuous Profiling ֓೦ͷొ৔ ʮௐࠪͷҰ؀Ͱ profiler Λࠩ͠ࠐΉʯ͔ΒʮͣͬͱϓϩϑΝΠϧதʯͷ࣌୅΁ Datadog ΋ Continuous Profiler ػೳΛϦϦʔε Observability (o11y) ͷจ຺ͱͷؔ࿈Ͱ΋஫໨ 2020 - 2022? Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε 2009 2013 2013 2013 2014 2015 2016-2019? 2018
46. Observability ֓೦ͷྲྀߦ ௨শ o11y ͳΜ͔͜͏…… ෳࡶͳγεςϜશମΛͪΌΜͱ೺ѲͰ͖ΔΑ͏ʹ͠Α͏ʂ తͳ΍ͭ 2021- 2023? Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ 2009 2013 2013 2013 2014 2015 2016-2019? 2018 2020-2022?
47. rb_profile_thread_frames() API ͷొ৔ rb_profile_frames() API ͷ֦ுɻελοΫΛऔಘ͢Δ Ruby Thread ΛࢦఆͰ͖Δ͜ͱͰ ΑΓਫ਼ࡉͳϓϩϑΝΠϦϯά͕Մೳʹ 2023 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ Observability ֓೦ͷྲྀߦ 2009 2013 2013 2013 2014 2015 2016-2019? 2018 2020-2022? 2021-2023?
48. Pf2, Vernier ͷϦϦʔε ͓ͦΒ྆͘ํ2023? ࠓ·Ͱʹͳ͔ͬͨػೳΛ࣮૷ͨ͠ϓϩϑΝΠϥ͕Ұؾʹొ৔ 2023 2009 2013 2013 2013 2014 2015 2016-2019? 2018 2020-2022? 2021-2023? 2023 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ Observability ֓೦ͷྲྀߦ rb_profile_thread_frames() API ͷొ৔
49. Pf2, Vernier ͷϦϦʔε ͓ͦΒ྆͘ํ2023? ࠓ·Ͱʹͳ͔ͬͨػೳΛ࣮૷ͨ͠ϓϩϑΝΠϥ͕Ұؾʹొ৔ 2023 2009 2013 2013 2013 2014 2015 2016-2019? 2018 2020-2022? 2021-2023? 2023 Linux 2.6 Ͱ perf_events (perf) ͕ొ৔ TracePoint API ͷొ৔ Brendan Gregg ઌੜͷ sysperf ຊ͕ग़Δ Ruby ʹ DTrace αϙʔτ͕ೖΔ rb_profile_frames() API ͷొ৔ & Stackprof ͷϦϦʔε eBPF ͕࣮༻తʹͳͬͯ͘Δ Kubernetes ͷຄڵ, out-of-process profiling ΁ͷػӡ rbspy ͷϦϦʔε Continuous Profiling ֓೦ͷొ৔ Observability ֓೦ͷྲྀߦ rb_profile_thread_frames() API ͷొ৔ ϏδϡΞϥΠβ΋ٸ଎ʹॆ࣮͖ͯͨ͠ Speedscope, Firefox Pro fi ler, Perfetto, Chrome Pro fi ler, ...
50. 2024೥ɺϓϩϑΝΠϥ͕೤͍ • Continuous Pro fi ling Λ͸͡Ίͱͨ͠ "धཁ" ͕ߴ·͍ͬͯΔ • Linux / Ruby ΁ͷ API ௥ՃΛ௨ͯ͡ "ڙڅ" ΋ߴ·͍ͬͯΔ ೾ʹ৐Γ஗ΕΔͳʂʂʂʂ
51. ࠓ೥ͷRubyKaigi͸Profiler Year✌
52. Day 1 14:50- དྷͯ͘Εʂʂʂ