[RailsConf 2024] From slow to go: Rails profiling hands-on

https://evilmartians.com/events/from-slow-to-go-rails-test-profiling-hands-on-railsconf-2024

Transcript

2. From slow to go: Rails test profiling hands-on Vladimir Dementyev Evil Martians
3. palkan_tula palkan Why are we here? What's the plan? Who are you? 3 Warming up Or waiting for everyone's dip rspec to pass
4. palkan_tula palkan You are here = = your tests are slow 4
5. palkan_tula palkan Longer CI builds (also more expensive) Longer local runs (or "local CI" builds) 5 Slow tests
6. palkan_tula palkan Longer CI builds (also more expensive) Longer local runs (or "local CI" builds) Longer feedback loops More distraction, less productivity 6 Slow tests
7. palkan_tula palkan 🔍 Identify slowness causes and patterns 💊 Treat them well 🔁 Repeat until satisfied 7 From slow to go
8. palkan_tula palkan 8
9. palkan_tula palkan Specialized profiler for Ruby test suites Optimization toolbox to speed up tests with less refactoring 9 TestProf test-prof.evilmartians.io
10. palkan_tula palkan Used by GitHub, Discourse, Gitlab, Dev.to, and many more Used by YOU at this workshop! 10 TestProf test-prof.evilmartians.io
11. 11 github.com/palkan
12. 12 evilmartians.com
14. palkan_tula palkan 14 X
15. palkan_tula palkan 15 Disclaimer: Mastodon team already did some test performance work (via TestProf), so I had to revert a few optimizations in the workshop's version of the codebase (so we can have more fun)
16. palkan_tula palkan General profiling (Stackprof, Vernier, sampling) Focused profiling (TagProf, EventProf) Factories overhead elimination (let_it_be) 16 The plan
17. palkan_tula palkan Local time: 8 : 30 ➡ 2 : 30 CI time: 14 : 30 ➡ 8 : 00 17 Results
18. palkan_tula palkan Rails tests occupy only for a portion of a CI build time No need to optimize beyond other components 18 CI vs Rails tests
19. palkan_tula palkan General profiling 19
21. palkan_tula palkan Workshop instructions 21 # rails_helper.rb TestProf::EventProf.monitor( Paperclip::Attachment, 'paperclip.post_process', :post_process )
22. palkan_tula palkan Workshop instructions 22 TAG_PROF=type \ TAG_PROF_FORMAT=html \ TAG_PROF_EVENT=sql.active_record,factory.create, sidekiq.inline,paperclip.post_process \ be rspec
23. palkan_tula palkan Coverage: opt-out ➡ opt-in gem "debug", require: false Webpacker: cache_manifest: true 23 Findings
24. palkan_tula palkan Logging—no effect when disk write is fast (SSD, Docker volumes on Mac) Thousands of I18n YAMLs—handled by Bootsnap 24 Findings
26. palkan_tula palkan Do not require test-specific profiling skills Takes the most significant amount of time (sampling helps) Highlights low-hanging fruits 26 General profiling
27. palkan_tula palkan Focused profiling 27
28. palkan_tula palkan Similar tests has similar problems Focus on most time consuming test types, not individual outliers 28 Focused profiling
30. palkan_tula palkan Reduced the number of tests to analyze by ~2x Divided tests into smaller groups (~1min run time) for further analysis 30 Focused profiling
31. palkan_tula palkan TagProf EventProf RspecStamp (RSTAMP) 31 Tools
32. palkan_tula palkan Sidekiq: inline! ➡ fake! Paperclip::Testing.fake! 32 Findings
34. palkan_tula palkan Workshop instructions 34 # .rspec-local --tag ~paperclip:process
35. palkan_tula palkan 35 What's next?
36. palkan_tula palkan Factories 36
37. palkan_tula palkan 37 FPROF = 1 \ EVENT_PROF=factory.create \ be rspec spec/models Workshop instructions
38. palkan_tula palkan Factory cascades Too many disposable records (no reuse or recycle) Create vs. build 38 Factories vs. slow tests
40. palkan_tula palkan FactoryProf RSpecDissect before_all / let_it_be 40 Tools
41. palkan_tula palkan No cascades found Many tests share the same context Reusing records between examples brings up to 6x speed up! 41 Findings
42. palkan_tula palkan Replacing let/let! with let_it_be works in ~90% of tests The rest may require some refactoring (usually minor) 42 Findings
43. palkan_tula palkan Every file must be treated individually (routine) Extracting shared contexts may help simplify future optimizations (e.g., adding fixtures) 43 Findings
44. palkan_tula palkan Homework 44
45. palkan_tula palkan Discover more TestProf tools (Autopilot, RuboCop cops, Factory Doctor) Make YOUR tests faster Become a Mastodon contributor! 45 Homework
46. Thank you Visit TestProf and Evil Martians at the Hack Day!