[RailsConf 2024] From slow to go: Rails profiling hands-on

by Vladimir Dementyev

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

From slow to go: Rails test profiling hands-on Vladimir Dementyev Evil Martians

Slide 3

Slide 3 text

palkan_tula palkan Why are we here? What's the plan? Who are you? 3 Warming up Or waiting for everyone's dip rspec to pass

Slide 4

Slide 4 text

palkan_tula palkan You are here = = your tests are slow 4

Slide 5

Slide 5 text

palkan_tula palkan Longer CI builds (also more expensive) Longer local runs (or "local CI" builds) 5 Slow tests

Slide 6

Slide 6 text

palkan_tula palkan Longer CI builds (also more expensive) Longer local runs (or "local CI" builds) Longer feedback loops More distraction, less productivity 6 Slow tests

Slide 7

Slide 7 text

palkan_tula palkan 🔍 Identify slowness causes and patterns 💊 Treat them well 🔁 Repeat until satisfied 7 From slow to go

Slide 8

Slide 8 text

palkan_tula palkan 8

Slide 9

Slide 9 text

palkan_tula palkan Specialized profiler for Ruby test suites Optimization toolbox to speed up tests with less refactoring 9 TestProf test-prof.evilmartians.io

Slide 10

Slide 10 text

palkan_tula palkan Used by GitHub, Discourse, Gitlab, Dev.to, and many more Used by YOU at this workshop! 10 TestProf test-prof.evilmartians.io

Slide 11

Slide 11 text

11 github.com/palkan

Slide 12

Slide 12 text

12 evilmartians.com

Slide 13

Slide 13 text

Slide 14

Slide 14 text

palkan_tula palkan 14 X

Slide 15

Slide 15 text

palkan_tula palkan 15 Disclaimer: Mastodon team already did some test performance work (via TestProf), so I had to revert a few optimizations in the workshop's version of the codebase (so we can have more fun)

Slide 16

Slide 16 text

palkan_tula palkan General profiling (Stackprof, Vernier, sampling) Focused profiling (TagProf, EventProf) Factories overhead elimination (let_it_be) 16 The plan

Slide 17

Slide 17 text

palkan_tula palkan Local time: 8 : 30 ➡ 2 : 30 CI time: 14 : 30 ➡ 8 : 00 17 Results

Slide 18

Slide 18 text

palkan_tula palkan Rails tests occupy only for a portion of a CI build time No need to optimize beyond other components 18 CI vs Rails tests

Slide 19

Slide 19 text

palkan_tula palkan General profiling 19

Slide 20

Slide 20 text

Slide 21

Slide 21 text

palkan_tula palkan Workshop instructions 21 # rails_helper.rb TestProf::EventProf.monitor( Paperclip::Attachment, 'paperclip.post_process', :post_process )

Slide 22

Slide 22 text

palkan_tula palkan Workshop instructions 22 TAG_PROF=type \ TAG_PROF_FORMAT=html \ TAG_PROF_EVENT=sql.active_record,factory.create, sidekiq.inline,paperclip.post_process \ be rspec

Slide 23

Slide 23 text

palkan_tula palkan Coverage: opt-out ➡ opt-in gem "debug", require: false Webpacker: cache_manifest: true 23 Findings

Slide 24

Slide 24 text

palkan_tula palkan Logging—no effect when disk write is fast (SSD, Docker volumes on Mac) Thousands of I18n YAMLs—handled by Bootsnap 24 Findings

Slide 25

Slide 25 text

Slide 26

Slide 26 text

palkan_tula palkan Do not require test-specific profiling skills Takes the most significant amount of time (sampling helps) Highlights low-hanging fruits 26 General profiling

Slide 27

Slide 27 text

palkan_tula palkan Focused profiling 27

Slide 28

Slide 28 text

palkan_tula palkan Similar tests has similar problems Focus on most time consuming test types, not individual outliers 28 Focused profiling

Slide 29

Slide 29 text

Slide 30

Slide 30 text

palkan_tula palkan Reduced the number of tests to analyze by ~2x Divided tests into smaller groups (~1min run time) for further analysis 30 Focused profiling

Slide 31

Slide 31 text

palkan_tula palkan TagProf EventProf RspecStamp (RSTAMP) 31 Tools

Slide 32

Slide 32 text

palkan_tula palkan Sidekiq: inline! ➡ fake! Paperclip::Testing.fake! 32 Findings

Slide 33

Slide 33 text

Slide 34

Slide 34 text

palkan_tula palkan Workshop instructions 34 # .rspec-local --tag ~paperclip:process

Slide 35

Slide 35 text

palkan_tula palkan 35 What's next?

Slide 36

Slide 36 text

palkan_tula palkan Factories 36

Slide 37

Slide 37 text

palkan_tula palkan 37 FPROF = 1 \ EVENT_PROF=factory.create \ be rspec spec/models Workshop instructions

Slide 38

Slide 38 text

palkan_tula palkan Factory cascades Too many disposable records (no reuse or recycle) Create vs. build 38 Factories vs. slow tests

Slide 39

Slide 39 text

Slide 40

Slide 40 text

palkan_tula palkan FactoryProf RSpecDissect before_all / let_it_be 40 Tools

Slide 41

Slide 41 text

palkan_tula palkan No cascades found Many tests share the same context Reusing records between examples brings up to 6x speed up! 41 Findings

Slide 42

Slide 42 text

palkan_tula palkan Replacing let/let! with let_it_be works in ~90% of tests The rest may require some refactoring (usually minor) 42 Findings

Slide 43

Slide 43 text

palkan_tula palkan Every file must be treated individually (routine) Extracting shared contexts may help simplify future optimizations (e.g., adding fixtures) 43 Findings

Slide 44

Slide 44 text

palkan_tula palkan Homework 44

Slide 45

Slide 45 text

palkan_tula palkan Discover more TestProf tools (Autopilot, RuboCop cops, Factory Doctor) Make YOUR tests faster Become a Mastodon contributor! 45 Homework

Slide 46

Slide 46 text

Thank you Visit TestProf and Evil Martians at the Hack Day!