Experimenting in Production

Together, we explore experimenting in production with Ruby on Rails. We learn about various techniques and how the `scientist` and `lab_tech` Ruby gems can help us achieve safe and scientifically-driven experimentation.

Transcript

1. Experimenting in Production @garyhtou 🔬
2. 🏦 class BankAccount < ApplicationRecord has_many :transactions def balance transactions.reduce 0 do |balance, transaction| balance + transaction.amount end end end class BankAccount < ApplicationRecord has_many :transactions def balance transactions.sum(:amount) end end @garyhtou
3. class BankAccount < ApplicationRecord has_many :transactions def balance # this might give inaccurate results 💸 new_balance # calculation using `sum` end end class BankAccount < ApplicationRecord has_many :transactions def balance old_balance # calculation using `reduce` end end @garyhtou
4. class BankAccount < ApplicationRecord has_many :transactions def balance if rand < 0.5 old_balance else new_balance end end end class BankAccount < ApplicationRecord has_many :transactions def balance old_balance # calculation using `reduce` end end A/B Testing @garyhtou
5. Why not run both? class BankAccount < ApplicationRecord has_many :transactions def balance old_result = old_balance # calculation using `reduce` new_result = new_balance # calculation using `sum` if new_result != old_result puts "oh no! new method is wrong! 😰" end old_result end end @garyhtou
6. 📊 Compare the old and new results 📒 Test odd edge cases with production data ✅ Return accurate results 🐌 CON: Decrease in performance Why you should run both
7. github/scientist 🔬 A Ruby library for carefully refactoring critical paths RealGeeks/lab_tech Rails engine for using GitHub's 'Scientist' library with ActiveRecord + @garyhtou
8. Scientist + Lab Tech class BankAccount < ApplicationRecord has_many :transactions def balance LabTech.science "my_balance_experiment" do |exp| exp.use { old_balance } exp.try { new_balance } end end end @garyhtou
9. Experimenting in Production Scientist + Lab Tech Gary Tou @garyhtou garytou.com