Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Death By a Thousand Commits

Death By a Thousand Commits

On the 1st commit, things are getting started. On the 10th commit, the feature is live and users are giving feedback. On the 100th commit, users are delighted to be using the application. But on the 1000th commit, users are unhappy with the responsiveness of the application and the developers are struggling to move at the velocity they once were. Does this sound familiar?

We will go over some of the pieces of technical debt that can accumulate and can cause application performance and development velocity issues, and the strategies Clio uses to keep these kinds of technical debt under control.

06841bbcde7df91dc306d22ea0293bf9?s=128

Kyle d'Oliveira

May 02, 2019
Tweet

Transcript

  1. Death By a Thousand Commits Kyle d’Oliveira RailsConf 2019

  2. About me Kyle d’Oliveira https://www.linkedin.com/in/doliveirakn https://github.com/doliveirakn

  3. None
  4. Technical Debt

  5. Technical Debt

  6. Technical Debt

  7. Technical Debt Quadrants – Martin Fowler Reckless Prudent Deliberate Inadvertent

  8. Technical Debt Quadrants – Martin Fowler Reckless Prudent Deliberate Inadvertent

    “What’s layering”
  9. Technical Debt Quadrants – Martin Fowler Reckless Prudent Deliberate “We

    don’t have time for design” Inadvertent “What’s layering”
  10. Technical Debt Quadrants – Martin Fowler Reckless Prudent Deliberate “We

    don’t have time for design” “We must ship now and deal with consequences” Inadvertent “What’s layering”
  11. Technical Debt Quadrants – Martin Fowler Reckless Prudent Deliberate “We

    don’t have time for design” “We must ship now and deal with consequences” Inadvertent “What’s layering” “Now we know how we should have done it”
  12. Paying down technical debt ▪ Deal with the consequences

  13. Paying down technical debt ▪ Deal with the consequences ▪

    Be proactive
  14. Paying down technical debt ▪ Deal with the consequences ▪

    Be proactive ▪ Invest in tools
  15. Lessons ▪ Lesson 1 – Automate the debt away ▪

    Lesson 2 – Clean up code you don't use ▪ Lesson 3 – Make keeping the lights on easier ▪ Lesson 4 – Keep the bad patterns out
  16. Automate the debt away Lesson 1

  17. N+1 Queries The silent performance tax

  18. SELECT * FROM contacts SELECT * FROM emails WHERE contact_id

    = 1 SELECT * FROM emails WHERE contact_id = 2 SELECT * FROM emails WHERE contact_id = 3 SELECT * FROM emails WHERE contact_id = 4 SELECT * FROM emails WHERE contact_id = 5 SELECT * FROM emails WHERE contact_id = 6
  19. SELECT * FROM contacts SELECT * FROM emails WHERE contact_id

    IN (1, 2, 3, 4, 5, 6)
  20. class ContactsController < ApplicationController def index render json: Contact.limit(200) end

    end
  21. class ContactSerializer < ActiveModel::Serializer attribute :id, :name end

  22. [ { “id”:1, “name”: “Jordan” }, { “id”:2, “name”: “Erin”

    }, { “id”:3, “name”: “Anna” }, ]
  23. class ContactSerializer < ActiveModel::Serializer attribute :id, :name has_one :email end

  24. class ContactSerializer < ActiveModel::Serializer attribute :id, :name has_one :email end

  25. class ContactSerializer < ActiveModel::Serializer attribute :id, :name has_one :email has_one

    :phone_number has_one :address has_one :emergency_contact end
  26. class ContactsController < ApplicationController def index render json: Contact.includes(:email).limit(200) end

    end
  27. Limitations ▪ Manual human effort

  28. Limitations ▪ Manual human effort ▪ Only fixes one instance

    at a time
  29. Limitations ▪ Manual human effort ▪ Only fixes one instance

    at a time ▪ Doesn’t handle associations no longer being needed
  30. Automate the debt away Lesson 1 Tools

  31. jit_preloader https://github.com/clio/jit_preloader

  32. # Gemfile gem “jit_preloader” # config/initializers/jit_preloader.rb JitPreloader.globally_enable = true

  33. None
  34. None
  35. 95 percentile with N+1 queries 95 percentile without N+1 queries

  36. 99 percentile with N+1 queries 99 percentile without N+1 queries

  37. Automate the debt away Lesson 1

  38. Clean up code you don't use Lesson 2

  39. None
  40. None
  41. Clean up code you don't use Lesson 2 Tools

  42. TombstoneError = Class.new(StandardError) def Tombstone(date_of_death) exception = TombstoneError.new( “Code was

    laid to rest on #{date_of_death}” ) Bugsnag.notify(exception) end
  43. class UsersController def index Tombstone(“2019-05-02”) # Original Behaviour end end

  44. None
  45. dead_code_detector https://github.com/clio/dead_code_detector

  46. coverband https://github.com/danmayer/coverband

  47. # Gemfile gem “dead_code_detector”

  48. Deleted so far: 300 controller actions 1000 methods

  49. Clean up code you don't use Lesson 2

  50. Make it easier to keep the lights on Lesson 3

  51. None
  52. SELECT * FROM users

  53. Make it easier to keep the lights on Lesson 3

    Tools
  54. Make it easier to keep the lights on Lesson 3

    Tools
  55. marginalia https://github.com/basecamp/marginalia

  56. # Gemfile gem “marginalia”

  57. SELECT * FROM users /* controller:users, action:index */

  58. ActiveSupport::CurrentAttributes https://api.rubyonrails.org/classes/ActiveSupport/CurrentAttributes.html

  59. class Current < ActiveSupport::CurrentAttributes attribute :user_id, :request_id end

  60. class ApplicationController before_action do Current.user_id = current_user&.id Current.request_id = request.uuid

    end end
  61. module Marginalia module Comment def self.user_id Current.user_id end def self.request_id

    Current.request_id end end end
  62. Marginalia::Comment.component = [ :controller, :action, :user_id, :request_id ]

  63. SELECT * FROM users /* controller:users, action:index, user_id:5, request_id:4cdd8083-5f10 */

  64. ActiveSupport::Notifications https://api.rubyonrails.org/classes/ActiveSupport/Notifications.html

  65. ActiveSupport::Notifications .subscribe(‘active_record.sql’) do |*args| event = ActiveSupport::Notifications::Event.new(*args) if event.duration >

    THRESHOLD # Do something end end
  66. ActiveSupport::Notifications .subscribe(‘active_record.sql’) do |*args| event = ActiveSupport::Notifications::Event.new(*args) if event.duration >

    THRESHOLD exception = LongQueryException.new(“a clear message”) Bugsnag.notify(exception) end end
  67. None
  68. Make it easier to keep the lights on Lesson 3

  69. Make it easier to keep the lights on Lesson 3

  70. Keep the bad patterns out Lesson 4

  71. begin file = Tempfile.new(“contact.csv”) csv = CSV.new(file) csv << [“id”,

    “name”] Contact.all.each do |contact| csv << [contact.id, contact.name] end ensure file.close file.unlink end
  72. begin file = Tempfile.new(“contact.csv”) csv = CSV.new(file) csv << [“id”,

    “name”] Contact.all.each do |contact| csv << [contact.id, contact.name] end ensure file.close file.unlink end
  73. None
  74. None
  75. Keep the bad patterns out Lesson 4 Tools

  76. rubocop https://github.com/rubocop-hq/rubocop

  77. None
  78. None
  79. Shitlist-driven development Shopify https://sirupsen.com/shitlists/ Florian Weingarten - RedDotRubyConf 2017

  80. Shitlist = [ClassA, ClassB, ClassC] def push_job_that_does_crazy_things(klass) if Shitlist.include?(klass) #

    existing deprecated behavior else raise Shitlist::Error.new(“a clear message”) end end
  81. RedisShitlist = [ Session, FragmentCache, AuthenticationTokens ] test “no new

    redis models introduced” do assert_equal RedisShitList, RedisModel.descendants end
  82. None
  83. Keep the bad patterns out Lesson 4

  84. Lessons ▪ Lesson 1 – Automate the debt away –

    jit_preloader ▪ Lesson 2 – Clean up code you don't use – Tombstone – dead_code_detector – coverband ▪ Lesson 3 – Make keeping the lights on easier – marginalia/ActiveSupport::CurrentAttributes – ActiveSupport::Notifications ▪ Lesson 4 – Keep the bad patterns out – rubocop – Shitlist-driven development
  85. Death By a Thousand Commits Kyle d’Oliveira RailsConf 2019