Dependency Isolation by Donald Ball

Dependency Isolation by Donald Ball

Isolating your systems from their dependencies can yield code that is more reasonable, testable, and easier to change. Ruby provides some excellent tools to help you decouple your systems from their dependencies flexibly and pragmatically. In his talk, Donald will report on a system he wrote with a very disciplined approach to managing its dependencies, and the benefits and drawbacks obtained thereby.

About the Speaker:

Donald Ball has been trying to write good software since the Atari days. After a long sojourn in enterprise Java convinced him to quit the pursuit, he was drawn back in by the fun of ruby. He currently works for LivingSocial, where he's responsible for delivering your spam.

04370f40413ad7b5d14137511c50b98b?s=128

Triangle Ruby

October 24, 2013
Tweet

Transcript

  1. Dependency Isolation Donald Ball @donaldball github.com/dball

  2. What is a dependency? • Libraries • Code you use

    but didn't write
  3. Why isolate your app?

  4. Upgrading Rails • ActiveRecord query changes • Abandoned plugins •

    Monkey patches • Change all the tests?! • Change all the files!!
  5. Tightly Coupled Code • Difficult to change • Expensive to

    change • Risky to change
  6. Tightly Coupled Dependencies • Change on someone else's schedule •

    May become abandoned
  7. Let's Experiment! • All code is experimental • Constraints inspire

    creativity
  8. Global Variables Are Evil • And sneaky

  9. class ClicksController < ApplicationController skip_before_filter :check_xhr def track params =

    track_params.merge(ip: request.remote_ip) if params[:topic_id].present? || params[:post_id].present? params.merge!({ user_id: current_user.id }) if current_user.present? @redirect_url = TopicLinkClick.create_from(params) end if params[:redirect] == 'false' || @redirect_url.blank? render nothing: true else redirect_to(@redirect_url) end end end
  10. class ClicksController < ApplicationController skip_before_filter :check_xhr def track params =

    track_params.merge(ip: request.remote_ip) if params[:topic_id].present? || params[:post_id].present? params.merge!({ user_id: current_user.id }) if current_user.present? @redirect_url = TopicLinkClick.create_from(params) end if params[:redirect] == 'false' || @redirect_url.blank? render nothing: true else redirect_to(@redirect_url) end end end
  11. • In most languages, they're constants • But this is

    ruby: they're variables Class names are global variables
  12. No Global Variables? • How would you coordinate classes?

  13. The Experiment • Classes from each external library may appear

    in only one file
  14. Coupled

  15. require 'sequel' class CustomerManager attr_reader :database def initialize(config) @database =

    Sequel.connect(config[:database][:url]) end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
  16. Isolate the require • But do require, explicitly • Autoloading

    obscures dependencies
  17. require_relative 'database' class CustomerManager attr_reader :database def initialize(config) @database =

    Sequel.connect(config[:database][:url]) end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
  18. require 'sequel'

  19. • Give it a name based on its role in

    your app, not its given name Name the dependency
  20. Isolate the entry point

  21. require_relative 'database' class CustomerManager attr_reader :database def initialize(config) @database =

    Database.build(config[:database][:url]) end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
  22. require 'sequel' module Database module_function def build(url) Sequel.connect(url) end end

  23. Mission Accomplished?

  24. Dependency Injection

  25. require_relative 'services' class CustomerManager attr_reader :database def initialize(services) @database =

    services.database end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
  26. require_relative 'database' class Services attr_reader :database def initialize(config) @database =

    Database.build(config[:database][:url]) end end
  27. • Builds and holds references to adapters • Easy to

    replace in tests Services Container
  28. Requirement: Updates

  29. require_relative 'services' class CustomerManager ... def update(update_attributes, filter_attributes) database.transaction do

    database.dataset(:customers). filter(Sequel.|(filter_attributes)). update(update_attributes) end end end
  30. Delegate Adapter

  31. require 'sequel' require 'delegate' class Database < SimpleDelegator def self.build(url)

    new(url) end def initialize(url) super(Sequel.connect(url)) end def update_all(dataset, update_attributes, filter_attributes) dataset.filter(Sequel.|(filter_attributes)). update(update_attributes) end end
  32. require_relative 'services' class CustomerManager attr_reader :database def initialize(services) @database =

    services.database end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end def update(update_attributes, filter_attributes) database.transaction do database.update_all(database.dataset(:customers), update_attributes, filter_attributes) end end
  33. require_relative 'services' class CustomerManager attr_reader :database def initialize(services) @database =

    services.database end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.import(:customers, columns, rows) end end def update(update_attributes, filter_attributes) database.transaction do database.update_all(:customers, update_attributes, filter_attributes) end end end
  34. require 'sequel' require 'delegate' class Database < SimpleDelegator def self.build(url)

    new(url) end def initialize(url) super(Sequel.connect(url)) end def import(table, columns, rows) self[table].import(columns, rows) end def update_all(table, update_attributes, filter_attributes) self[table]. filter(Sequel.|(filter_attributes)). update(update_attributes) end end
  35. require 'sequel' require 'delegate' class Database < SimpleDelegator ... def

    transaction(options = {}) retries = options.fetch(:retries, 3) retried = 0 begin __getobj__.transaction do yield end rescue Sequel::DatabaseError => error if deadlock?(error) if retried < retries sleep 1.5 ** retried retried += 1 retry else raise DeadlockError, error end else raise error end end end private DEADLOCK_MESSAGE = "Mysql2::Error: " + "Deadlock found when trying to get lock; try restarting transaction" def deadlock?(error) error.message == DEADLOCK_MESSAGE end # Error class wrapping a mysql deadlock error class DeadlockError < StandardError attr_reader :cause def initialize(cause) super(cause.message) @cause = cause end end end
  36. • Pure enhancement • Extant API is still available Delegate

    Adapter
  37. Testing with adapters • Test the application with an in-memory

    implementation • Test the adapter integrated with the dependency
  38. class TestDatabase def initialize @tables = Hash.new { |tables, name|

    tables[name] = {} } @next_id = 1 end def import(table_name, columns, rows) imports = rows.map do |row| row = row.dup id = get_next_id row[:id] = id insert_row(table_name, row) end end private def get_next_id id = @next_id @next_id += 1 id end def insert_row(table_name, row) table = @tables[table_name] raise IllegalArgumentException if table[row[:id]] table[row[:id]] = row end end
  39. it "retries deadlocked transactions" do trigger = Thread.new do tries

    = 0 database.transaction(retries: 1) do tries += 1 deadlocker.for_update.select.to_a end tries.should == 2 end database.transaction do deadlocker.insert(a: 1) trigger.run deadlocker.update(a: 0) end trigger.join end
  40. Full Adapter

  41. require 'sequel' class Database def self.build(url) new(url) end def initialize(url)

    @database = Sequel.connect(url) end def import(table, columns, rows) !!dataset(table).import(columns, rows) end def update_all(table, update_attributes, filter_attributes) dataset(table). filter(Sequel.|(filter_attributes)). update(update_attributes).count end def transaction(options = {}) ... end private def dataset(table) @database[table] end ... # Error class wrapping a non-specific sequel error class Error < StandardError attr_reader :cause def initialize(cause) super(cause.message) @cause = cause end end # Error class wrapping a mysql deadlock error class DeadlockError < Error end end
  42. • Explicit API, specific to your app • Easy to

    swap implementations Full Adapter
  43. Experiment Results

  44. Simpler Application Code • App expresses intent to adapter •

    Adapter implements intent • Reducing incidental complexity in business logic is always a win
  45. Changing Adapters • Every single adapter ended up doing something

    I hadn't anticipated • Introducing most of them involved no changes to the application code
  46. Database Adapter • Automatically retry deadlocks and lock timeouts

  47. Email Address Parser • Switched to regex to avoid the

    performance nightmare of ruby mail (treetop)
  48. CSV Parser • Record from malformed csv rows • Specify

    the encoding and row separator • Yield the row and the row number
  49. Drawbacks • Development time • Runtime performance • SimpleDelegate uses

    method_missing
  50. Will I Do It Again? • For services, absolutely •

    For stuff in stdlib, depends on the complexity • For frameworks, probably not worth the impedance mismatch
  51. It's a continuum • Ruby makes it easy to move

    along the the continuum • Pick the point at which you derive the most benefit
  52. Summary • Write your app code to depend on interfaces

    you design to fit your needs • Try allowing the interfaces to emerge using delegate adapters
  53. Thank You!