Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dependency Isolation by Donald Ball

Dependency Isolation by Donald Ball

Isolating your systems from their dependencies can yield code that is more reasonable, testable, and easier to change. Ruby provides some excellent tools to help you decouple your systems from their dependencies flexibly and pragmatically. In his talk, Donald will report on a system he wrote with a very disciplined approach to managing its dependencies, and the benefits and drawbacks obtained thereby.

About the Speaker:

Donald Ball has been trying to write good software since the Atari days. After a long sojourn in enterprise Java convinced him to quit the pursuit, he was drawn back in by the fun of ruby. He currently works for LivingSocial, where he's responsible for delivering your spam.

Triangle Ruby

October 24, 2013
Tweet

More Decks by Triangle Ruby

Other Decks in Programming

Transcript

  1. Dependency Isolation
    Donald Ball
    @donaldball
    github.com/dball

    View full-size slide

  2. What is a dependency?
    • Libraries
    • Code you use but didn't write

    View full-size slide

  3. Why isolate your app?

    View full-size slide

  4. Upgrading Rails
    • ActiveRecord query changes
    • Abandoned plugins
    • Monkey patches
    • Change all the tests?!
    • Change all the files!!

    View full-size slide

  5. Tightly Coupled Code
    • Difficult to change
    • Expensive to change
    • Risky to change

    View full-size slide

  6. Tightly Coupled
    Dependencies
    • Change on someone else's schedule
    • May become abandoned

    View full-size slide

  7. Let's Experiment!
    • All code is experimental
    • Constraints inspire creativity

    View full-size slide

  8. Global Variables Are Evil
    • And sneaky

    View full-size slide

  9. class ClicksController < ApplicationController
    skip_before_filter :check_xhr
    def track
    params = track_params.merge(ip: request.remote_ip)
    if params[:topic_id].present? || params[:post_id].present?
    params.merge!({ user_id: current_user.id }) if
    current_user.present?
    @redirect_url = TopicLinkClick.create_from(params)
    end
    if params[:redirect] == 'false' || @redirect_url.blank?
    render nothing: true
    else
    redirect_to(@redirect_url)
    end
    end
    end

    View full-size slide

  10. class ClicksController < ApplicationController
    skip_before_filter :check_xhr
    def track
    params = track_params.merge(ip: request.remote_ip)
    if params[:topic_id].present? || params[:post_id].present?
    params.merge!({ user_id: current_user.id }) if
    current_user.present?
    @redirect_url = TopicLinkClick.create_from(params)
    end
    if params[:redirect] == 'false' || @redirect_url.blank?
    render nothing: true
    else
    redirect_to(@redirect_url)
    end
    end
    end

    View full-size slide

  11. • In most languages, they're constants
    • But this is ruby: they're variables
    Class names are global
    variables

    View full-size slide

  12. No Global Variables?
    • How would you coordinate classes?

    View full-size slide

  13. The Experiment
    • Classes from each external library may
    appear in only one file

    View full-size slide

  14. require 'sequel'
    class CustomerManager
    attr_reader :database
    def initialize(config)
    @database = Sequel.connect(config[:database][:url])
    end
    def import(records)
    columns = records.first.attribute_names
    rows = records.map(&:attribute_values)
    database.transaction do
    database.dataset(:customers).import(columns, rows)
    end
    end
    end

    View full-size slide

  15. Isolate the require
    • But do require, explicitly
    • Autoloading obscures dependencies

    View full-size slide

  16. require_relative 'database'
    class CustomerManager
    attr_reader :database
    def initialize(config)
    @database = Sequel.connect(config[:database][:url])
    end
    def import(records)
    columns = records.first.attribute_names
    rows = records.map(&:attribute_values)
    database.transaction do
    database.dataset(:customers).import(columns, rows)
    end
    end
    end

    View full-size slide

  17. require 'sequel'

    View full-size slide

  18. • Give it a name based on its role in your
    app, not its given name
    Name the dependency

    View full-size slide

  19. Isolate the entry point

    View full-size slide

  20. require_relative 'database'
    class CustomerManager
    attr_reader :database
    def initialize(config)
    @database = Database.build(config[:database][:url])
    end
    def import(records)
    columns = records.first.attribute_names
    rows = records.map(&:attribute_values)
    database.transaction do
    database.dataset(:customers).import(columns, rows)
    end
    end
    end

    View full-size slide

  21. require 'sequel'
    module Database
    module_function
    def build(url)
    Sequel.connect(url)
    end
    end

    View full-size slide

  22. Mission Accomplished?

    View full-size slide

  23. Dependency Injection

    View full-size slide

  24. require_relative 'services'
    class CustomerManager
    attr_reader :database
    def initialize(services)
    @database = services.database
    end
    def import(records)
    columns = records.first.attribute_names
    rows = records.map(&:attribute_values)
    database.transaction do
    database.dataset(:customers).import(columns, rows)
    end
    end
    end

    View full-size slide

  25. require_relative 'database'
    class Services
    attr_reader :database
    def initialize(config)
    @database = Database.build(config[:database][:url])
    end
    end

    View full-size slide

  26. • Builds and holds references to adapters
    • Easy to replace in tests
    Services Container

    View full-size slide

  27. Requirement: Updates

    View full-size slide

  28. require_relative 'services'
    class CustomerManager
    ...
    def update(update_attributes, filter_attributes)
    database.transaction do
    database.dataset(:customers).
    filter(Sequel.|(filter_attributes)).
    update(update_attributes)
    end
    end
    end

    View full-size slide

  29. Delegate Adapter

    View full-size slide

  30. require 'sequel'
    require 'delegate'
    class Database < SimpleDelegator
    def self.build(url)
    new(url)
    end
    def initialize(url)
    super(Sequel.connect(url))
    end
    def update_all(dataset, update_attributes, filter_attributes)
    dataset.filter(Sequel.|(filter_attributes)).
    update(update_attributes)
    end
    end

    View full-size slide

  31. require_relative 'services'
    class CustomerManager
    attr_reader :database
    def initialize(services)
    @database = services.database
    end
    def import(records)
    columns = records.first.attribute_names
    rows = records.map(&:attribute_values)
    database.transaction do
    database.dataset(:customers).import(columns, rows)
    end
    end
    def update(update_attributes, filter_attributes)
    database.transaction do
    database.update_all(database.dataset(:customers),
    update_attributes,
    filter_attributes)
    end
    end

    View full-size slide

  32. require_relative 'services'
    class CustomerManager
    attr_reader :database
    def initialize(services)
    @database = services.database
    end
    def import(records)
    columns = records.first.attribute_names
    rows = records.map(&:attribute_values)
    database.transaction do
    database.import(:customers, columns, rows)
    end
    end
    def update(update_attributes, filter_attributes)
    database.transaction do
    database.update_all(:customers, update_attributes,
    filter_attributes)
    end
    end
    end

    View full-size slide

  33. require 'sequel'
    require 'delegate'
    class Database < SimpleDelegator
    def self.build(url)
    new(url)
    end
    def initialize(url)
    super(Sequel.connect(url))
    end
    def import(table, columns, rows)
    self[table].import(columns, rows)
    end
    def update_all(table, update_attributes, filter_attributes)
    self[table].
    filter(Sequel.|(filter_attributes)).
    update(update_attributes)
    end
    end

    View full-size slide

  34. require 'sequel'
    require 'delegate'
    class Database < SimpleDelegator
    ...
    def transaction(options = {})
    retries = options.fetch(:retries, 3)
    retried = 0
    begin
    __getobj__.transaction do
    yield
    end
    rescue Sequel::DatabaseError => error
    if deadlock?(error)
    if retried < retries
    sleep 1.5 ** retried
    retried += 1
    retry
    else
    raise DeadlockError, error
    end
    else
    raise error
    end
    end
    end
    private
    DEADLOCK_MESSAGE =
    "Mysql2::Error: " +
    "Deadlock found when trying to get lock; try restarting transaction"
    def deadlock?(error)
    error.message == DEADLOCK_MESSAGE
    end
    # Error class wrapping a mysql deadlock error
    class DeadlockError < StandardError
    attr_reader :cause
    def initialize(cause)
    super(cause.message)
    @cause = cause
    end
    end
    end

    View full-size slide

  35. • Pure enhancement
    • Extant API is still available
    Delegate Adapter

    View full-size slide

  36. Testing with adapters
    • Test the application with an in-memory
    implementation
    • Test the adapter integrated with the
    dependency

    View full-size slide

  37. class TestDatabase
    def initialize
    @tables = Hash.new { |tables, name| tables[name] = {} }
    @next_id = 1
    end
    def import(table_name, columns, rows)
    imports = rows.map do |row|
    row = row.dup
    id = get_next_id
    row[:id] = id
    insert_row(table_name, row)
    end
    end
    private
    def get_next_id
    id = @next_id
    @next_id += 1
    id
    end
    def insert_row(table_name, row)
    table = @tables[table_name]
    raise IllegalArgumentException if table[row[:id]]
    table[row[:id]] = row
    end
    end

    View full-size slide

  38. it "retries deadlocked transactions" do
    trigger = Thread.new do
    tries = 0
    database.transaction(retries: 1) do
    tries += 1
    deadlocker.for_update.select.to_a
    end
    tries.should == 2
    end
    database.transaction do
    deadlocker.insert(a: 1)
    trigger.run
    deadlocker.update(a: 0)
    end
    trigger.join
    end

    View full-size slide

  39. Full Adapter

    View full-size slide

  40. require 'sequel'
    class Database
    def self.build(url)
    new(url)
    end
    def initialize(url)
    @database = Sequel.connect(url)
    end
    def import(table, columns, rows)
    !!dataset(table).import(columns, rows)
    end
    def update_all(table, update_attributes, filter_attributes)
    dataset(table).
    filter(Sequel.|(filter_attributes)).
    update(update_attributes).count
    end
    def transaction(options = {})
    ...
    end
    private
    def dataset(table)
    @database[table]
    end
    ...
    # Error class wrapping a non-specific sequel error
    class Error < StandardError
    attr_reader :cause
    def initialize(cause)
    super(cause.message)
    @cause = cause
    end
    end
    # Error class wrapping a mysql deadlock error
    class DeadlockError < Error
    end
    end

    View full-size slide

  41. • Explicit API, specific to your app
    • Easy to swap implementations
    Full Adapter

    View full-size slide

  42. Experiment Results

    View full-size slide

  43. Simpler Application Code
    • App expresses intent to adapter
    • Adapter implements intent
    • Reducing incidental complexity in
    business logic is always a win

    View full-size slide

  44. Changing Adapters
    • Every single adapter ended up doing
    something I hadn't anticipated
    • Introducing most of them involved no
    changes to the application code

    View full-size slide

  45. Database Adapter
    • Automatically retry deadlocks and lock
    timeouts

    View full-size slide

  46. Email Address Parser
    • Switched to regex to avoid the
    performance nightmare of ruby mail
    (treetop)

    View full-size slide

  47. CSV Parser
    • Record from malformed csv rows
    • Specify the encoding and row separator
    • Yield the row and the row number

    View full-size slide

  48. Drawbacks
    • Development time
    • Runtime performance
    • SimpleDelegate uses method_missing

    View full-size slide

  49. Will I Do It Again?
    • For services, absolutely
    • For stuff in stdlib, depends on the
    complexity
    • For frameworks, probably not worth the
    impedance mismatch

    View full-size slide

  50. It's a continuum
    • Ruby makes it easy to move along the
    the continuum
    • Pick the point at which you derive the
    most benefit

    View full-size slide

  51. Summary
    • Write your app code to depend on
    interfaces you design to fit your needs
    • Try allowing the interfaces to emerge
    using delegate adapters

    View full-size slide