Dependency Isolation by Donald Ball

Isolating your systems from their dependencies can yield code that is more reasonable, testable, and easier to change. Ruby provides some excellent tools to help you decouple your systems from their dependencies flexibly and pragmatically. In his talk, Donald will report on a system he wrote with a very disciplined approach to managing its dependencies, and the benefits and drawbacks obtained thereby.

About the Speaker:

Donald Ball has been trying to write good software since the Atari days. After a long sojourn in enterprise Java convinced him to quit the pursuit, he was drawn back in by the fun of ruby. He currently works for LivingSocial, where he's responsible for delivering your spam.

Transcript

1. Dependency Isolation Donald Ball @donaldball github.com/dball
2. What is a dependency? • Libraries • Code you use but didn't write
3. Why isolate your app?
4. Upgrading Rails • ActiveRecord query changes • Abandoned plugins • Monkey patches • Change all the tests?! • Change all the files!!
5. Tightly Coupled Code • Difficult to change • Expensive to change • Risky to change
6. Tightly Coupled Dependencies • Change on someone else's schedule • May become abandoned
7. Let's Experiment! • All code is experimental • Constraints inspire creativity
8. Global Variables Are Evil • And sneaky
9. class ClicksController < ApplicationController skip_before_filter :check_xhr def track params = track_params.merge(ip: request.remote_ip) if params[:topic_id].present? || params[:post_id].present? params.merge!({ user_id: current_user.id }) if current_user.present? @redirect_url = TopicLinkClick.create_from(params) end if params[:redirect] == 'false' || @redirect_url.blank? render nothing: true else redirect_to(@redirect_url) end end end
10. class ClicksController < ApplicationController skip_before_filter :check_xhr def track params = track_params.merge(ip: request.remote_ip) if params[:topic_id].present? || params[:post_id].present? params.merge!({ user_id: current_user.id }) if current_user.present? @redirect_url = TopicLinkClick.create_from(params) end if params[:redirect] == 'false' || @redirect_url.blank? render nothing: true else redirect_to(@redirect_url) end end end
11. • In most languages, they're constants • But this is ruby: they're variables Class names are global variables
12. No Global Variables? • How would you coordinate classes?
13. The Experiment • Classes from each external library may appear in only one file
14. Coupled
15. require 'sequel' class CustomerManager attr_reader :database def initialize(config) @database = Sequel.connect(config[:database][:url]) end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
16. Isolate the require • But do require, explicitly • Autoloading obscures dependencies
17. require_relative 'database' class CustomerManager attr_reader :database def initialize(config) @database = Sequel.connect(config[:database][:url]) end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
18. require 'sequel'
19. • Give it a name based on its role in your app, not its given name Name the dependency
20. Isolate the entry point
21. require_relative 'database' class CustomerManager attr_reader :database def initialize(config) @database = Database.build(config[:database][:url]) end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
22. require 'sequel' module Database module_function def build(url) Sequel.connect(url) end end
23. Mission Accomplished?
24. Dependency Injection
25. require_relative 'services' class CustomerManager attr_reader :database def initialize(services) @database = services.database end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end end
26. require_relative 'database' class Services attr_reader :database def initialize(config) @database = Database.build(config[:database][:url]) end end
27. • Builds and holds references to adapters • Easy to replace in tests Services Container
28. Requirement: Updates
29. require_relative 'services' class CustomerManager ... def update(update_attributes, filter_attributes) database.transaction do database.dataset(:customers). filter(Sequel.|(filter_attributes)). update(update_attributes) end end end
30. Delegate Adapter
31. require 'sequel' require 'delegate' class Database < SimpleDelegator def self.build(url) new(url) end def initialize(url) super(Sequel.connect(url)) end def update_all(dataset, update_attributes, filter_attributes) dataset.filter(Sequel.|(filter_attributes)). update(update_attributes) end end
32. require_relative 'services' class CustomerManager attr_reader :database def initialize(services) @database = services.database end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.dataset(:customers).import(columns, rows) end end def update(update_attributes, filter_attributes) database.transaction do database.update_all(database.dataset(:customers), update_attributes, filter_attributes) end end
33. require_relative 'services' class CustomerManager attr_reader :database def initialize(services) @database = services.database end def import(records) columns = records.first.attribute_names rows = records.map(&:attribute_values) database.transaction do database.import(:customers, columns, rows) end end def update(update_attributes, filter_attributes) database.transaction do database.update_all(:customers, update_attributes, filter_attributes) end end end
34. require 'sequel' require 'delegate' class Database < SimpleDelegator def self.build(url) new(url) end def initialize(url) super(Sequel.connect(url)) end def import(table, columns, rows) self[table].import(columns, rows) end def update_all(table, update_attributes, filter_attributes) self[table]. filter(Sequel.|(filter_attributes)). update(update_attributes) end end
35. require 'sequel' require 'delegate' class Database < SimpleDelegator ... def transaction(options = {}) retries = options.fetch(:retries, 3) retried = 0 begin __getobj__.transaction do yield end rescue Sequel::DatabaseError => error if deadlock?(error) if retried < retries sleep 1.5 ** retried retried += 1 retry else raise DeadlockError, error end else raise error end end end private DEADLOCK_MESSAGE = "Mysql2::Error: " + "Deadlock found when trying to get lock; try restarting transaction" def deadlock?(error) error.message == DEADLOCK_MESSAGE end # Error class wrapping a mysql deadlock error class DeadlockError < StandardError attr_reader :cause def initialize(cause) super(cause.message) @cause = cause end end end
36. • Pure enhancement • Extant API is still available Delegate Adapter
37. Testing with adapters • Test the application with an in-memory implementation • Test the adapter integrated with the dependency
38. class TestDatabase def initialize @tables = Hash.new { |tables, name| tables[name] = {} } @next_id = 1 end def import(table_name, columns, rows) imports = rows.map do |row| row = row.dup id = get_next_id row[:id] = id insert_row(table_name, row) end end private def get_next_id id = @next_id @next_id += 1 id end def insert_row(table_name, row) table = @tables[table_name] raise IllegalArgumentException if table[row[:id]] table[row[:id]] = row end end
39. it "retries deadlocked transactions" do trigger = Thread.new do tries = 0 database.transaction(retries: 1) do tries += 1 deadlocker.for_update.select.to_a end tries.should == 2 end database.transaction do deadlocker.insert(a: 1) trigger.run deadlocker.update(a: 0) end trigger.join end
40. Full Adapter
41. require 'sequel' class Database def self.build(url) new(url) end def initialize(url) @database = Sequel.connect(url) end def import(table, columns, rows) !!dataset(table).import(columns, rows) end def update_all(table, update_attributes, filter_attributes) dataset(table). filter(Sequel.|(filter_attributes)). update(update_attributes).count end def transaction(options = {}) ... end private def dataset(table) @database[table] end ... # Error class wrapping a non-specific sequel error class Error < StandardError attr_reader :cause def initialize(cause) super(cause.message) @cause = cause end end # Error class wrapping a mysql deadlock error class DeadlockError < Error end end
42. • Explicit API, specific to your app • Easy to swap implementations Full Adapter
43. Experiment Results
44. Simpler Application Code • App expresses intent to adapter • Adapter implements intent • Reducing incidental complexity in business logic is always a win
45. Changing Adapters • Every single adapter ended up doing something I hadn't anticipated • Introducing most of them involved no changes to the application code
46. Database Adapter • Automatically retry deadlocks and lock timeouts
47. Email Address Parser • Switched to regex to avoid the performance nightmare of ruby mail (treetop)
48. CSV Parser • Record from malformed csv rows • Specify the encoding and row separator • Yield the row and the row number
49. Drawbacks • Development time • Runtime performance • SimpleDelegate uses method_missing
50. Will I Do It Again? • For services, absolutely • For stuff in stdlib, depends on the complexity • For frameworks, probably not worth the impedance mismatch
51. It's a continuum • Ruby makes it easy to move along the the continuum • Pick the point at which you derive the most benefit
52. Summary • Write your app code to depend on interfaces you design to fit your needs • Try allowing the interfaces to emerge using delegate adapters
53. Thank You!