Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Better OpenStruct (WindyCityRails 2016)

Ariel Caplan
September 16, 2016

Building a Better OpenStruct (WindyCityRails 2016)

Talk abstract:

OpenStruct, part of Ruby’s standard library, is prized for its beautiful API. It provides dynamic data objects with automatically generated getters and setters. Unfortunately, OpenStruct also carries a hefty performance penalty.

Recently, Rubyists have tried various approaches to speed up OpenStruct or provide alternatives. We will study these attempts, learning how to take advantage of the tools in our ecosystem while advancing the state of the Ruby community.

Sometimes, we can have our cake and eat it too. But it takes creativity, hard work, and willingness to question why things are the way they are.

Link from the talk:

http://jamesgolick.com/2013/4/14/mris-method-caches.html

Ariel Caplan

September 16, 2016
Tweet

More Decks by Ariel Caplan

Other Decks in Technology

Transcript

  1. OPENSTRUCT IS RUBY’S JAVASCRIPT OBJECT require 'ostruct' open_struct = OpenStruct.new(foo:

    :bar) open_struct.baz = 4 open_struct[:something] = 'whatever'
  2. OPENSTRUCT IS RUBY’S JAVASCRIPT OBJECT require 'ostruct' open_struct = OpenStruct.new(foo:

    :bar) open_struct.baz = 4 open_struct[:something] = 'whatever' open_struct.foo => :bar open_struct.baz => 4 open_struct.quux => nil
  3. OPENSTRUCT IS RUBY’S JAVASCRIPT OBJECT require 'ostruct' open_struct = OpenStruct.new(foo:

    :bar) open_struct.baz = 4 open_struct[:something] = 'whatever' open_struct[:foo] => :bar open_struct["baz"] => 4 open_struct[:quux] => nil open_struct.foo => :bar open_struct.baz => 4 open_struct.quux => nil
  4. WHY USE OPENSTRUCT? 3 common use cases for OpenStruct (Erik

    Michaels-Ober's list): • Consume an API • Configuration object • Simple test double
  5. CONSUME AN API hash = JSON.parse(APICall.execute) results = OpenStruct.new(hash) class

    APICall < OpenStruct def self.execute response = Net::HTTP.get(URI('domain.com/endpoint')) new(JSON.parse(response)) end end results = APICall.execute SIMPLE STRATEGY: PASS RESPONSE INTO AN OPENSTRUCT ADVANCED STRATEGY: SUBCLASS OPENSTRUCT
  6. CONSUME AN API { "thing": [1, 2, 3], "other_thing": "hello"

    } If the API response is… Instead of… response['thing'] => [1, 2, 3]
  7. CONSUME AN API { "thing": [1, 2, 3], "other_thing": "hello"

    } If the API response is… response.thing => [1, 2, 3] You can write this instead! Instead of… response['thing'] => [1, 2, 3]
  8. CONSUME AN API You can also define your own methods!

    class User < OpenStruct def name "#{first_name} #{last_name}" end end api_call = JSON.parse(API.get('/users/1')) # => {"first_name"=>"Ariel","last_name"=>"Caplan"} User.new(api_call).name # => "Ariel Caplan"
  9. CONFIGURATION OBJECT class MyGem def self.configure yield configuration end def

    self.configuration @configuration ||= OpenStruct.new end end MyGem.configure do |configuration| configuration.setting = true end
  10. TEST DOUBLE class Order def initialize(payment_gateway, products) @payment_gateway = payment_gateway

    @products = products end def pay payment_gateway.total = total_cost payment_gateway.charge end def total_cost # sum up the products end end
  11. TEST DOUBLE class Order def initialize(payment_gateway, products) @payment_gateway = payment_gateway

    @products = products end def pay payment_gateway.total = total_cost payment_gateway.charge end def total_cost # sum up the products end end Expensive to test!
  12. TEST DOUBLE # Stub out methods payment_gateway = OpenStruct.new(charge: :PAID)

    # Run the test payment_result = Order.new(payment_gateway, products).pay # Assert a value is returned assert_equal payment_result, :PAID # Assert a value is set on the OpenStruct assert_equal payment_gateway.total, total_cost(products)
  13. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class.
  14. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. foo = Object.new def foo.bar "hello from bar" end foo.bar => "hello from bar" foo.singleton_methods => [:bar]
  15. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class.
  16. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. OpenStruct.new(foo: :bar)
  17. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end OpenStruct.new(foo: :bar)
  18. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end OpenStruct.new(foo: :bar) open_struct.baz = 4
  19. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4)
  20. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end def method_missing(mid, *args) mname = mid.id2name if mname.chomp!('=') @table[new_ostruct_member(mname)] = args[0] else @table[mid] end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4)
  21. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end def method_missing(mid, *args) mname = mid.id2name if mname.chomp!('=') @table[new_ostruct_member(mname)] = args[0] else @table[mid] end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4)
  22. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end def method_missing(mid, *args) mname = mid.id2name if mname.chomp!('=') @table[new_ostruct_member(mname)] = args[0] else @table[mid] end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4) open_struct.baz
  23. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end
  24. HOW DOES IT WORK? • Under the hood, OpenStruct defines

    attribute setter/getter methods on the object’s singleton class. def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end This is the singleton class version of attr_accessor :foo but it uses the internal @table, not instance variables!
  25. • Under the hood, OpenStruct defines attribute setter/getter methods on

    the object’s singleton class. HOW DOES IT WORK?
  26. • No 2 OpenStructs share a set of methods •

    Under the hood, OpenStruct defines attribute setter/getter methods on the object’s singleton class. HOW DOES IT WORK?
  27. • No 2 OpenStructs share a set of methods •

    The methods must be defined each time! • Under the hood, OpenStruct defines attribute setter/getter methods on the object’s singleton class. HOW DOES IT WORK?
  28. • Q: How slow is OpenStruct? • A: OPENSTRUCT IS

    SLOW • Translation: 10-40x slower than an explicitly defined class
  29. • Q: How slow is OpenStruct? • A: OPENSTRUCT IS

    SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow?
  30. • Q: How slow is OpenStruct? • A: OPENSTRUCT IS

    SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? • Defining methods is slow
  31. • Q: How slow is OpenStruct? • A: OPENSTRUCT IS

    SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? • Defining methods is slow • method_missing is slow
  32. • Q: How slow is OpenStruct? • A: OPENSTRUCT IS

    SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? • Defining methods is slow • method_missing is slow • In < 2.1, reset the global method cache, no longer a problem
  33. HI EVERYONE! MY NAME IS ARIEL CAPLAN. YOU CAN FIND

    ME ONLINE AT @AMCAPLAN AND AMCAPLAN.NINJA
  34. WE USE A SYSTEM OF MICROSERVICES AND EXTERNAL APIS COMMUNICATING

    VIA JSON PARSED INTO RUBY HASHES FED INTO OPENSTRUCTS
  35. NO.

  36. — https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)

    end def update(args) args.each { |k, v| assign(k, v) } end
 def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end
  37. — https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)

    end def update(args) args.each { |k, v| assign(k, v) } end
 def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end open_fast_struct.baz # baz exists
  38. — https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)

    end def update(args) args.each { |k, v| assign(k, v) } end
 def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end open_fast_struct.baz = 4 open_fast_struct.baz # baz exists
  39. — https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)

    end def update(args) args.each { |k, v| assign(k, v) } end
 def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end open_fast_struct.baz = 4 open_fast_struct.baz # baz does not exist open_fast_struct.baz # baz exists
  40. SAMPLE USAGE PersistentOpenStruct class Animal < PersistentOpenStruct def speak "The

    #{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal type="dog", sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal type="dog", sound="woof", ears="droopy", nose=["cold", "wet"], tail="waggable">
  41. SAMPLE USAGE PersistentOpenStruct class Animal < PersistentOpenStruct def speak "The

    #{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal type="dog", sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal type="dog", sound="woof", ears="droopy", nose=["cold", "wet"], tail="waggable"> Animal.instance_methods(false) # => [:speak, :type=, :type, :sound=, :sound, :ears=, :ears,
 :nose=, :nose, :tail=, :tail]
  42. HOW IT WORKS PersistentOpenStruct def new_ostruct_member(name) name = name.to_sym unless

    respond_to?(name) self.class.send(:define_method, name) { @table[name] } self.class.send(:define_method, "#{name}=") { |x| @table[name] = x } end name end def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end OpenStruct PersistentOpenStruct
  43. HOW IT WORKS PersistentOpenStruct def new_ostruct_member(name) name = name.to_sym unless

    respond_to?(name) self.class.send(:define_method, name) { @table[name] } self.class.send(:define_method, "#{name}=") { |x| @table[name] = x } end name end def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end OpenStruct PersistentOpenStruct def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) self.class.send(:define_method, name) { @table[name] } self.class.send(:define_method, "#{name}=") { |x| @table[name] = x } end name end
  44. Comparison: RegularClass: 3673779.6 i/s PersistentOpenStruct: 764359.4 i/s - 4.81x slower

    OpenFastStruct: 546808.8 i/s - 6.72x slower OpenStruct: 155130.1 i/s - 23.68x slower
  45. THE INFORMATION YOU CRAVE • Let n = number of

    methods to define USING THE NOTATION YOU LOVE!
  46. THE INFORMATION YOU CRAVE • Let n = number of

    methods to define • Let o = objects to create USING THE NOTATION YOU LOVE!
  47. THE INFORMATION YOU CRAVE • Let n = number of

    methods to define • Let o = objects to create • Using PersistentOpenStruct: O(n) USING THE NOTATION YOU LOVE!
  48. THE INFORMATION YOU CRAVE • Let n = number of

    methods to define • Let o = objects to create • Using PersistentOpenStruct: O(n) • Using OpenStruct: O(no) USING THE NOTATION YOU LOVE!
  49. MATH PROVIDES A USEFUL FRAMEWORK FOR THINKING ABOUT PROBLEMS. AS

    ENGINEERS, WE NEED ANSWERS GROUNDED IN REALITY.
  50. SAMPLE USAGE DynamicClass Animal = DynamicClass.new do def speak "The

    #{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal:0x007fdb2b818ba8 @type="dog", @sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal:0x007fc26b1841d0 @type="dog", @sound="woof", @ears="droopy", @nose=["cold", "wet"], @tail=“waggable">
  51. SAMPLE USAGE DynamicClass Animal = DynamicClass.new do def speak "The

    #{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal:0x007fdb2b818ba8 @type="dog", @sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal:0x007fc26b1841d0 @type="dog", @sound="woof", @ears="droopy", @nose=["cold", "wet"], @tail=“waggable"> Animal.instance_methods(false) # => [:to_h, :speak, :type=, :type, :sound=, :sound, :ears=, :ears,
 :nose=, :nose, :tail=, :tail]
  52. HOW IT WORKS DynamicClass class << self def attributes @attributes

    ||= Set.new end def add_methods!(key) class_exec do attr_writer key unless method_defined?("#{key}=") attr_reader key unless method_defined?("#{key}") attributes << key end end end
  53. HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =

    mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end
  54. HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =

    mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4
  55. HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =

    mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 animal.baz
  56. HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =

    mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def []=(key, value) key = key.to_sym instance_variable_set(:"@#{key}", value) unless self.class.attributes.include?(key) self.class.add_methods!(key) end end def [](key) instance_variable_get(:"@#{key}") end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 animal.baz
  57. HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =

    mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def []=(key, value) key = key.to_sym instance_variable_set(:"@#{key}", value) unless self.class.attributes.include?(key) self.class.add_methods!(key) end end def [](key) instance_variable_get(:"@#{key}") end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 animal.baz Instead of an internal hash @table, we use instance variables just as you would in a standard class!
  58. IS IT FAST? Comparison: RegularClass: 3757567.8 i/s DynamicClass: 1250634.2 i/s

    - 3.00x slower PersistentOpenStruct: 766792.7 i/s - 4.90x slower OpenFastStruct: 525565.1 i/s - 7.15x slower OpenStruct: 147361.4 i/s - 25.50x slower
  59. ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library

    may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool.
  60. ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library

    may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. • Benchmark, benchmark, benchmark!
  61. ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library

    may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. • Benchmark, benchmark, benchmark! • Experiments are probably the most satisfying, internally rewarding activity in all of programming. Sometimes you gain something useful, other times it’s just fun.
  62. ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library

    may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. • Benchmark, benchmark, benchmark! • Experiments are probably the most satisfying, internally rewarding activity in all of programming. Sometimes you gain something useful, other times it’s just fun. • Whoever you are, you have something to offer!