Share
Embed
Building a Better OpenStruct (WindyCityRails 2016)
Talk abstract:
OpenStruct, part of Ruby’s standard library, is prized for its beautiful API. It provides dynamic data objects with automatically generated getters and setters. Unfortunately, OpenStruct also carries a hefty performance penalty.
Recently, Rubyists have tried various approaches to speed up OpenStruct or provide alternatives. We will study these attempts, learning how to take advantage of the tools in our ecosystem while advancing the state of the Ruby community.
Sometimes, we can have our cake and eat it too. But it takes creativity, hard work, and willingness to question why things are the way they are.
Link from the talk:
http://jamesgolick.com/2013/4/14/mris-method-caches.html
Ariel Caplan
September 16, 2016
Transcript
143 slides
-
OPENSTRUCT IS RUBY’S JAVASCRIPT OBJECT require 'ostruct' open_struct = OpenStruct.new(foo:
:bar) open_struct.baz = 4 open_struct[:something] = 'whatever' -
OPENSTRUCT IS RUBY’S JAVASCRIPT OBJECT require 'ostruct' open_struct = OpenStruct.new(foo:
:bar) open_struct.baz = 4 open_struct[:something] = 'whatever' open_struct.foo => :bar open_struct.baz => 4 open_struct.quux => nil -
OPENSTRUCT IS RUBY’S JAVASCRIPT OBJECT require 'ostruct' open_struct = OpenStruct.new(foo:
:bar) open_struct.baz = 4 open_struct[:something] = 'whatever' open_struct[:foo] => :bar open_struct["baz"] => 4 open_struct[:quux] => nil open_struct.foo => :bar open_struct.baz => 4 open_struct.quux => nil -
WHY USE OPENSTRUCT? 3 common use cases for OpenStruct (Erik
Michaels-Ober's list): • Consume an API • Configuration object • Simple test double -
CONSUME AN API hash = JSON.parse(APICall.execute) results = OpenStruct.new(hash) SIMPLE
STRATEGY: PASS RESPONSE INTO AN OPENSTRUCT -
CONSUME AN API hash = JSON.parse(APICall.execute) results = OpenStruct.new(hash) class
APICall < OpenStruct def self.execute response = Net::HTTP.get(URI('domain.com/endpoint')) new(JSON.parse(response)) end end results = APICall.execute SIMPLE STRATEGY: PASS RESPONSE INTO AN OPENSTRUCT ADVANCED STRATEGY: SUBCLASS OPENSTRUCT -
CONSUME AN API { "thing": [1, 2, 3], "other_thing": "hello"
} If the API response is… -
CONSUME AN API { "thing": [1, 2, 3], "other_thing": "hello"
} If the API response is… Instead of… response['thing'] => [1, 2, 3] -
CONSUME AN API { "thing": [1, 2, 3], "other_thing": "hello"
} If the API response is… response.thing => [1, 2, 3] You can write this instead! Instead of… response['thing'] => [1, 2, 3] -
CONSUME AN API You can also define your own methods!
class User < OpenStruct def name "#{first_name} #{last_name}" end end api_call = JSON.parse(API.get('/users/1')) # => {"first_name"=>"Ariel","last_name"=>"Caplan"} User.new(api_call).name # => "Ariel Caplan" -
CONFIGURATION OBJECT class MyGem def self.configure yield configuration end def
self.configuration @configuration ||= OpenStruct.new end end MyGem.configure do |configuration| configuration.setting = true end -
TEST DOUBLE class Order def initialize(payment_gateway, products) @payment_gateway = payment_gateway
@products = products end def pay payment_gateway.total = total_cost payment_gateway.charge end def total_cost # sum up the products end end -
TEST DOUBLE class Order def initialize(payment_gateway, products) @payment_gateway = payment_gateway
@products = products end def pay payment_gateway.total = total_cost payment_gateway.charge end def total_cost # sum up the products end end Expensive to test! -
TEST DOUBLE # Stub out methods payment_gateway = OpenStruct.new(charge: :PAID)
# Run the test payment_result = Order.new(payment_gateway, products).pay # Assert a value is returned assert_equal payment_result, :PAID # Assert a value is set on the OpenStruct assert_equal payment_gateway.total, total_cost(products) -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. foo = Object.new def foo.bar "hello from bar" end foo.bar => "hello from bar" foo.singleton_methods => [:bar] -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. OpenStruct.new(foo: :bar) -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end OpenStruct.new(foo: :bar) -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end OpenStruct.new(foo: :bar) open_struct.baz = 4 -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4) -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end def method_missing(mid, *args) mname = mid.id2name if mname.chomp!('=') @table[new_ostruct_member(mname)] = args[0] else @table[mid] end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4) -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end def method_missing(mid, *args) mname = mid.id2name if mname.chomp!('=') @table[new_ostruct_member(mname)] = args[0] else @table[mid] end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4) -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def initialize(hash=nil) @table = {} if hash hash.each_pair do |k, v| k = k.to_sym @table[k] = v new_ostruct_member(k) end end end def method_missing(mid, *args) mname = mid.id2name if mname.chomp!('=') @table[new_ostruct_member(mname)] = args[0] else @table[mid] end end OpenStruct.new(foo: :bar) open_struct.baz = 4 open_struct.baz=(4) open_struct.baz -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end -
HOW DOES IT WORK? • Under the hood, OpenStruct defines
attribute setter/getter methods on the object’s singleton class. def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end This is the singleton class version of attr_accessor :foo but it uses the internal @table, not instance variables! -
• Under the hood, OpenStruct defines attribute setter/getter methods on
the object’s singleton class. HOW DOES IT WORK? -
• No 2 OpenStructs share a set of methods •
Under the hood, OpenStruct defines attribute setter/getter methods on the object’s singleton class. HOW DOES IT WORK? -
• No 2 OpenStructs share a set of methods •
The methods must be defined each time! • Under the hood, OpenStruct defines attribute setter/getter methods on the object’s singleton class. HOW DOES IT WORK? -
• Q: How slow is OpenStruct? • A: OPENSTRUCT IS
SLOW • Translation: 10-40x slower than an explicitly defined class -
• Q: How slow is OpenStruct? • A: OPENSTRUCT IS
SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? -
• Q: How slow is OpenStruct? • A: OPENSTRUCT IS
SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? • Defining methods is slow -
• Q: How slow is OpenStruct? • A: OPENSTRUCT IS
SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? • Defining methods is slow • method_missing is slow -
• Q: How slow is OpenStruct? • A: OPENSTRUCT IS
SLOW • Translation: 10-40x slower than an explicitly defined class • Q: Why is OpenStruct slow? • Defining methods is slow • method_missing is slow • In < 2.1, reset the global method cache, no longer a problem -
IN RUBY >= 2.1, WE CAN CREATE OPENSTRUCTS WITHOUT SLOWING
DOWN THE REST OF OUR APPLICATION! -
HI EVERYONE! MY NAME IS ARIEL CAPLAN. YOU CAN FIND
ME ONLINE AT @AMCAPLAN AND AMCAPLAN.NINJA -
WE USE A SYSTEM OF MICROSERVICES AND EXTERNAL APIS COMMUNICATING
VIA JSON PARSED INTO RUBY HASHES -
WE USE A SYSTEM OF MICROSERVICES AND EXTERNAL APIS COMMUNICATING
VIA JSON PARSED INTO RUBY HASHES FED INTO OPENSTRUCTS -
— https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)
end def update(args) args.each { |k, v| assign(k, v) } end def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end -
— https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)
end def update(args) args.each { |k, v| assign(k, v) } end def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end open_fast_struct.baz # baz exists -
— https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)
end def update(args) args.each { |k, v| assign(k, v) } end def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end open_fast_struct.baz = 4 open_fast_struct.baz # baz exists -
— https://github.com/arturoherrero/ofstruct/blob/master/lib/ofstruct.rb def initialize(args = {}) @members = {} update(args)
end def update(args) args.each { |k, v| assign(k, v) } end def assign(key, value) @members[key.to_sym] = value end def method_missing(name, *args) @members.fetch(name) do if name[-1] == "=" assign(name[0..-2], args.first) else assign(key, self.class.new) end end end open_fast_struct.baz = 4 open_fast_struct.baz # baz does not exist open_fast_struct.baz # baz exists -
OpenStruct.new.foo => nil OpenFastStruct.new.foo => #<OpenFastStruct> if open_struct_response.foo “ falsey
if open_fast_struct_response.foo -
OpenStruct.new.foo => nil OpenFastStruct.new.foo => #<OpenFastStruct> if open_struct_response.foo “ falsey
“ truthy if open_fast_struct_response.foo -
SAMPLE USAGE PersistentOpenStruct class Animal < PersistentOpenStruct def speak "The
#{type} makes a #{sound} sound!" end end -
SAMPLE USAGE PersistentOpenStruct class Animal < PersistentOpenStruct def speak "The
#{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal type="dog", sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal type="dog", sound="woof", ears="droopy", nose=["cold", "wet"], tail="waggable"> -
SAMPLE USAGE PersistentOpenStruct class Animal < PersistentOpenStruct def speak "The
#{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal type="dog", sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal type="dog", sound="woof", ears="droopy", nose=["cold", "wet"], tail="waggable"> Animal.instance_methods(false) # => [:speak, :type=, :type, :sound=, :sound, :ears=, :ears, :nose=, :nose, :tail=, :tail] -
HOW IT WORKS PersistentOpenStruct def new_ostruct_member(name) name = name.to_sym unless
respond_to?(name) self.class.send(:define_method, name) { @table[name] } self.class.send(:define_method, "#{name}=") { |x| @table[name] = x } end name end def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end OpenStruct PersistentOpenStruct -
HOW IT WORKS PersistentOpenStruct def new_ostruct_member(name) name = name.to_sym unless
respond_to?(name) self.class.send(:define_method, name) { @table[name] } self.class.send(:define_method, "#{name}=") { |x| @table[name] = x } end name end def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end OpenStruct PersistentOpenStruct def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) define_singleton_method(name) { @table[name] } define_singleton_method("#{name}=") { |x| @table[name] = x } end name end def new_ostruct_member(name) name = name.to_sym unless respond_to?(name) self.class.send(:define_method, name) { @table[name] } self.class.send(:define_method, "#{name}=") { |x| @table[name] = x } end name end -
THIS IS A TERRIBLE HACK. IT MAY HAVE SECURITY IMPLICATIONS.
AND IT MADE OUR APP 10% FASTER. -
Comparison: RegularClass: 3673779.6 i/s PersistentOpenStruct: 764359.4 i/s - 4.81x slower
OpenFastStruct: 546808.8 i/s - 6.72x slower OpenStruct: 155130.1 i/s - 23.68x slower -
THE INFORMATION YOU CRAVE • Let n = number of
methods to define USING THE NOTATION YOU LOVE! -
THE INFORMATION YOU CRAVE • Let n = number of
methods to define • Let o = objects to create USING THE NOTATION YOU LOVE! -
THE INFORMATION YOU CRAVE • Let n = number of
methods to define • Let o = objects to create • Using PersistentOpenStruct: O(n) USING THE NOTATION YOU LOVE! -
THE INFORMATION YOU CRAVE • Let n = number of
methods to define • Let o = objects to create • Using PersistentOpenStruct: O(n) • Using OpenStruct: O(no) USING THE NOTATION YOU LOVE! -
MATH PROVIDES A USEFUL FRAMEWORK FOR THINKING ABOUT PROBLEMS. AS
ENGINEERS, WE NEED ANSWERS GROUNDED IN REALITY. -
SAMPLE USAGE DynamicClass Animal = DynamicClass.new do def speak "The
#{type} makes a #{sound} sound!" end end -
SAMPLE USAGE DynamicClass Animal = DynamicClass.new do def speak "The
#{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal:0x007fdb2b818ba8 @type="dog", @sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal:0x007fc26b1841d0 @type="dog", @sound="woof", @ears="droopy", @nose=["cold", "wet"], @tail=“waggable"> -
SAMPLE USAGE DynamicClass Animal = DynamicClass.new do def speak "The
#{type} makes a #{sound} sound!" end end dog = Animal.new(type: 'dog', sound: 'woof') # => #<Animal:0x007fdb2b818ba8 @type="dog", @sound="woof"> dog.speak # => "The dog makes a woof sound!" dog.ears = 'droopy' dog[:nose] = ['cold', 'wet'] dog['tail'] = 'waggable' dog # => #<Animal:0x007fc26b1841d0 @type="dog", @sound="woof", @ears="droopy", @nose=["cold", "wet"], @tail=“waggable"> Animal.instance_methods(false) # => [:to_h, :speak, :type=, :type, :sound=, :sound, :ears=, :ears, :nose=, :nose, :tail=, :tail] -
HOW IT WORKS DynamicClass class << self def attributes @attributes
||= Set.new end def add_methods!(key) class_exec do attr_writer key unless method_defined?("#{key}=") attr_reader key unless method_defined?("#{key}") attributes << key end end end -
HOW IT WORKS DynamicClass def initialize(attributes = {}) attributes.each_pair do
|key, value| __send__(:"#{key}=", value) end end -
HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =
mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end -
HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =
mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 -
HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =
mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 animal.baz -
HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =
mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def []=(key, value) key = key.to_sym instance_variable_set(:"@#{key}", value) unless self.class.attributes.include?(key) self.class.add_methods!(key) end end def [](key) instance_variable_get(:"@#{key}") end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 animal.baz -
HOW IT WORKS DynamicClass def method_missing(mid, *args) if (mname =
mid[/.*(?==\z)/m]) self[mname] = args.first else self[mid] end end def []=(key, value) key = key.to_sym instance_variable_set(:"@#{key}", value) unless self.class.attributes.include?(key) self.class.add_methods!(key) end end def [](key) instance_variable_get(:"@#{key}") end def initialize(attributes = {}) attributes.each_pair do |key, value| __send__(:"#{key}=", value) end end animal.baz = 4 animal.baz Instead of an internal hash @table, we use instance variables just as you would in a standard class! -
IS IT FAST? Comparison: RegularClass: 3757567.8 i/s DynamicClass: 1250634.2 i/s
- 3.00x slower PersistentOpenStruct: 766792.7 i/s - 4.90x slower OpenFastStruct: 525565.1 i/s - 7.15x slower OpenStruct: 147361.4 i/s - 25.50x slower -
ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library
may have room for improvement! -
ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library
may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. -
ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library
may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. • Benchmark, benchmark, benchmark! -
ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library
may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. • Benchmark, benchmark, benchmark! • Experiments are probably the most satisfying, internally rewarding activity in all of programming. Sometimes you gain something useful, other times it’s just fun. -
ROUNDUP: WHAT HAVE WE LEARNED? • Even the Standard Library
may have room for improvement! • By focusing on a particular use case, you might create something that works better than a standard tool. • Benchmark, benchmark, benchmark! • Experiments are probably the most satisfying, internally rewarding activity in all of programming. Sometimes you gain something useful, other times it’s just fun. • Whoever you are, you have something to offer!