ABExperimentBuilder Applying the Module Builder Pattern Bradley.Schaefer@stitchfix.com @soulcutter 

A/B Experiments The Way Things Were 

A/B Experiments The Way Things Were class BodyShape attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

A/B Experiments The Way Things Were class ItemTypeGroupPreferences attr_reader :api, :client NAME = 'New Womens Item Type Group Preferences' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_new_womens_item_type_group_preferences_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

A/B Experiments The Way Things Were class PrimaryObjectives attr_reader :api, :client NAME = 'New Womens Primary Reasons Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_primary_objectives_question? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

A/B Experiments The Way Things Were class WomensBrands attr_reader :api, :client NAME = 'New Womens Brands Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_womens_brands_question? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

A/B Experiments The Way Things Were class BottomsAttributePreferences attr_reader :api, :client NAME = 'New Womens Bottoms Attribute Preferences' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_new_bottoms_attribute_preferences_question? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

What Stands Out? 

What Stands Out? class BodyShape attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new( experiment_name: NAME, test_subject: client ) @client = client end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end class PrimaryObjectives attr_reader :api, :client NAME = 'New Womens Primary Reasons Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new( experiment_name: NAME, test_subject: client ) @client = client end def show_primary_objectives_question? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Duplication! class BodyShape attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new( experiment_name: NAME, test_subject: client ) @client = client end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end class PrimaryObjectives attr_reader :api, :client NAME = 'New Womens Primary Reasons Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new( experiment_name: NAME, test_subject: client ) @client = client end def show_primary_objectives_question? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Removing Duplication def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end 

Removing Duplication module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end end 

Removing Duplication class BodyShape attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Replace all initializers class BodyShape include ABExperiment attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Looking better! class BodyShape include ABExperiment attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

But wait… there's a problem module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end end 

But wait… there's a problem module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end end Let's run the tests… 

But wait… there's a problem module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end end Failure/Error: @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) NameError: uninitialized constant ABExperiment::NAME 

But wait… there's a problem module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end end Need a way to look up a constant in the correct scope 

Scope? Credit: Craig McMillan 2008 https://mccraigmccraig.wordpress.com/2008/10/29/ruby-objects-classes-and-eigenclasses/ 

But wait… there's a problem module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) @client = client end end Failure/Error: @api = StitchFix::Experiments::API.new(experiment_name: NAME, test_subject: client) NameError: uninitialized constant ABExperiment::NAME 

const_get does the trick module ABExperiment def initialize(client) @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) @client = client end end 

What stands out now? class BodyShape include ABExperiment attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

What stands out now? class BodyShape include ABExperiment attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Encapsulate api/client module ABExperiment attr_reader :api, :client def initialize(client) @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) @client = client end end 

Encapsulate api/client class BodyShape include ABExperiment attr_reader :api, :client NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Looking even better! class BodyShape include ABExperiment NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

What stands out now? class BodyShape include ABExperiment NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

What stands out now? class BodyShape include ABExperiment NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def self.define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

We can extract that too! module ABExperiment extend ActiveSupport::Concern # ... module ClassMethods def define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end end end 

Oh yeah, I've seen this before module ABExperiment extend ActiveSupport::Concern # ... module ClassMethods def define! StitchFix::Experiments::API.define_experiment do name NAME treatment CONTROL_CELL, control: true treatment TEST_CELL allocation Time.now..6.months.from_now live true owner Client.first end end end end 

Oh yeah, I've seen this before module ABExperiment extend ActiveSupport::Concern # ... module ClassMethods def define! StitchFix::Experiments::API.define_experiment do name const_get(:NAME) treatment const_get(:CONTROL_CELL), control: true treatment const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end end 

Oh yeah, I've seen this before module ABExperiment extend ActiveSupport::Concern # ... module ClassMethods def define! StitchFix::Experiments::API.define_experiment do name const_get(:NAME) treatment const_get(:CONTROL_CELL), control: true treatment const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end end Let's run the tests… 

This is new… Failure/Error: name const_get(:NAME) NoMethodError: undefined method `const_get' for # 

Not the expected scope Failure/Error: name const_get(:NAME) NoMethodError: undefined method `const_get' for # def define! StitchFix::Experiments::API.define_experiment do name const_get(:NAME) treatment const_get(:CONTROL_CELL), control: true treatment const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end 

Let's fix that Failure/Error: name const_get(:NAME) NoMethodError: undefined method `const_get' for # def define! klass = self StitchFix::Experiments::API.define_experiment do name klass.const_get(:NAME) treatment klass.const_get(:CONTROL_CELL), control: true treatment klass.const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end 

module ABExperiment extend ActiveSupport::Concern included do attr_reader :client, :api end def initialize(client) @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end module ClassMethods def define! klass = self StitchFix::Experiments::API.define_experiment do name klass.const_get(:NAME) treatment klass.const_get(:CONTROL_CELL), control: true treatment klass.const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end end 

Looking really good! class BodyShape include ABExperiment NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

But Wait, There's MORE! 

There's something still bothering me class BodyShape include ABExperiment NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

An Encapsulation Problem class BodyShape include ABExperiment NAME = 'New Womens Body Shape Question' CONTROL_CELL = 'Control' TEST_CELL = 'New Question' def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Wouldn't this be nice? class BodyShape include ABExperimentBuilder.new( name: 'New Womens Body Shape Question', control_cell: 'Control', test_cell: 'New Question' ) def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

class ABExperimentBuilder < Module def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end module ClassMethods def define! klass = self StitchFix::Experiments::API.define_experiment do name klass.const_get(:NAME) treatment klass.const_get(:CONTROL_CELL), control: true treatment klass.const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end def included(base) base.extend ClassMethods end end 

We did it! 

class ABExperimentBuilder < Module def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end module ClassMethods def define! klass = self StitchFix::Experiments::API.define_experiment do name klass.const_get(:NAME) treatment klass.const_get(:CONTROL_CELL), control: true treatment klass.const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end def included(base) base.extend ClassMethods end end 

Extending Module? class ABExperimentBuilder < Module 

Module is a Class module ABExperiment # … end ABExperiment = Module.new do # … end Module.class # => Class 

class ABExperimentBuilder < Module def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end module ClassMethods def define! klass = self StitchFix::Experiments::API.define_experiment do name klass.const_get(:NAME) treatment klass.const_get(:CONTROL_CELL), control: true treatment klass.const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end def included(base) base.extend ClassMethods end end 

ABExperimentBuilder builds Modules def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end 

ABExperimentBuilder builds Modules def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end 

ABExperimentBuilder builds Modules def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end 

ABExperimentBuilder builds Modules def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end 

ABExperimentBuilder builds Modules def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end ABExperimentBuilder.new( name: 'New Womens Body Shape Question', test_cell: 'New Question' ) 

This is what a built module would look like module ABExperimentTemplate NAME = 'Foo' CONTROL_CELL = 'Control' TEST_CELL = 'Test' attr_reader :api, :client def initialize(client) @client = client @api = StitchFix::Experiments::API.new( experiment_name: NAME, test_subject: client ) end end ABExperimentBuilder.new( name: 'Foo', test_cell: 'Test' ) { 

class ABExperimentBuilder < Module def initialize(name:, test_cell:, control_cell: 'Control') const_set(:NAME, name) const_set(:CONTROL_CELL, control_cell) const_set(:TEST_CELL, test_cell) attr_reader :api, :client define_method(:initialize) do |client| @client = client @api = StitchFix::Experiments::API.new( experiment_name: self.class.const_get(:NAME), test_subject: client ) end end module ClassMethods def define! klass = self StitchFix::Experiments::API.define_experiment do name klass.const_get(:NAME) treatment klass.const_get(:CONTROL_CELL), control: true treatment klass.const_get(:TEST_CELL) allocation Time.now..6.months.from_now live true owner Client.first end end end def included(base) base.extend ClassMethods end end 

You don't need a Concern module ClassMethods # … end def included(base) base.extend ClassMethods end 

The Final Result class BodyShape include ABExperimentBuilder.new( name: 'New Womens Body Shape Question', control_cell: 'Control', test_cell: 'New Question' ) def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

The Interesting Bits, Front and Center class BodyShape include ABExperimentBuilder.new( name: 'New Womens Body Shape Question', control_cell: 'Control', test_cell: 'New Question' ) def show_new_body_shape_questions? api.allocating?(time: client.create_date) && api.in_cell?(cell_name: TEST_CELL) end end 

Tradeoffs Good Bad Experiment Class Readability I wrote a talk to explain the code in ABExperimentBuilder Can test module builder code and be confident it works across all experiments Too much magic? It's a neat Ruby technique metaprogramming: bad Didn't change tests/interface Changing interface potentially valuable 

Credits Chris Salzberg http://dejimata.com/2017/5/20/the-ruby-module-builder-pattern Gal Steinitz 

Questions? 

ABExperimentBuilder Applying the Module Builder Pattern Bradley.Schaefer@stitchfix.com @soulcutter