Technically, a talk | Ruby Day 2020

by Eileen M. Uchitelle

Slide 1

Slide 1 text

Technically, a Talk

Slide 2

Slide 2 text

Ciao! Sono Eileen M. Uchitelle! Staff Engineer @GitHub Member of Rails Core Find me @eileencodes

Slide 3

Slide 3 text

Technically, a Talk

Slide 4

Slide 4 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38670 #38920 #38684 #38770

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #38770 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #38684 #38670 #38920

Slide 8

Slide 8 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #38684 #38670 #38920 #38770

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

What are multiple databases?

Slide 12

Slide 12 text

Application Setup

Slide 13

Slide 13 text

Design & Architecture

Slide 14

Slide 14 text

What are multiple databases?

Slide 15

Slide 15 text

PRODUCTION APPLICATION Standard setup

Slide 16

Slide 16 text

APPLICATION Horizontal sharding ROWS A-I ROWS J-R ROWS S-Z PRODUCTION

Slide 17

Slide 17 text

APPLICATION Functional partitioning PRODUCTION MEALS PRODUCTION SIDES PRODUCTION DESSERTS

Slide 18

Slide 18 text

APPLICATION Read replicas PRODUCTION REPLICA 1 REPLICA 2 REPLICA 3

Slide 19

Slide 19 text

Application Setup

Slide 20

Slide 20 text

Recipes Application APPLICATION

Slide 21

Slide 21 text

APPLICATION Recipes Application PRIMARY

Slide 22

Slide 22 text

Recipes Application MEALS APPLICATION PRIMARY

Slide 23

Slide 23 text

Recipes Application MEALS APPLICATION PRIMARY REPLICA REPLICAS

Slide 24

Slide 24 text

development: primary: database: recipes_development primary_replica: database: recipes_development replica: true meals_default: database: meals_default_development migrations_paths: db/meals_migrate meals_default_replica: database: meals_default_development replica: true meals_one: database: meals_one_development migrations_paths: db/meals_migrate meals_one_replica: database: meals_one_development replica: true meals_two: database: meals_two_development migrations_paths: db/meals_migrate meals_two_replica: database: meals_two_development replica: true

Slide 25

Slide 25 text

class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end

Slide 26

Slide 26 text

class MealApplicationRecord < ApplicationRecord self.abstract_class = true connects_to shards: { default: { writing: :meals_default, reading: :meals_default_replica }, shard_one: { writing: :meals_one, reading: :meals_one_replica } shard_two: { writing: :meals_two, reading: :meals_two_replica } } end

Slide 27

Slide 27 text

Design & Architecture

Slide 28

Slide 28 text

Slide 29

Slide 29 text

bin/rails g scaffold DinnerRecipe title:string recipe:text --database meals_default

Slide 30

Slide 30 text

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 Database Conﬁgurations #38670 #38920

Slide 33

Slide 33 text

development: database: recipes_development test: database: recipes_test production: database: recipes_production

Slide 34

Slide 34 text

{ "development" => { "database" => "recipes_development" }, "test" => { "database" => "recipes_test" }, "production" => { "database" => "recipes_production" } }

Slide 35

Slide 35 text

development: primary: database: recipes_development primary_replica: database: recipes_development replica: true test: primary: database: recipes_test primary_replica: database: recipes_test replica: true production: primary: database: recipes_production primary_replica: database: recipes_production replica: true

Slide 36

Slide 36 text

{ "development" => { "primary" => { "database" => "recipes_development" }, "primary_replica" => { "database" => "recipes_development", "replica" => true }, } "test" => { "primary" => { "database" => "recipes_test" }, "primary_replica" => { "database" => "recipes_test", "replica" => true }, }, "production" => { "primary" => { "database" => "recipes_production" }, "primary_replica" => { "database" => "recipes_production", "replica" => true }, } }

Slide 37

Slide 37 text

Conﬁguration as Objects

Slide 38

Slide 38 text

# "mysql2", :database => "recipes_app_development" } >

Slide 39

Slide 39 text

DATABASE_URL= "postgres://root:pass123@localhost:9000/my_database"

Slide 40

Slide 40 text

DATABASE_URL="postgres://root:pass123@localhost:9000/my_database" # "postgres", :database => "my_database", :host => "localhost", :username => "root", :password => "pass123" }, @url = "postgres://root:pass123@localhost:9000/my_database" >

Slide 41

Slide 41 text

initializer "active_record.initialize_database" do ActiveSupport.on_load(:active_record) do self.connection_handlers = { writing_role => ActiveRecord::Base.default_connection_handler } self.configurations = Rails.application.config.database_configuration establish_connection end end

Slide 42

Slide 42 text

def self.configurations=(config) @@configurations = ActiveRecord::DatabaseConfigurations.new(config) end module ActiveRecord class DatabaseConfigurations def initialize(configurations = {}) @configurations = build_configs(configurations) end end end

Slide 43

Slide 43 text

# "mysql2", :database => "recipes_app_development" } >, # "mysql2", :database => "meals_one_development" } > ] >

Slide 44

Slide 44 text

ActiveRecord::Base.configurations.configs_for( env_name: "production" )

Slide 45

Slide 45 text

ActiveRecord::Base.configurations.configs_for( env_name: "production" ) # "mysql2", :database => "recipes_app_development" }>, # "mysql2", :database => "meals_one_development" }>, # "mysql2", :database => "meals_two_development" }> ] >

Slide 46

Slide 46 text

ActiveRecord::Base.configurations.configs_for( env_name: "production", name: "meals_one" )

Slide 47

Slide 47 text

ActiveRecord::Base.configurations.configs_for( env_name: "production", name: "meals_one" ) # "mysql2", :database => "meals_one_development" }>

Slide 48

Slide 48 text

Using Objects Everywhere

Slide 49

Slide 49 text

>> db_config.configuration_hash[:database] => "meals_one_development"

Slide 50

Slide 50 text

>> db_config.configuration_hash[:database] => "meals_one_development" >> db_config.database => "meals_one_development"

Slide 51

Slide 51 text

{ "database" => "recipes_development", "replica" => true "adapter" => "mysql2", "reaping_frequency" => 60, "username" => "user_ro", "password" => ENV["PASSWORD"]}

Slide 52

Slide 52 text

{ "database" => "recipes_development", "replica" => true "adapter" => "mysql2", "reaping_frequency" => 60, "username" => "user_ro", "password" => ENV["PASSWORD"]...} Values Rails Needs • database • host • adapter • replica • pool • reaping_frequency • checkout_timeout • idle_timeout

Slide 53

Slide 53 text

{ "database" => "recipes_development", "replica" => true "adapter" => "mysql2", "reaping_frequency" => 60, "username" => "user_ro", "password" => ENV["PASSWORD"]...} • database • host • adapter • replica • pool • reaping_frequency • checkout_timeout • idle_timeout • database • host • username • password • timeout • custom settings Values Rails Needs Values Clients Need

Slide 54

Slide 54 text

class HashConfig ... def checkout_timeout (configuration_hash[:checkout_timeout] || 5).to_f end def reaping_frequency configuration_hash.fetch(:reaping_frequency, 60)&.to_f end def idle_timeout timeout = configuration_hash.fetch(:idle_timeout, 300).to_f timeout if timeout > 0 end ... end

Slide 55

Slide 55 text

- @checkout_timeout = (spec.db_config.configuration_hash[:checkout_timeout] && spec.db_config.configuration_hash[:checkout_timeout].to_f) || 5 - if @idle_timeout = spec.db_config.configuration_hash.fetch(:idle_timeout, 300) - @idle_timeout = @idle_timeout.to_f - @idle_timeout = nil if @idle_timeout <= 0 - end - # default max pool size to 5 - @size = (spec.db_config.configuration_hash[:pool] && spec.db_config.configuration_hash[:pool].to_i) || 5 + @checkout_timeout = spec.db_config.checkout_timeout + @idle_timeout = spec.db_config.idle_timeout + @size = spec.db_config.pool

Slide 56

Slide 56 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 Rails Tasks #38770

Slide 57

Slide 57 text

rails db:create rails db:drop rails db:migrate rails db:reset rails db:abort_if_pending_migrations rails db:migrate:status rails db:structure:dump rails db:schema:cache:dump

Slide 58

Slide 58 text

task migrate: :load_config do original_db_config = ActiveRecord::Base.connection_db_config ActiveRecord::Base.configurations.configs_for( env_name: ActiveRecord::Tasks::DatabaseTasks.env ).each do |db_config| ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate end db_namespace["_dump"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

rails db:create:primary rails db:create:meals_default rails db:create:meals_one rails db:create:meals_two rails db:drop:primary rails db:drop:meals_default rails db:drop:meals_one rails db:drop:meals_two rails db:migrate:primary rails db:drop:meals_default rails db:migrate:meals_one rails db:migrate:meals_two

Slide 64

Slide 64 text

namespace :migrate do ActiveRecord::Tasks::DatabaseTasks.for_each(databases) do |name| desc "Migrate #{name} database for current environment" task name => :load_config do original_db_config = ActiveRecord::Base.connection_db_config db_config = ActiveRecord::Base.configurations.configs_for( env_name: Rails.env, name: name ) ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate db_namespace["_dump:#{name}"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end end end

Slide 65

Slide 65 text

Slide 66

Slide 66 text

def load_database_yaml if path = paths["config/database"].existent.first require "rails/application/dummy_erb_compiler" yaml = Pathname.new(path) erb = DummyERB.new(yaml.read) YAML.load(erb.result) || {} else {} end end

Slide 67

Slide 67 text

class DummyERB < ERB # :nodoc: def make_compiler(trim_mode) DummyCompiler.new trim_mode end end class DummyCompiler < ERB::Compiler # :nodoc: def compile_content(stag, out) if stag == "<%=" out.push "_erbout << ''" end end end

Slide 68

Slide 68 text

Slide 69

Slide 69 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 Connection APIs #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672

Slide 70

Slide 70 text

connects_to API

Slide 71

Slide 71 text

Slide 72

Slide 72 text

class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end

Slide 73

Slide 73 text

class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end Role

Slide 74

Slide 74 text

class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end Name

Slide 75

Slide 75 text

class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Slide 80

Slide 80 text

connected_to API

Slide 81

Slide 81 text

ActiveRecord::Base.connected_to(role: :reading) do User.first DinnerRecipe.first end

Slide 82

Slide 82 text

ActiveRecord::Base.connected_to(role: :reading) do DinnerRecipe.create! # boom end => ActiveRecord::ReadOnlyError

Slide 83

Slide 83 text

ActiveRecord::Base.connected_to(role: :writing, prevent_writes: true) do DinnerRecipe.create! # boom end => ActiveRecord::ReadOnlyError

Slide 84

Slide 84 text

ActiveRecord::Base.connected_to(shard: :shard_two) do DinnerRecipe.create! end

Slide 85

Slide 85 text

ActiveRecord::Base.connected_to( shard: :shard_two, role: :reading ) do DinnerRecipe.create! end

Slide 86

Slide 86 text

:writing

Slide 87

Slide 87 text

:writing ActiveRecord::Base

Slide 88

Slide 88 text

:writing ActiveRecord::Base PoolConﬁg

Slide 89

Slide 89 text

:writing ActiveRecord::Base conn_spec_name PoolConﬁg

Slide 90

Slide 90 text

:writing ActiveRecord::Base conn_spec_name db_conﬁg PoolConﬁg

Slide 91

Slide 91 text

:writing ActiveRecord::Base conn_spec_name db_conﬁg schema_cache PoolConﬁg

Slide 92

Slide 92 text

:writing ActiveRecord::Base conn_spec_name db_conﬁg schema_cache PoolConﬁg ConnectionPool

Slide 93

Slide 93 text

:writing ActiveRecord::Base PoolConﬁg ConnectionPool MealApplicationR... PoolConﬁg ConnectionPool

Slide 94

Slide 94 text

:writing ActiveRecord::Base :writing MealApplicationR... :reading ActiveRecord::Base PoolConfig MealApplicationR... PoolConfig ConnectionPool ConnectionPool PoolConfig ConnectionPool PoolConfig ConnectionPool

Slide 95

Slide 95 text

:writing ActiveRecord::Base MealApplicationR... PoolConﬁg ConnectionPool PoolConﬁg ConnectionPool

Slide 96

Slide 96 text

:writing ActiveRecord::Base PoolConfig MealApplicationR... PoolConfig ConnectionPool ConnectionPool MealApplicationR... PoolConfig ConnectionPool

Slide 97

Slide 97 text

:writing ActiveRecord::Base MealApplicationR... MealApplicationR... PoolConfig ConnectionPool MealApplicationR... PoolConfig ConnectionPool PoolConfig ConnectionPool PoolConfig ConnectionPool

Slide 98

Slide 98 text

:writing ActiveRecord::Base Pool Conﬁg Connection Pool :default PoolConﬁg ConnectionPool ActiveRecord::Base

Slide 99

Slide 99 text

:writing :default PoolConfig ConnectionPool :shard_one PoolConfig ConnectionPool :shard_two PoolConfig ConnectionPool MealApplicationBase ActiveRecord::Base Pool Config Connection Pool :default PoolConfig ActiveRecord::Base ConnectionPool

Slide 100

Slide 100 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #35073 #35130 #35237 #36469 #36834 #36868 #38042 Automatic connection swapping

Slide 101

Slide 101 text

Rails.application.configure do config.active_record.database_selector = { delay: 2.seconds } config.active_record.database_resolver = ActiveRecord::Middleware::DatabaseSelector::Resolver config.active_record.database_resolver_context = ActiveRecord::Middleware::DatabaseSelector::Resolver::Session end

Slide 102

Slide 102 text

class DatabaseSelector def initialize(app, resolver_klass = nil, context_klass = nil, options = {}) @app = app @resolver_klass = resolver_klass || Resolver @context_klass = context_klass || Resolver::Session @options = options end ... end

Slide 103

Slide 103 text

class DatabaseSelector def call(env) request = ActionDispatch::Request.new(env) select_database(request) do @app.call(env) end end ... end

Slide 104

Slide 104 text

class DatabaseSelector def select_database(request, &blk) context = context_klass.call(request) resolver = resolver_klass.call(context, options) response = if reading_request?(request) resolver.read(&blk) else resolver.write(&blk) end resolver.update_context(response) response end end

Slide 105

Slide 105 text

Slide 106

Slide 106 text

class Resolver # :nodoc: ... def read(&blk) if read_from_primary? read_from_primary(&blk) else read_from_replica(&blk) end end end

Slide 107

Slide 107 text

class Resolver ... def read_from_primary? !time_since_last_write_ok? end def time_since_last_write_ok? Time.now - context.last_write_timestamp >= delay end end

Slide 108

Slide 108 text

class Resolver ... def read_from_primary(&blk) ActiveRecord::Base.connected_to( role: ActiveRecord::Base.writing_role, prevent_writes: true ) do instrumenter.instrument("database_selector") do yield end end end end

Slide 109

Slide 109 text

class Resolver ... def read_from_replica(&blk) ActiveRecord::Base.connected_to( role: ActiveRecord::Base.reading_role, prevent_writes: true ) do instrumenter.instrument("database_selector") do yield end end end end

Slide 110

Slide 110 text

class Resolver ... def write_to_primary(&blk) ActiveRecord::Base.connected_to( role: ActiveRecord::Base.writing_role, prevent_writes: false ) do instrumenter.instrument("database_selector") do yield ensure context.update_last_write_timestamp end end end end

Slide 111

Slide 111 text

class Resolver class Cookie def self.call(request) new(request.cookies) end def initialize(cookies) @cookies = cookies end def update_last_write_timestamp @cookies[:last_write] = self.class.convert_time_to_timestamp(Time.now) end end end

Slide 112

Slide 112 text

Slide 113

Slide 113 text

Thank You ❤

Slide 114

Slide 114 text

@tenderlove @seejohnrun

Slide 115

Slide 115 text

And all of you

Slide 116

Slide 116 text

Extracting to Upstream a

Slide 117

Slide 117 text

No content

Slide 118

Slide 118 text

No content

Slide 119

Slide 119 text

Migrations didn’t run,

Slide 120

Slide 120 text

Migrations didn’t run, conﬁgurations were broken,

Slide 121

Slide 121 text

Rails tasks didn’t work correctly, Migrations didn’t run, conﬁgurations were broken,

Slide 122

Slide 122 text

Migrations didn’t run, conﬁgurations were broken, Rails tasks didn’t work correctly, no way for models to establish multiple connections,

Slide 123

Slide 123 text

no API or pattern for auto-switching connections... Migrations didn’t run, conﬁgurations were broken, Rails tasks didn’t work correctly, no way for models to establish multiple connections,

Slide 124

Slide 124 text

No content

Slide 125

Slide 125 text

Slide 126

Slide 126 text

Slide 127

Slide 127 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #38770 #38684 #38670 #38920

Slide 128

Slide 128 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38684 #38670 #38920 #38770

Slide 129

Slide 129 text

Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #38684 #38670 #38920 #38770 #38581 #38672