Technically, a talk | Ruby Day 2020

Peer deep into Rails' database handling and you may find the code overly complex, hard to follow, and full of technical debt. On the surface you're right - it is complex, but that complexity represents the strong foundation that keeps your applications simple and focused on your product code. In this talk we'll look at how to use multiple databases, the beauty (and horror) of Rails connection management, and why we built this feature for you.

Transcript

1. Technically, a Talk
2. Ciao! Sono Eileen M. Uchitelle! Staff Engineer @GitHub Member of Rails Core Find me @eileencodes
3. Technically, a Talk
4. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38670 #38920 #38684 #38770
5. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #38684
6. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #38670 #38920 #38684
7. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #38770 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #38684 #38670 #38920
8. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #38684 #38670 #38920 #38770
9. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38684 #38670 #38920 #38770 #38581 #38672
10. a
11. What are multiple databases?
12. Application Setup
13. Design & Architecture
14. What are multiple databases?
15. PRODUCTION APPLICATION Standard setup
16. APPLICATION Horizontal sharding ROWS A-I ROWS J-R ROWS S-Z PRODUCTION
17. APPLICATION Functional partitioning PRODUCTION MEALS PRODUCTION SIDES PRODUCTION DESSERTS
18. APPLICATION Read replicas PRODUCTION REPLICA 1 REPLICA 2 REPLICA 3
19. Application Setup
20. Recipes Application APPLICATION
21. APPLICATION Recipes Application PRIMARY
22. Recipes Application MEALS APPLICATION PRIMARY
23. Recipes Application MEALS APPLICATION PRIMARY REPLICA REPLICAS
24. development: primary: database: recipes_development primary_replica: database: recipes_development replica: true meals_default: database: meals_default_development migrations_paths: db/meals_migrate meals_default_replica: database: meals_default_development replica: true meals_one: database: meals_one_development migrations_paths: db/meals_migrate meals_one_replica: database: meals_one_development replica: true meals_two: database: meals_two_development migrations_paths: db/meals_migrate meals_two_replica: database: meals_two_development replica: true
25. class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end
26. class MealApplicationRecord < ApplicationRecord self.abstract_class = true connects_to shards: { default: { writing: :meals_default, reading: :meals_default_replica }, shard_one: { writing: :meals_one, reading: :meals_one_replica } shard_two: { writing: :meals_two, reading: :meals_two_replica } } end
27. Design & Architecture
28. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 Migrations #38684
29. bin/rails g scaffold DinnerRecipe title:string recipe:text --database meals_default
30. development: primary: database: recipes_development primary_replica: database: recipes_development replica: true meals_default: database: meals_default_development migrations_paths: db/meals_migrate meals_default_replica: database: meals_default_development replica: true meals_one: database: meals_one_development migrations_paths: db/meals_migrate meals_one_replica: database: meals_one_development replica: true meals_two: database: meals_two_development migrations_paths: db/meals_migrate meals_two_replica: database: meals_two_development replica: true
32. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 Database Configurations #38670 #38920
33. development: database: recipes_development test: database: recipes_test production: database: recipes_production
34. { "development" => { "database" => "recipes_development" }, "test" => { "database" => "recipes_test" }, "production" => { "database" => "recipes_production" } }
35. development: primary: database: recipes_development primary_replica: database: recipes_development replica: true test: primary: database: recipes_test primary_replica: database: recipes_test replica: true production: primary: database: recipes_production primary_replica: database: recipes_production replica: true
36. { "development" => { "primary" => { "database" => "recipes_development" }, "primary_replica" => { "database" => "recipes_development", "replica" => true }, } "test" => { "primary" => { "database" => "recipes_test" }, "primary_replica" => { "database" => "recipes_test", "replica" => true }, }, "production" => { "primary" => { "database" => "recipes_production" }, "primary_replica" => { "database" => "recipes_production", "replica" => true }, } }
37. Configuration as Objects
38. #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "primary", @configuration_hash = { :adapter => "mysql2", :database => "recipes_app_development" } >
39. DATABASE_URL= "postgres://root:pass123@localhost:9000/my_database"
40. DATABASE_URL="postgres://root:pass123@localhost:9000/my_database" #<ActiveRecord::DatabaseConfigurations::UrlConfig @env_name = "production", @name = "primary", @configuration_hash = { :adapter => "postgres", :database => "my_database", :host => "localhost", :username => "root", :password => "pass123" }, @url = "postgres://root:pass123@localhost:9000/my_database" >
41. initializer "active_record.initialize_database" do ActiveSupport.on_load(:active_record) do self.connection_handlers = { writing_role => ActiveRecord::Base.default_connection_handler } self.configurations = Rails.application.config.database_configuration establish_connection end end
42. def self.configurations=(config) @@configurations = ActiveRecord::DatabaseConfigurations.new(config) end module ActiveRecord class DatabaseConfigurations def initialize(configurations = {}) @configurations = build_configs(configurations) end end end
43. #<ActiveRecord::DatabaseConfigurations @configurations=[ #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "primary", @configuration_hash = { :adapter => "mysql2", :database => "recipes_app_development" } >, #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "meals_one", @configuration_hash = { :adapter => "mysql2", :database => "meals_one_development" } > ] >
44. ActiveRecord::Base.configurations.configs_for( env_name: "production" )
45. ActiveRecord::Base.configurations.configs_for( env_name: "production" ) #<ActiveRecord::DatabaseConfigurations @configurations=[ #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "primary", @configuration_hash = { :adapter => "mysql2", :database => "recipes_app_development" }>, #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "meals_one", @configuration_hash = { :adapter => "mysql2", :database => "meals_one_development" }>, #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "meals_two", @configuration_hash = { :adapter => "mysql2", :database => "meals_two_development" }> ] >
46. ActiveRecord::Base.configurations.configs_for( env_name: "production", name: "meals_one" )
47. ActiveRecord::Base.configurations.configs_for( env_name: "production", name: "meals_one" ) #<ActiveRecord::DatabaseConfigurations::HashConfig @env_name = "production", @name = "meals_one", @configuration_hash = { :adapter => "mysql2", :database => "meals_one_development" }>
48. Using Objects Everywhere
49. >> db_config.configuration_hash[:database] => "meals_one_development"
50. >> db_config.configuration_hash[:database] => "meals_one_development" >> db_config.database => "meals_one_development"
51. { "database" => "recipes_development", "replica" => true "adapter" => "mysql2", "reaping_frequency" => 60, "username" => "user_ro", "password" => ENV["PASSWORD"]}
52. { "database" => "recipes_development", "replica" => true "adapter" => "mysql2", "reaping_frequency" => 60, "username" => "user_ro", "password" => ENV["PASSWORD"]...} Values Rails Needs • database • host • adapter • replica • pool • reaping_frequency • checkout_timeout • idle_timeout
53. { "database" => "recipes_development", "replica" => true "adapter" => "mysql2", "reaping_frequency" => 60, "username" => "user_ro", "password" => ENV["PASSWORD"]...} • database • host • adapter • replica • pool • reaping_frequency • checkout_timeout • idle_timeout • database • host • username • password • timeout • custom settings Values Rails Needs Values Clients Need
54. class HashConfig ... def checkout_timeout (configuration_hash[:checkout_timeout] || 5).to_f end def reaping_frequency configuration_hash.fetch(:reaping_frequency, 60)&.to_f end def idle_timeout timeout = configuration_hash.fetch(:idle_timeout, 300).to_f timeout if timeout > 0 end ... end
55. - @checkout_timeout = (spec.db_config.configuration_hash[:checkout_timeout] && spec.db_config.configuration_hash[:checkout_timeout].to_f) || 5 - if @idle_timeout = spec.db_config.configuration_hash.fetch(:idle_timeout, 300) - @idle_timeout = @idle_timeout.to_f - @idle_timeout = nil if @idle_timeout <= 0 - end - # default max pool size to 5 - @size = (spec.db_config.configuration_hash[:pool] && spec.db_config.configuration_hash[:pool].to_i) || 5 + @checkout_timeout = spec.db_config.checkout_timeout + @idle_timeout = spec.db_config.idle_timeout + @size = spec.db_config.pool
56. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 Rails Tasks #38770
57. rails db:create rails db:drop rails db:migrate rails db:reset rails db:abort_if_pending_migrations rails db:migrate:status rails db:structure:dump rails db:schema:cache:dump
58. task migrate: :load_config do original_db_config = ActiveRecord::Base.connection_db_config ActiveRecord::Base.configurations.configs_for( env_name: ActiveRecord::Tasks::DatabaseTasks.env ).each do |db_config| ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate end db_namespace["_dump"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end
59. task migrate: :load_config do original_db_config = ActiveRecord::Base.connection_db_config ActiveRecord::Base.configurations.configs_for( env_name: ActiveRecord::Tasks::DatabaseTasks.env ).each do |db_config| ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate end db_namespace["_dump"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end
60. task migrate: :load_config do original_db_config = ActiveRecord::Base.connection_db_config ActiveRecord::Base.configurations.configs_for( env_name: ActiveRecord::Tasks::DatabaseTasks.env ).each do |db_config| ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate end db_namespace["_dump"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end
61. task migrate: :load_config do original_db_config = ActiveRecord::Base.connection_db_config ActiveRecord::Base.configurations.configs_for( env_name: ActiveRecord::Tasks::DatabaseTasks.env ).each do |db_config| ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate end db_namespace["_dump"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end
62. task migrate: :load_config do original_db_config = ActiveRecord::Base.connection_db_config ActiveRecord::Base.configurations.configs_for( env_name: ActiveRecord::Tasks::DatabaseTasks.env ).each do |db_config| ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate end db_namespace["_dump"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end
63. rails db:create:primary rails db:create:meals_default rails db:create:meals_one rails db:create:meals_two rails db:drop:primary rails db:drop:meals_default rails db:drop:meals_one rails db:drop:meals_two rails db:migrate:primary rails db:drop:meals_default rails db:migrate:meals_one rails db:migrate:meals_two
64. namespace :migrate do ActiveRecord::Tasks::DatabaseTasks.for_each(databases) do |name| desc "Migrate #{name} database for current environment" task name => :load_config do original_db_config = ActiveRecord::Base.connection_db_config db_config = ActiveRecord::Base.configurations.configs_for( env_name: Rails.env, name: name ) ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate db_namespace["_dump:#{name}"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end end end
65. namespace :migrate do ActiveRecord::Tasks::DatabaseTasks.for_each(databases) do |name| desc "Migrate #{name} database for current environment" task name => :load_config do original_db_config = ActiveRecord::Base.connection_db_config db_config = ActiveRecord::Base.configurations.configs_for( env_name: Rails.env, name: name ) ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate db_namespace["_dump:#{name}"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end end end
66. def load_database_yaml if path = paths["config/database"].existent.first require "rails/application/dummy_erb_compiler" yaml = Pathname.new(path) erb = DummyERB.new(yaml.read) YAML.load(erb.result) || {} else {} end end
67. class DummyERB < ERB # :nodoc: def make_compiler(trim_mode) DummyCompiler.new trim_mode end end class DummyCompiler < ERB::Compiler # :nodoc: def compile_content(stag, out) if stag == "<%=" out.push "_erbout << ''" end end end
68. namespace :migrate do ActiveRecord::Tasks::DatabaseTasks.for_each(databases) do |name| desc "Migrate #{name} database for current environment" task name => :load_config do original_db_config = ActiveRecord::Base.connection_db_config db_config = ActiveRecord::Base.configurations.configs_for( env_name: Rails.env, name: name ) ActiveRecord::Base.establish_connection(db_config) ActiveRecord::Tasks::DatabaseTasks.migrate db_namespace["_dump:#{name}"].invoke ensure ActiveRecord::Base.establish_connection(original_db_config) end end end
69. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 Connection APIs #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672
70. connects_to API
71. initializer "active_record.initialize_database" do ActiveSupport.on_load(:active_record) do self.connection_handlers = { writing_role => ActiveRecord::Base.default_connection_handler } self.configurations = Rails.application.config.database_configuration establish_connection end end
72. class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end
73. class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end Role
74. class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end Name
75. class ApplicationRecord < ActiveRecord::Base self.abstract_class = true connects_to database: { writing: :primary, reading: :primary_replica } end
76. class MealApplicationRecord < ApplicationRecord self.abstract_class = true connects_to shards: { default: { writing: :meals_default, reading: :meals_default_replica }, shard_one: { writing: :meals_one, reading: :meals_one_replica } shard_two: { writing: :meals_two, reading: :meals_two_replica } } end
77. class MealApplicationRecord < ApplicationRecord self.abstract_class = true connects_to shards: { default: { writing: :meals_default, reading: :meals_default_replica }, shard_one: { writing: :meals_one, reading: :meals_one_replica } shard_two: { writing: :meals_two, reading: :meals_two_replica } } end Shard key
78. class MealApplicationRecord < ApplicationRecord self.abstract_class = true connects_to shards: { default: { writing: :meals_default, reading: :meals_default_replica }, shard_one: { writing: :meals_one, reading: :meals_one_replica } shard_two: { writing: :meals_two, reading: :meals_two_replica } } end Role
79. class MealApplicationRecord < ApplicationRecord self.abstract_class = true connects_to shards: { default: { writing: :meals_default, reading: :meals_default_replica }, shard_one: { writing: :meals_one, reading: :meals_one_replica } shard_two: { writing: :meals_two, reading: :meals_two_replica } } end Name
80. connected_to API
81. ActiveRecord::Base.connected_to(role: :reading) do User.first DinnerRecipe.first end
82. ActiveRecord::Base.connected_to(role: :reading) do DinnerRecipe.create! # boom end => ActiveRecord::ReadOnlyError
83. ActiveRecord::Base.connected_to(role: :writing, prevent_writes: true) do DinnerRecipe.create! # boom end => ActiveRecord::ReadOnlyError
84. ActiveRecord::Base.connected_to(shard: :shard_two) do DinnerRecipe.create! end
85. ActiveRecord::Base.connected_to( shard: :shard_two, role: :reading ) do DinnerRecipe.create! end
86. :writing
87. :writing ActiveRecord::Base
88. :writing ActiveRecord::Base PoolConfig
89. :writing ActiveRecord::Base conn_spec_name PoolConfig
90. :writing ActiveRecord::Base conn_spec_name db_config PoolConfig
91. :writing ActiveRecord::Base conn_spec_name db_config schema_cache PoolConfig
92. :writing ActiveRecord::Base conn_spec_name db_config schema_cache PoolConfig ConnectionPool
93. :writing ActiveRecord::Base PoolConfig ConnectionPool MealApplicationR... PoolConfig ConnectionPool
94. :writing ActiveRecord::Base :writing MealApplicationR... :reading ActiveRecord::Base PoolConfig MealApplicationR... PoolConfig ConnectionPool ConnectionPool PoolConfig ConnectionPool PoolConfig ConnectionPool
95. :writing ActiveRecord::Base MealApplicationR... PoolConfig ConnectionPool PoolConfig ConnectionPool
96. :writing ActiveRecord::Base PoolConfig MealApplicationR... PoolConfig ConnectionPool ConnectionPool MealApplicationR... PoolConfig ConnectionPool
97. :writing ActiveRecord::Base MealApplicationR... MealApplicationR... PoolConfig ConnectionPool MealApplicationR... PoolConfig ConnectionPool PoolConfig ConnectionPool PoolConfig ConnectionPool
98. :writing ActiveRecord::Base Pool Config Connection Pool :default PoolConfig ConnectionPool ActiveRecord::Base
99. :writing :default PoolConfig ConnectionPool :shard_one PoolConfig ConnectionPool :shard_two PoolConfig ConnectionPool MealApplicationBase ActiveRecord::Base Pool Config Connection Pool :default PoolConfig ActiveRecord::Base ConnectionPool
100. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #35073 #35130 #35237 #36469 #36834 #36868 #38042 Automatic connection swapping
101. Rails.application.configure do config.active_record.database_selector = { delay: 2.seconds } config.active_record.database_resolver = ActiveRecord::Middleware::DatabaseSelector::Resolver config.active_record.database_resolver_context = ActiveRecord::Middleware::DatabaseSelector::Resolver::Session end
102. class DatabaseSelector def initialize(app, resolver_klass = nil, context_klass = nil, options = {}) @app = app @resolver_klass = resolver_klass || Resolver @context_klass = context_klass || Resolver::Session @options = options end ... end
103. class DatabaseSelector def call(env) request = ActionDispatch::Request.new(env) select_database(request) do @app.call(env) end end ... end
104. class DatabaseSelector def select_database(request, &blk) context = context_klass.call(request) resolver = resolver_klass.call(context, options) response = if reading_request?(request) resolver.read(&blk) else resolver.write(&blk) end resolver.update_context(response) response end end
105. class DatabaseSelector def select_database(request, &blk) context = context_klass.call(request) resolver = resolver_klass.call(context, options) response = if reading_request?(request) resolver.read(&blk) else resolver.write(&blk) end resolver.update_context(response) response end end
106. class Resolver # :nodoc: ... def read(&blk) if read_from_primary? read_from_primary(&blk) else read_from_replica(&blk) end end end
107. class Resolver ... def read_from_primary? !time_since_last_write_ok? end def time_since_last_write_ok? Time.now - context.last_write_timestamp >= delay end end
108. class Resolver ... def read_from_primary(&blk) ActiveRecord::Base.connected_to( role: ActiveRecord::Base.writing_role, prevent_writes: true ) do instrumenter.instrument("database_selector") do yield end end end end
109. class Resolver ... def read_from_replica(&blk) ActiveRecord::Base.connected_to( role: ActiveRecord::Base.reading_role, prevent_writes: true ) do instrumenter.instrument("database_selector") do yield end end end end
110. class Resolver ... def write_to_primary(&blk) ActiveRecord::Base.connected_to( role: ActiveRecord::Base.writing_role, prevent_writes: false ) do instrumenter.instrument("database_selector") do yield ensure context.update_last_write_timestamp end end end end
111. class Resolver class Cookie def self.call(request) new(request.cookies) end def initialize(cookies) @cookies = cookies end def update_last_write_timestamp @cookies[:last_write] = self.class.convert_time_to_timestamp(Time.now) end end end
112. Rails.application.configure do config.active_record.database_selector = { delay: 2.seconds } config.active_record.database_resolver = ActiveRecord::Middleware::DatabaseSelector::Resolver config.active_record.database_resolver_context = ActiveRecord::Middleware::DatabaseSelector::Resolver::Cookies end
113. Thank You ❤
114. @tenderlove @seejohnrun
115. And all of you
116. Extracting to Upstream a
119. Migrations didn’t run,
120. Migrations didn’t run, configurations were broken,
121. Rails tasks didn’t work correctly, Migrations didn’t run, configurations were broken,
122. Migrations didn’t run, configurations were broken, Rails tasks didn’t work correctly, no way for models to establish multiple connections,
123. no API or pattern for auto-switching connections... Migrations didn’t run, configurations were broken, Rails tasks didn’t work correctly, no way for models to establish multiple connections,
125. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #38684
126. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #38670 #38920 #38684
127. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #38770 #38684 #38670 #38920
128. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38684 #38670 #38920 #38770
129. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #38684 #38670 #38920 #38770 #38581 #38672
130. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38670 #38920 #38684 #38770 Public APIs / User Experience
131. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38670 #38920 #38684 #38770 Private APIs / Internals
132. Rails already has a strong foundation
133. Don't build for perfect, imaginary cases
134. We built Rails for you
135. We optimize for your happiness
136. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38670 #38920 #38684 #38770
137. Technically, a Talk
138. But actually, a love letter
139. Jan 18 Feb 18 Mar 18 Apr 18 May 18 Jun 18 Jul 18 Aug 18 Sep 18 Oct 18 Nov 18 Dec 18 Jan 19 Feb 19 Mar 19 Apr 19 May 19 Jun 19 Jul 19 Aug 19 Sep 19 Oct 19 Nov 19 Dec 19 Jan 20 Feb 20 Mar 20 Apr 20 May 20 #31727 #33760 #36439 #36371 #36886 #38235 #38444 #32075 #32271 #32396 #32274 #33637 #33770 #34495 #34969 #35102 #35242 #36560 #36565 #36756 #36770 #36814 #37185 #37182 #37231 #37276 #37277 #37279 #37280 #37695 #37873 #37963 #38008 #38004 #34495 #38536 #38609 #32774 #33748 #35479 #35663 #36023 #36039 #36237 *d83ab #36824 #37223 #37242 #37291 #37230 #37365 *48d58 *0a353 #35073 #35130 #35237 #36469 #36834 #36868 #38042 #37298 #37364 #37368 #37443 #38011 #37280 #38209 #33877 #34052 #34505 #34054 #34491 #34632 #34753 #35042 #35123 #35089 #35899 #36394 #37065 #37180 #37199 #37503 #37296 #37408 #37622 #37874 #38258 #38339 #39190 #38531 #38580 #38581 #38672 #38670 #38920 #38684 #38770 ❤