Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Developer happiness with DB Charmer

89e0c5e7bfe1c613b1b9287d89963e73?s=47 Lance Gleason
February 28, 2014

Developer happiness with DB Charmer

Presentation by Lance Gleason from Polyglot Programming Inc at the Atlanta Ruby Meetup and Jozi.rb in Johannesburg South Africa.

http://www.polyglotprogramminginc.com
http://www.polyglotprogramminginc.com/charming-large-databases-with-octopuses/

89e0c5e7bfe1c613b1b9287d89963e73?s=128

Lance Gleason

February 28, 2014
Tweet

Transcript

  1. None
  2. Introductions

  3. Twitter @lgleasain Github lgleasain ! www.lancegleason.com www.polyglotprogrammincinc.com lgleason@polyglotprogramminginc.com !

  4. None
  5. None
  6. None
  7. None
  8. http://www.polyglotprogramminginc.com/purr- programming-2-0/

  9. www.purrprogramming.com

  10. ?

  11. None
  12. None
  13. None
  14. None
  15. None
  16. None
  17. None
  18. None
  19. None
  20. None
  21. ?

  22. Scaling

  23. None
  24. None
  25. None
  26. Supporting More Users

  27. SQL

  28. None
  29. Optimized Queries

  30. None
  31. Replication

  32. None
  33. Sharding

  34. None
  35. How do you support this with ActiveRecord?

  36. DB-Charmer

  37. None
  38. production:      blah:          adapter:  mysql

             username:  blah          host:  blah.local          database:  blah   !    foo:          adapter:  mysql          username:  foo          host:  foo.local          database:  foo  
  39. class  Foo  <  ActiveRecord::Model;  end   ! Foo.switch_connection_to(:blah)   Foo.switch_connection_to('foo')

      Foo.switch_connection_to(Bar)   Foo.switch_connection_to(Baz.connectio n)   Foo.switch_connection_to(nil)  
  40. Migrations

  41. class  MultiDbTest  <  ActiveRecord::Migration        db_magic  :connection  =>

     :second_db   !      def  self.up            create_table  :test_table,  :force  =>  true  do  |t|              t.string  :test_string                t.timestamps            end        end   !      def  self.down            drop_table  :test_table        end    end  
  42. class  MultiDbTest  <  ActiveRecord::Migration      def  self.up    

         on_db  :second_db  do              create_table  :test_table,  :force  =>  true  do   |t|                  t.string  :test_string                  t.timestamps              end          end      end   !    def  self.down          on_db  :second_db  {  drop_table  :test_table  }      end   end  
  43. class  MultiDbTest  <  ActiveRecord::Migration      db_magic  :connections  =>  [:second_db,

     :default]   !    def  self.up          create_table  :test_table,  :force  =>  true  do  |t|              t.string  :test_string              t.timestamps          end      end   !    def  self.down          drop_table  :test_table      end   end  
  44. Replication

  45. Auto Switch All Connections To Slaves

  46. class  Foo  <  ActiveRecord::Base      db_magic  :slave  =>  :slave01

      end   ! class  Bar  <  ActiveRecord::Base      db_magic  :slaves  =>   [  :slave01,  :slave02  ]   end  
  47. Default Master Slave Class Connections

  48. class  Foo  <  ActiveRecord::Base      db_magic  :connection  =>  :foo

      end   class  Bar  <  ActiveRecord::Base      db_magic  :connection  =>  :bar,  :slave   =>  :bar_slave   end  
  49. Per Query Connection Management

  50. class  Foo  <  ActiveRecord::Model;  end   ! Foo.switch_connection_to(:blah)   Foo.switch_connection_to('foo')

      Foo.switch_connection_to(Bar)   Foo.switch_connection_to(Baz.connectio n)   Foo.switch_connection_to(nil)  
  51. User.on_master  do      user  =  User.find_by_login('foo')      user.update_attributes!(:activated

      =>  true)   end  
  52. Comment.on_master.last(:limit  =>  5)   User.on_master.find_by_activation_code(code)   User.on_master.exists?(:login  =>  login,  :password

      =>  password)  
  53. Comment.on_db(:olap).count   Post.on_db(:foo).find(:first)  

  54. Sharding

  55. Sharding • Range

  56. Sharding • Range • hash_map

  57. Sharding • Range • hash_map • db_block_map

  58. Sharding • Range • hash_map • db_block_map • db_block_group_map

  59. Custom Sharding Methods

  60. None
  61. Is it thread safe?

  62. Sort of

  63. Testing

  64. No transaction rollbacks

  65. Limitations

  66. None
  67. •MySQL Only

  68. •MySQL Only •Doesn’t Support Rails 4

  69. None
  70. None
  71. •Supports Postgres and MySQL

  72. •Supports Postgres and MySQL •Includes migrations for moving data between

    shards
  73. octopus:      replicated:  true      production:    

         slave1:              adapter:  mysql              host:  localhost              database:  octopus_shard2          slave2:              adapter:  mysql              host:  localhost              database:  octopus_shard3          slave3:              adapter:  mysql              host:  localhost              database:  octopus_shard4          slave4:              adapter:  mysql              host:  localhost              database:  octopus_shard5  
  74. User.where(:name  =>   "Thiago").limit(3).using(:slave_one)   Octopus.using(:slave_two)  do      User.create(:name

     =>  "Mike")   end  
  75. class  CreateUsersOnBothShards  <   ActiveRecord::Migration      using(:brazil,  :canada)  

    !    def  self.up          User.create!(:name  =>  "Both")      end   !    def  self.down          User.delete_all      end   end  
  76. class  ApplicationController  <   ActionController::Base      around_filter  :select_shard  

    !    def  select_shard(&block)          Octopus.using(:brazil,  &block)      end   end  
  77. Which One?

  78. None
  79. Alternatives

  80. Amazon Cloud Search

  81. Asari

  82. Active Asari

  83. Cacheing

  84. None
  85. None
  86. None
  87. None
  88. None
  89. Twitter @lgleasain Github lgleasain ! www.lancegleason.com www.polyglotprogrammincinc.com lgleason@polyglotprogramminginc.com !