Ruby on Rails & Vitessによる DBスケールアウト

by junsumida

Slide 1

Slide 1 text

> next Ruby on Rails & VitessʹΑΔ  DBεέʔϧΞ΢τ Jun Sumida / mixi,inc. XFLAG™ ελδΦ Meetup

Slide 2

Slide 2 text

> < next previous Jun Sumida ΤοΫεϑϥοάελδΦ ւ֎൛Ϟϯετ & ৽نλΠτϧ 2013೥৽ଔೖࣾ

Slide 3

Slide 3 text

> < next previous RailsΛ࢖ͬͨ͜ͱͷ͋Δํ?

Slide 4

Slide 4 text

> < next previous RailsͰԿ͔͠ΒͷDBͷਫฏ/ਨ௚ ෼ׂΛߦͬͨ͜ͱͷ͋Δํʁ

Slide 5

Slide 5 text

> < next previous Railsʹ͓͚ΔDB෼ׂ ‣ Octopus ‣ ਫฏਨ௚෼ׂʹରԠ ‣ SwitchPoint ‣ Cookpad͞Μ ‣ ਨ௚෼ׂʹରԠ ‣ ActiveRecord::Turntable ‣ Drecom͞Μ ‣ ਫฏਨ௚෼ׂʹରԠ ‣ ฐࣾͰ΋࢖͍ͬͯ·͢ * https://github.com/thiagopradi/octopus * http://techlife.cookpad.com/entry/2014/08/28/194147 * https://github.com/drecom/activerecord-turntable

Slide 6

Slide 6 text

> < next previous DB෼ׂ / shardingपΓͰۤ ࿑ͨ͜͠ͱ͋Γ·ͤΜ͔?

Slide 7

Slide 7 text

> < next previous MySQLͷindex΍σʔλू໿΋ࣺͯ೉͍  NoSQLͷεέʔϧੑ΋ࣺͯ೉͍

Slide 8

Slide 8 text

> < next previous ͦ͜Ͱग़ձͬͨͷ͕

Slide 9

Slide 9 text

> < next previous Vitess by youtube * http://vitess.io

Slide 10

Slide 10 text

> < next previous Vitessʹ͍ͭͯ؆୯ʹ ‣ ݩʑ͸Youtube༻ͷconnection-pooling proxy ‣ ঃʑʹ͍Ζ͍Ζͳػೳ͕௥Ճ͞Ε͍ͯͬͨ ‣ Go੡ ‣ gRPC, BSON RPC ͰΫϥΠΞϯτͱ௨৴ ‣ PHP, Go, Python, JavaͷެࣜΫϥΠΞϯτ ‣ 2011೥͔Βyoutubeͷຊ൪؀ڥͰ࢖༻ * Youtube Vitess (Cloud Native Platform) * https://github.com/youtube/vitess

Slide 11

Slide 11 text

> < next previous Կ͕Ͱ͖Δͷʁ

Slide 12

Slide 12 text

> < next previous ৭ʑͰ͖·͢

Slide 13

Slide 13 text

> < next previous Vitess͕Ͱ͖Δ͜ͱ ‣ Connection pooling ‣ QueryͷΩϟογϡ ‣ Queryղੳ, ةݥͳΫΤϦͷॻ͖׵͑ ‣ ߏ੒֬ೝ / εΩʔϚมߋ༻ͷ web console ‣ DBΠϯελϯεͷϞχλϦϯά ‣ Live Resharding, master failover, etc… * Youtube Vitess (Scaling MySQL with Vitess Kubernetes)

Slide 14

Slide 14 text

> < next previous Live Resharding * http://vitess.io/resources/openworld-2015-vitess.pdf

Slide 15

Slide 15 text

> < next previous Vitess vs MySQL ‣ ܰྔͳconnection (BSON vs MySQL) ‣ SQL Parser͕ةݥͳΫΤϦΛ๷ࢭ ‣ ਫฏɺਨ௚෼ׂΛαϙʔτ ‣ Rangeϕʔεͷsharding ‣ master failover΍data backupʹ΋ରԠ * http://vitess.io/overview/

Slide 16

Slide 16 text

> < next previous Vitess vs NoSQL ‣ ୯७ͳKVSͱ͸ҟͳΓɺwhere, join, count, group byͳ Ͳ͕Ͱ͖Δ ‣ VTGate V3Ͱ͸·ͩcount, group byͳͲͷू໿ܥ͕ಈ͔ͳ͍ ‣ Transaction͕࢖͑Δ ‣ cross shardͳTransaction͸·ͩͰ͖ͳ͍ ‣ MySQLΛ࢖͑ΔϢʔβʔͰ͋Ε͹ֶशίετ͕௿͍ ‣ MySQLͰ࢖͑Δindexػೳ͕͢΂ͯ࢖͑Δ * http://vitess.io/overview/

Slide 17

Slide 17 text

> < next previous Vitessͷߏ੒

Slide 18

Slide 18 text

> < next previous app mysql vttablet mysql vttablet mysql vttablet * Youtube Vitess (Anatomy of a Distributed Database on Kubernetes) app app vtctl schema؅ཧ / ߏ੒֬ೝ gRPC / BSON RPC gRPC / BSON RPC vtgate ܰྔproxy / routing

Slide 19

Slide 19 text

> < next previous ߏ੒ʹ͍ͭͯ΋͏ͪΐͬͱ͚ͩ ‣ MySQL 5.6 / MariaDB 10.0 or higher ʹରԠ ‣ ੲ͸ΦϦδφϧͷMySQLͩͬͨ ‣ etcd or Zookeeper Λ࢖ͬͯɺSchema΍DBߏ੒ͳͲΛ ෼ࢄͯ͠ه࿥͍ͯ͠Δ ‣ Google಺෦Ͱ͸Goolgle ChubbyΛ࢖͍ͬͯΔΒ͍͠ ‣ vttablet, vtctldʹ͸web console͕෇͍͍ͯͯschemaΛ ૢ࡞ͨ͠ΓɺmysqldͷϞχλϦϯά͕Ͱ͖Δ * http://vitess.io/overview/ * https://www.youtube.com/watch?v=midJ6b1LkA0

Slide 20

Slide 20 text

> < next previous Ͳ͏Railsͱܨ͛Δͷ͔

Slide 21

Slide 21 text

> < next previous app mysql vttablet mysql vttablet mysql vttablet * Youtube Vitess (Anatomy of a Distributed Database on Kubernetes) app app gRPC / BSON RPC vtgate ܰྔproxy / routing

Slide 22

Slide 22 text

> < next previous A high performance, open source, general RPC framework that puts mobile and HTTP/2 ﬁrst. * http://www.grpc.io

Slide 23

Slide 23 text

> < next previous gRPCͱ͸ ‣ ͍ΘΏΔRPCϑϨʔϜϫʔΫ ‣ HTTP/2 ඪ४ʹଇ͍ͬͯΔ ‣ ίΞ෦෼͸ڞ௨ɺCͰ࣮૷ ‣ 10ݴޠʹରԠ ‣ C, C++, Ruby, NodeJS, Python, PHP, Objective-C, C#, Go, Java ‣ σϑΥϧτͷσʔλߏ଄ͱͯ͠Protocol Buffers * http://www.grpc.io * https://github.com/grpc/grpc

Slide 24

Slide 24 text

> < next previous * What is gRPC? (www.grpc.io/docs/#what-is-grpc) gRPCͷྫ

Slide 25

Slide 25 text

> < next previous Protocol Buﬀers?

Slide 26

Slide 26 text

> < next previous Protocol Buﬀersͱ͸ ‣ ݴޠʹ΋ɺϓϥοτϑΥʔϜʹ΋தཱͳɺߏ଄Խ͞ ΕͨσʔλΛγϦΞϥΠζ͢ΔͨΊͷ֦ுՄೳͳػ ߏ (௚༁) ‣ ΑΓγϯϓϧͰߴ଎Ͱখ͍͞XML (௚༁) ‣ ΋ͷ͘͢͝ࡶʹݴ͏ͱɺ࠷ۙ͸΍ΓͷJSON Schema Έ͍ͨͳ΍ͭɺͨͩ͠όΠφϦ ‣ MessagePackͱൺֱ͞ΕΔ͜ͱ͕ଟ͍Ͱ͢ * https://developers.google.com/protocol-buffers/

Slide 27

Slide 27 text

> < next previous * Service Definition (www.grpc.io/docs/guides/concepts.html#service-definition) Protocol Buffersͷྫ

Slide 28

Slide 28 text

> < next previous * What is gRPC? (www.grpc.io/docs/#what-is-grpc) Protocol Buﬀersͷྫ (࣮ࡍͷVitessͷ.proto ﬁle) Rubyͩͱ͜Μͳײ͡ʹͳΓ·͢

Slide 29

Slide 29 text

> < next previous gRPC w/ Protocol Buﬀers ‣ ଞݴޠରԠͯ͠ΔͨΊɺClient΍Mock Server·ͰΛ ࣗಈੜ੒͢Δ͜ͱ͕Մೳ ‣ ௨৴ର৅ͷΞϓϦέʔγϣϯͷ۱ʑ·ͰίʔυΛಡ ·ͳͯ͘΋ClientΛ࡞Δ/͔ͭ͏͜ͱ͕Ͱ͖Δ * https://developers.google.com/protocol-buffers/

Slide 30

Slide 30 text

> < next previous gRPCͰΫϥΠΞϯτ͕ॻ͚Δ protocol buﬀerͰΫϥΠΞϯτ͸ࣗಈੜ੒Ͱ͖Δ ΫϥΠΞϯτͷwrapper͚ͩRubyͰॻ͚͹͍͍ͷͰ͸? ͋ͱ͸ActiveRecordͷAdapterͱͭͳ͛Δ͚ͩͰOKͩͶ! ͜͜Ͱࢥ͍͍ͭͯ͠·ͬͨ…

Slide 31

Slide 31 text

> < next previous ͦ͏ࢥ͍ͬͯͨ࣌ظ͕ ࢲʹ΋͋Γ·ͨ͠

Slide 32

Slide 32 text

> < next previous VitessͷRuby ClientΛ࡞Δ্ͰͰਏ͔ͬͨ͜ͱ ‣ ҰԠ·ͩgRPC͕beta൛ɺVitess Client΋৭ʑ଍Γͳ͍ ‣ Protocol Buffer͔ΒRubyͷίʔυ͕ࣗಈੜ੒Ͱ͖Δ͕ɺੜ੒͞ ΕΔίʔυͷ໊લۭ͕ؒRubyͱ૬ੑ͕ѱ͘ɺࣗಈੜ੒͞Εͨ ίʔυΛ͍͡Γճ͢Ӌ໨ʹ ‣ Mock serverͷΑ͏ͳ΋ͷ΋࡞ΕΔ͕ɺbinaryͷarrayΛ͏·͘ σ/γϦΞϥΠζͰ͖ͳ͍ͷ͔ΤϥʔΛى͜͢ ‣ ͨ·ʹMock server͕segvͰࢮ͵ ‣ ͜ͷลͷ໰୊ͷ੾Γ෼͚͕Ұ൪ਏ͔ͬͨ ‣ ͦ΋ͦ΋RubyͷgRPC gem͕versionʹΑͬͯίϯύΠϧͰ͖ͳ͔ͬͨΓ͢Δ ‣ Railsͷίʔυ(ಛʹActiveRecord)ΛԆʑͱ௥͏Ӌ໨ʹ ‣ ͓͔͛ͰActiveRecordपΓͷίʔυʹ͸ৄ͘͠ͳΕ·ͨ͠^^

Slide 33

Slide 33 text

> < next previous Rails mysql vttablet mysql vttablet mysql vttablet vtctl gRPC / BSON RPC gRPC / BSON RPC vtgate ruby  client schema༻ mysql Rails w/ Vitess

Slide 34

Slide 34 text

> < next previous ͜Ε͔Βͷ՝୊ ‣ ·ͩ৭ʑͱݕূத (2016 2݄ݱࡏ) ‣ ήʔϜͰ࢖ͬͨ৔߹ͷύϑΥʔϚϯεͳͲ ‣ εΩʔϚ؅ཧ͋ͨΓ΋ ‣ ڊେͳΞϓϦέʔγϣϯͳͷͰɺ·ͩ·ͩClientʹ଍Γͳ͍ػ ೳ͕͋Δ ‣ GoogleͷBorgʹ࠷దԽ͞Ε͍ͯΔͷͰɺAWS EC2ɺmonitoring πʔϧ, ߏ੒؅ཧπʔϧͳͲͱͲ͏ંΓ߹͍Λ͚ͭΔ͔ ‣ ӡ༻ϑϩʔ͕ͳΔ΂͘มΘΒͳ͍Α͏ʹ͍ͨ͠