Tweet
Tweet

Slide 1

Slide 1 text

Riak 2.0ʹ͓͚Δશจݕࡧ ͱ੔߹ੑͷ໰୊ WebDB Forum 2014 Basho Japan ্੢ 2014/11/19

Slide 2

Slide 2 text

ࣗݾ঺հ • ෼ࢄγεςϜྺ6೥ • Bashoδϟύϯͷํ͔Βདྷ·ͨ͠ • Riak CSͷ։ൃ • ͦͷଞ೔ຊͷ͜ͱ

Slide 3

Slide 3 text

BashoͱRiak •෼ࢄσʔλϕʔεʁ •RiakΛ஌͍ͬͯΔʁ •BashoΛ஌͍ͬͯΔʁ

Slide 4

Slide 4 text

We are hiring •࣮ੈքͷ෼ࢄγεςϜͷ ໰୊ʹڵຯ͋Δਓʂ •@BashoJapan •[email protected]

Slide 5

Slide 5 text

•APࢦ޲ͷσʔλϕʔεʢ݁Ռ੔߹ੑʣ •҆ఆੑɺ༧ଌՄೳੑ •ʮσʔλΛઈରʹͳ͘͞ͳ͍ʯ •৽͍͠ݕࡧΤϯδϯͱͯ͠SolrΛόϯυϧ

Slide 6

Slide 6 text

͜Μͳͱ͜ΖͰ ಈ͍͍ͯ·͢Riak •Rovio (Angry Birds) •Yahoo!JAPAN ͷΫϥ΢υετϨʔδ •NHS (ΠΪϦε ࠃຽอݥαʔϏε) •League of Legends (MMORPG) •ۜߦɺήʔϜɺখചɺηϯαʔɺetc…

Slide 7

Slide 7 text

CAPఆཧͱཧ૝ͷDB •ͲΜͳނোʹରͯ͠΋ (partition tolerance) •σʔλ͸ৗʹ੔߹͓ͯ͠Γ (consistency) •γεςϜ͕ࢭ·Δ͜ͱ͸ͳ͍ (availability) ͜ͷ3ͭΛಉ࣌ʹຬͨ͢γεςϜ͸ଘࡏ͠ͳ͍

Slide 8

Slide 8 text

CAP Theorem • C: ෳ਺ͷAtomic Objectʹର͢Δ࿈ ଓͨ͠ૢ࡞ (w1, w3, w4, ….) ͕શ ͯಉҰͰ͋Δ͜ͱ (linearizable) • A: Atomic Objectʹૢ࡞ w1, w2, …Λ࣮ߦͰ͖Δ͜ͱ • P: ωοτϫʔΫ͕੾ΕΔͱAtomic Object΁ͷϝοηʔδ͕౸ୡ͠ͳ͍

Slide 9

Slide 9 text

Is the network reliable?

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

How Riak Works (mostly about search)

Slide 13

Slide 13 text

run Riak with Search $ which java $ sudo apt-get install riak $ echo “search = on” >> /etc/riak/riak.conf $ riak start $ riak-admin bucket-type create tt $ riak-admin bucket-type activate tt $ curl -XPUT http://localhost:8098/search/index/t $ curl -XPUT -H ‘content-type: application/json’ \ http://localhost:8098/types/tt/props \ -d ‘{“props”:{“search_index”:”t”}}’

Slide 14

Slide 14 text

Consistent Hashing • 160-bit Ωʔۭؒ • ۭؒΛ౳෼͢Δ • ύʔςΟγϣϯ͸ϊʔ υ͕ݸผ؅ཧ • ϨϓϦΧ͸Nݸͷύʔ ςΟγϣϯʹίϐʔ͞ ΕΔ OPEF OPEF OPEF OPEF hash(“meetups/spamham”) N=3

Slide 15

Slide 15 text

σʔλϕʔεͷݕࡧ • DB૊ࠐܕ • groongaͳͲ • “LIKE” • Pros • ੔߹ੑΛҡ࣋͠΍͍͢ • Cons • ີ݁߹ʹͳΔ • ֎෦࿈ܞܕ • Solr • Elasticsearch • Pros • ૄ݁߹ʹͰ͖Δ • Cons • ੔߹ੑΛҡ࣋͠ʹ͍͘

Slide 16

Slide 16 text

DBͱΠϯσοΫεͷ੔߹ੑ • DBͷσʔλຊମͱɺΠϯσοΫεΛ౰࣌ʹ҆શʹߋ ৽͢ΔͨΊʹ͸τϥϯβΫγϣϯ͕ඞཁ • ີ݁߹ͷγεςϜͰ͋Ε͹ϩοΫͳͲΛ࢖ͬͯ੔߹ੑ Λอো͠΍͍͢

Slide 17

Slide 17 text

ૄ݁߹ʹ͢ΔϝϦοτ • ಉ͡ݕࡧΤϯδϯͰ͍ΖΜͳDBʹରԠͰ͖Δ • ӡ༻ϊ΢ϋ΢΍πʔϧɺ։ൃΛूதͰ͖Δ • όʔδϣϯ੔߹ͷ੍໿͕؇࿨͞ΕΔ • ยํ͕ނো͍ͯͯ͠΋΋͏ยํ͸ಈ࡞Ͱ͖Δ • ֦ு͠΍͘͢ͳΔ

Slide 18

Slide 18 text

྆ํͱ΋ ͍͍ͱ͜ͲΓ͍ͨ͠

Slide 19

Slide 19 text

Yokozuna

Slide 20

Slide 20 text

as Riak Search 2.0 = + σʔλͷӬଓԽ Ϋϥελͷ؅ཧ γϯϓϧͳૢ࡞ RVFSZ JOEFYJOH

Slide 21

Slide 21 text

Riak Search 2.0 • RiakʹSolrΛόϯυϧ • RiakͷKVΛߋ৽͢Δͱσʔλ͕ࣗಈతʹΠϯσο ΫεԽ͞ΕΔ • SolrͷϓϥάΠϯ΍εΩʔϚɺΫΤϦΛͦͷ··࢖ ͏͜ͱ͕Ͱ͖Δ • ৑௕Խɺϊʔυ௥Ճ࡟আ͸Riakͷ୲౰

Slide 22

Slide 22 text

ૄ݁߹ʁີ݁߹ʁ • ύοέʔδ͸ಉࠝ • ϓϩηεͱͯ͠͸ૄ݁߹ • ΠϯσοΫεͱσʔλؒ͸݁Ռ੔߹ੑΛอো • ΠϯσοΫεͷ৑௕Խ͸݁Ռ੔߹ੑΛอো

Slide 23

Slide 23 text

Solrͱ͸ૄ݁߹ • Riak ϓϩηε͕ Solr ͷϓϩηεΛ fork ͢Δ • ؅ཧɺ؂ࢹ͸શͯRiak͕ߦ͏ • ผϓϩηεɺϝϞϦۭؒ΋ผ

Slide 24

Slide 24 text

ΠϯσοΫεԽͷྲྀΕ • Riak΁ͷߋ৽ͱҰ৺ಉମʢdocument-based indexingʣ • ΠϯσοΫεԽʹࣦഊ͢ΔͱPUT͕ࣦഊ

Slide 25

Slide 25 text

ݕࡧΫΤϦͷॲཧ • Riakͷ2i΍MapReduceͱಉछͷΫΤϦ͕૸Δ • 3͓͖ͭͷ vnode ʹΞΫηε͢ΔͷͰɺॏෳ͸ͳ͍ ू໿

Slide 26

Slide 26 text

੔߹ੑͷ໰୊ ͱҟৗܥ ੔߹ੑҡ࣋ͷͨΊʹߋ৽ϩάΛอ࣋ͯ͠ ϦΧόϦ͢Δͱ͍͏ํ๏Ͱ͸ɺՄ༻ੑͱ ෼அ଱ੑΛ୲อͰ͖ͳ͍ɻRiakͰ͸ଞͷ ํ๏Ͱσʔλͷ੔߹ੑΛҡ࣋͢Δɻ

Slide 27

Slide 27 text

Α͋͘Δҟৗܥ #1 •ϊʔυނোͰෳ੡Λ3ͭ࡞Εͳ͍ •෮چͯ͠΋ɺ2ͭͷ··ʹͳͬͯ͠·͏

Slide 28

Slide 28 text

Α͋͘Δҟৗܥ #1 • ෮چͨ͋͠ͱʹRead Repair͢Δ • GETͨ͠ͱ͖ʹɺσʔλʹ͕ܽؕ͋ͬͨΒ RiakଆͰPUT͠௚͢ v2 v2 not found v2

Slide 29

Slide 29 text

Read Repair͕ ݺ͹Εͳ͍ͱ σʔλ͕௚Βͳ͍ʁ YES

Slide 30

Slide 30 text

ΫϥΠΞϯτ͔Β GET͕ݺ͹Εͳ͍ͱ σʔλ͕௚Βͳ͍ʁ NO

Slide 31

Slide 31 text

AAE (Active Anti Entropy) https://www.flickr.com/photos/51pct/7507525118/

Slide 32

Slide 32 text

Active Anti Entropy • APࢦ޲ͷDBͷσʔλྼԽΛ๷͙ ͨΊͷόοΫάϥ΢ϯυॲཧ • Merkle-TreeΛ࢖ͬͯύʔςΟγϣ ϯຖͷʮνΣοΫαϜʯΛܭࢉ • ࠩ෼Λݟ͚ͭͨΒͦ͜ΛRead Repair͢Δ hash(vnode=0, pid=0) hash(vnode=1, pid=0) hash(vnode=2, pid=0)

Slide 33

Slide 33 text

Α͋͘Δҟৗܥ #2 • Solr͕ΠϯσοΫεԽ͢Δલʹམͪͨ • Key, Value͸อଘ͞Ε͕ͨΠϯσοΫε͸ͳ͍

Slide 34

Slide 34 text

Read Repair͕ ݺ͹ΕΕ͹Πϯσο Ϋε͕म෮͞ΕΔʁ NO

Slide 35

Slide 35 text

AAE (Active Anti Entropy) https://www.flickr.com/photos/51pct/7507525118/

Slide 36

Slide 36 text

YZ Active Anti Entropy • ݕࡧΠϯσοΫεʹ͍ͭͯ΋ Merkle TreeΛ࡞੒ • ύʔςΟγϣϯຖʹKey-Valueͷ Treeͱൺֱ͠ɺ͕ࠩ͋Ε͹Πϯ σοΫεΛमਖ਼ hash(vnode=0, pid=0, kv) hash(vnode=0, pid=0, yz)

Slide 37

Slide 37 text

Α͋͘Δҟৗܥ #2 •Searchઐ༻AAE͕ಈ࡞͍ͯ͠Δ •ΠϯσοΫεͱσʔλͷෆ੔߹Λݟ͚ͭͯ म෮͢Δ AAE

Slide 38

Slide 38 text

Α͋͘Δҟৗܥ #3 • ωοτϫʔΫ͕੾Ε͍ͯͨͱ͖ʹ྆ଆͰߋ৽͕ى ͖ͨʢSplit Brainʣ • Hinted HandoffʹΑΓͲͪΒ΋ॻ͖ࠐΈ੒ޭ

Slide 39

Slide 39 text

Α͋͘Δҟৗܥ #3 • ෮چ࣌ʹHandoffͰฦ͞Εɺ྆ํͷ஋Λอ࣋ • GET·ͨ͸AAEʹΑͬͯRead Repair͞ΕΔ

Slide 40

Slide 40 text

ҟৗܥ #1, #2, #3 •Ͳͷҟৗ΋ɺಛʹۓٸͷରॲ͸ඞཁͳ͍ •੔߹ੑݕࠪͱम෮͕ࣗಈͰߦΘΕΔ •ϧʔνϯϫʔΫͷӡ༻࡞ۀ͕΄ͱΜͲͳ͍

Slide 41

Slide 41 text

·ͱΊ • σʔλϕʔεͱશจݕࡧͷ໰୊ • ੔߹ੑΛͱΔ͔ɺૄ݁߹ΛͱΔ͔ͷ໰୊ • ৽͍͠Riak Search 2.0 • ਫฏ෼ࢄͯ͠εέʔϧΞ΢τͰ͖Δݕࡧ • ݁Ռ੔߹ੑΛબͿ͜ͱʹΑΓૄ݁߹Խͱ෼ࢄԽ ʹ੒ޭʢ͔͠΋APܕʹʂʣ • ݁Ռ੔߹ੑΛબ୒ͨ͠ઃܭʹΑΓӡ༻͕؆୯ʹ

Slide 42

Slide 42 text

Questions?

Slide 43

Slide 43 text

FAQ •ΦϯϥΠϯͷεΩʔϚมߋʁ •μΠφϛοΫεΩʔϚʁ

Slide 44

Slide 44 text

outline • σʔλϕʔεʹೖ͍ͬͯΔσʔλΛશจݕࡧ͍ͨ͠→୔ࢁͷΠϯσοΫε͕Ͱ͖ΔɺFKeyͰ ΠϯσοΫε…ͱ͍͏͚ͩͰ͸ • શจݕࡧΤϯδϯͷछྨ • ֎෦ܕʢ੔߹ੑΛҡ࣋͠ʹ͍͕͘Ԡ༻͕ޮ͘ɺރΕͨιϑτ΢ΣΞ΍ϓϥάΠϯ͕ଟ͍ʣ • ૊ΈࠐΈܕʢ੔߹ੑΛҡ࣋͠΍͍͕͢ɺϝϯςφϯε͕೉͍͠ʣ • Riak͸ͦͷதؒɹόϯυϧͯ͠ΔͷͰָɺ͚ͩͲ࣮ମ͸Solrɺ੔߹ੑΛࣗಈతʹҡ࣋͢Δ • όϯυϧͷํ๏ɺ੔߹ੑɺ෼ࢄͷํ๏ • ݕࡧΠϯσοΫε͚ͩΛ෼ࢄͤ͞Δͷ͸೉͍͠ • DBͱΠϯσοΫεͷ੔߹ɺϨϓϦΧͱϨϓϦΧͷ੔߹ɺJepsenͷ͸ͳ͠