Riak on EC2 - Speaker Deck

Slide 1

Slide 1 text

Riak on EC2 JAWS Festa Osaka ্੢߁ଠ 2013/9/28

Slide 2

Slide 2 text

Who are you?? •  ্੢߁ଠ (@kuenishi) •  Bashoδϟύϯ •  ෼ࢄγεςϜྺ6೥ •  Bashoྺ1೥ɺ΍ͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ •  ޷͖ͳ˓˓

Slide 3

Slide 3 text

Agenda •  What is Riak •  U NO EC2? •  Riak + EC2 = ? •  ߏ੒ྫ •  ࣄྫʢলུ͞Ε·ͨ͠ʣ •  Software Design 10/18 ങͬͯͶʂ

Slide 4

Slide 4 text

WAT IS RIAK?

Slide 5

Slide 5 text

ཧ૝ͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜΋ σʔλΛৗʹ੔߹ͤͭͭ͞ ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍ ɹσʔλϕʔεʯ ͸ଘࡏ͠·ͤΜ = CAPఆཧ

Slide 6

Slide 6 text

•  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  Φʔϓϯιʔε •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ

Slide 7

Slide 7 text

Dynamo ࿦จΑΓ •  Dynamo: Amazon’s Highly Available Key-value Store •  Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ

Slide 8

Slide 8 text

Riak͸ Erlang/OTP্Ͱಈ࡞ •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/ OTPʹ·͔͍ͤͯΔ •  Erlang/OTPͱ͸ʁ •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ

Slide 9

Slide 9 text

Riak͸෼ࢄKVS ɾRiak ͸ӬଓੑΛ࣋ͬͨ Key Value Store

Slide 10

Slide 10 text

RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ଄ 3%#ͷσʔλߏ଄ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද K V PK

Slide 11

Slide 11 text

ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ

Slide 12

Slide 12 text

ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒

Slide 13

Slide 13 text

ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ

Slide 14

Slide 14 text

ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF OPEF

Slide 15

Slide 15 text

ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ

Slide 16

Slide 16 text

ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF OPEF

Slide 17

Slide 17 text

ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶഑ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF OPEF +PJO

Slide 18

Slide 18 text

ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ

Slide 19

Slide 19 text

RiakͷAPI •  HTTP API or Protocol Buffers •  put, get, delete, list_keys, counter •  PBͳΒ: http://github.com/basho/riak_pb •  src/riak.proto, src/riak_kv.proto •  Ruby, Java, Python, Erlang, PHP, …

Slide 20

Slide 20 text

ͭ·Γ… Riak͸ •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ

Slide 21

Slide 21 text

͔ࣗ͠͠લͷϋʔυͩͱ… •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷ Έʹ࡞ΕΔ •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ ࠷େ஋ʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ੒͠ͳ͍ͱ͍͚ͳ͍

Slide 22

Slide 22 text

͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋ •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม •  εέʔϥϏϦςΟ •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍

Slide 23

Slide 23 text

OMG

Slide 24

Slide 24 text

BUT… YOU KNOW EC2

Slide 25

Slide 25 text

AWS EC2 •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ αʔϏεԽͨ͠΋ͷ •  ҎԼͷಛ௃ •  Մ༻ੑʢAvailabilityʣ •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ

Slide 26

Slide 26 text

AWS EC2͕͋Ε͹… •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘ •  εέʔϥϏϦςΟ •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ

Slide 27

Slide 27 text

͜ΕͰউͭΔ

Slide 28

Slide 28 text

RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM

Slide 29

Slide 29 text

Riakͷߏங •  Bashoͷapt-lineΛ௥Ճ •  Πϯετʔϧ •  sudo apt-‐get install riak •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ •  ެࣜAMI΋͋ΔΑʂami-‐7d50c77c •  ˠ http://docs.basho.com/riak/latest

Slide 30

Slide 30 text

Riakͷӡ༻ͷૢ࡞ 1/2 •  ϊʔυΛىಈ •  ϊʔυΛΫϥελʹ௥Ճ •  riak start •  riak-‐admin cluster join \ [email protected] •  riak-‐admin cluster plan •  riak-‐admin cluster commit •  σʔλ഑ஔΛܭࢉ •  ΞΫηεΛྲྀ࢝͠ΊΔ

Slide 31

Slide 31 text

Riakͷӡ༻ͷૢ࡞ 2/2 •  ϊʔυΛΫϥελ͔Β֎͢ •  σʔλ഑ஔΛ࠶ܭࢉ •  σʔλΛ࠶഑ஔ •  riak-‐admin down \ [email protected] •  riak-‐admin cluster plan •  riak-‐admin cluster commit

Slide 32

Slide 32 text

͓͢͢Ίπʔϧ •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ •  ߏ੒ྫ΋͋Γ·͢ •  https://github.com/ansible/ansible •  https://github.com/ansible/ansible-‐examples/ tree/master/riak

Slide 33

Slide 33 text

͜Μͳӡ༻͕૊Ί·͢ ৸Δ ໷தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈ௥Ճ ໷தʹຊ౰ʹඞ ཁͳ਺·Ͱϊʔ υΛݮΒ͢ ى͖Δ ʢே͕͖ͨͷͰʣ ࡢ໷ͷ࠷େෛՙ ΛΈͯɺϊʔυ୆ ਺ͱൺֱ

Slide 34

Slide 34 text

খ࢝͘͞ΊΔͳΒ ˡWebServers •  m1.xlarge x3 •  Elastic Block Storage 256GB x3, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

Slide 35

Slide 35 text

;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 256GB x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

Slide 36

Slide 36 text

˓˓͕૿͖͑ͯͨΒ •  ୹ظతʢ1ʙ10िؒʣରࡦ •  ϚγϯΛ૿΍͢ •  ௕ظతରࡦ •  ϚγϯεϖοΫΛ্͛Δ

Slide 37

Slide 37 text

ෛՙ͕૿͖͑ͯͨΒ ˡWebServers •  m3.2xlarge x5 •  Elastic Block Storage 512GB x5, >4000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

Slide 38

Slide 38 text

σʔλ͕૿͖͑ͯͨΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 2TB x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

Slide 39

Slide 39 text

ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers •  hi1.4xlarge x5 •  SSD •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive

Slide 40

Slide 40 text

όοΫΞοϓઓུ •  tarball ΛS3ʹ •  LVM Snapshot •  ͍Ζ͍Ζ͋Γ·͕͢… •  MDC Replication

Slide 41

Slide 41 text

Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ

Slide 42

Slide 42 text

RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠

Slide 43

Slide 43 text

·ͱΊ •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍ •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ એ఻ ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ

Slide 44

Slide 44 text

One more thing… •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ •  ϙνϙν͢Δ͚ͩͰOK

Slide 45

Slide 45 text

Go to www.engineyard.com

Slide 46

Slide 46 text

࣭໰ʁ •  Repo: https://github.com/basho/riak •  Twitterެࣜ: @BashoJapan or #riakjp •  Riak ML: [email protected] •  Me: [email protected]