Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Riak on EC2

UENISHI Kota
September 28, 2013

Riak on EC2

AWSでRiakを使うときのいろいろお話しました。

UENISHI Kota

September 28, 2013
Tweet

More Decks by UENISHI Kota

Other Decks in Technology

Transcript

  1. Riak on EC2 JAWS Festa Osaka ্੢߁ଠ 2013/9/28

  2. Who are you?? •  ্੢߁ଠ (@kuenishi) •  Bashoδϟύϯ •  ෼ࢄγεςϜྺ6೥

    •  Bashoྺ1೥ɺ΍ͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ •  ޷͖ͳ˓˓
  3. Agenda •  What is Riak •  U NO EC2? • 

    Riak + EC2 = ? •  ߏ੒ྫ •  ࣄྫʢলུ͞Ε·ͨ͠ʣ •  Software Design 10/18 ങͬͯͶʂ
  4. WAT IS RIAK?

  5. ཧ૝ͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜΋ σʔλΛৗʹ੔߹ͤͭͭ͞ ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍ ɹσʔλϕʔεʯ ͸ଘࡏ͠·ͤΜ = CAPఆཧ

  6. •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  Φʔϓϯιʔε •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ • 

    ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
  7. Dynamo ࿦จΑΓ •  Dynamo: Amazon’s Highly Available Key-value Store • 

    Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
  8. Riak͸ Erlang/OTP্Ͱಈ࡞ •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/ OTPʹ·͔͍ͤͯΔ •  Erlang/OTPͱ͸ʁ •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ

  9. Riak͸෼ࢄKVS ɾRiak ͸ӬଓੑΛ࣋ͬͨ Key Value Store

  10. RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ଄ 3%#ͷσʔλߏ଄ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද  K

    V PK
  11. ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ

  12. ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒

  13. ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ

  14. ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF

    OPEF
  15. ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ

  16. ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF

    OPEF
  17. ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶഑ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF

    OPEF +PJO
  18. ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ

  19. RiakͷAPI •  HTTP  API  or  Protocol  Buffers   •  put,

     get,  delete,  list_keys,  counter   •  PBͳΒ:  http://github.com/basho/riak_pb   •  src/riak.proto,  src/riak_kv.proto   •  Ruby,  Java,  Python,  Erlang,  PHP,  …  
  20. ͭ·Γ… Riak͸ •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ • 

    ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
  21. ͔ࣗ͠͠લͷϋʔυͩͱ… •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷ Έʹ࡞ΕΔ •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ ࠷େ஋ʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ

    •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ੒͠ͳ͍ͱ͍͚ͳ͍
  22. ͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋ •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ • 

    ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม •  εέʔϥϏϦςΟ •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍
  23. OMG

  24. BUT… YOU KNOW EC2

  25. AWS EC2 •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ αʔϏεԽͨ͠΋ͷ •  ҎԼͷಛ௃ •  Մ༻ੑʢAvailabilityʣ

    •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
  26. AWS EC2͕͋Ε͹… •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ

    •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘ •  εέʔϥϏϦςΟ •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ
  27. ͜ΕͰউͭΔ

  28. RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM

  29. Riakͷߏங •  Bashoͷapt-lineΛ௥Ճ •  Πϯετʔϧ •  sudo  apt-­‐get  install  riak

      •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ •  ެࣜAMI΋͋ΔΑʂami-­‐7d50c77c   •  ˠ http://docs.basho.com/riak/latest
  30. Riakͷӡ༻ͷૢ࡞ 1/2 •  ϊʔυΛىಈ •  ϊʔυΛΫϥελʹ௥Ճ •  riak  start  

    •  riak-­‐admin  cluster  join  \   riak@10.0.0.1   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit   •  σʔλ഑ஔΛܭࢉ •  ΞΫηεΛྲྀ࢝͠ΊΔ
  31. Riakͷӡ༻ͷૢ࡞ 2/2 •  ϊʔυΛΫϥελ͔Β֎͢ •  σʔλ഑ஔΛ࠶ܭࢉ •  σʔλΛ࠶഑ஔ •  riak-­‐admin

     down  \   riak@10.1.2.3   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit  
  32. ͓͢͢Ίπʔϧ •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ •  ߏ੒ྫ΋͋Γ·͢ •  https://github.com/ansible/ansible   •  https://github.com/ansible/ansible-­‐examples/

    tree/master/riak  
  33. ͜Μͳӡ༻͕૊Ί·͢ ৸Δ ໷தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈ௥Ճ ໷தʹຊ౰ʹඞ ཁͳ਺·Ͱϊʔ υΛݮΒ͢ ى͖Δ

    ʢே͕͖ͨͷͰʣ ࡢ໷ͷ࠷େෛՙ ΛΈͯɺϊʔυ୆ ਺ͱൺֱ
  34. খ࢝͘͞ΊΔͳΒ ˡWebServers •  m1.xlarge x3 •  Elastic Block Storage 256GB

    x3, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
  35. ;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 256GB

    x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
  36. ˓˓͕૿͖͑ͯͨΒ •  ୹ظతʢ1ʙ10िؒʣରࡦ •  ϚγϯΛ૿΍͢ •  ௕ظతରࡦ •  ϚγϯεϖοΫΛ্͛Δ

  37. ෛՙ͕૿͖͑ͯͨΒ ˡWebServers •  m3.2xlarge x5 •  Elastic Block Storage 512GB

    x5, >4000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
  38. σʔλ͕૿͖͑ͯͨΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 2TB

    x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
  39. ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers •  hi1.4xlarge x5 •  SSD •  ηΩϡϦςΟάϧʔϓ • 

    TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
  40. όοΫΞοϓઓུ •  tarball ΛS3ʹ •  LVM Snapshot •  ͍Ζ͍Ζ͋Γ·͕͢… • 

    MDC Replication
  41. Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ

  42. RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠

  43. ·ͱΊ •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍ •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ

    એ఻ ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ
  44. One more thing… •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ •  ϙνϙν͢Δ͚ͩͰOK

  45. Go to www.engineyard.com

  46. ࣭໰ʁ •  Repo: https://github.com/basho/riak •  Twitterެࣜ: @BashoJapan or #riakjp • 

    Riak ML: riak-users-jp@lists.basho.com •  Me: kota@basho.com