Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Riak on EC2

UENISHI Kota
September 28, 2013

Riak on EC2

AWSでRiakを使うときのいろいろお話しました。

UENISHI Kota

September 28, 2013
Tweet

More Decks by UENISHI Kota

Other Decks in Technology

Transcript

  1. Riak on EC2
    JAWS Festa Osaka
    ্੢߁ଠ
    2013/9/28

    View full-size slide

  2. Who are you??
    •  ্੢߁ଠ (@kuenishi)

    •  Bashoδϟύϯ

    •  ෼ࢄγεςϜྺ6೥
    •  Bashoྺ1೥ɺ΍ͬͱίʔ
    υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ

    •  ޷͖ͳ˓˓

    View full-size slide

  3. Agenda
    •  What is Riak

    •  U NO EC2?

    •  Riak + EC2 = ?

    •  ߏ੒ྫ

    •  ࣄྫʢলུ͞Ε·ͨ͠ʣ

    •  Software Design 10/18 ങͬͯͶʂ

    View full-size slide

  4. WAT IS RIAK?

    View full-size slide

  5. ཧ૝ͷσʔλϕʔε
    ʮͲΜͳނো͕ىͬͯ͜΋

    σʔλΛৗʹ੔߹ͤͭͭ͞

    ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍

    ɹσʔλϕʔεʯ


    ͸ଘࡏ͠·ͤΜ = CAPఆཧ

    View full-size slide

  6. •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS

    •  Φʔϓϯιʔε

    •  ҎԼʹॏ఺Λஔ͍͍ͯΔ

    •  Մ༻ੑʢAvailabilityʣ

    •  ଱ো֐ੑʢFault-toleranceʣ

    •  ӡ༻ͷ༰қੑʢOperational simplicityʣ

    •  εέʔϥϏϦςΟʢScalabilityʣ



    View full-size slide

  7. Dynamo ࿦จΑΓ
    •  Dynamo: Amazon’s Highly Available Key-value Store

    •  Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ

    View full-size slide

  8. Riak͸ Erlang/OTP্Ͱಈ࡞
    •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/
    OTPʹ·͔͍ͤͯΔ

    •  Erlang/OTPͱ͸ʁ

    •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ

    View full-size slide

  9. Riak͸෼ࢄKVS
    ɾRiak ͸ӬଓੑΛ࣋ͬͨ Key Value Store

    View full-size slide

  10. RiakͷσʔλͱRDBͷσʔλൺֱ
    3JBLͷσʔλߏ଄ 3%#ͷσʔλߏ଄
    PCKFDU SFDPSEʢߦʣ
    #VDLFU 5BCMF ද

    K V PK

    View full-size slide

  11. ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ

    View full-size slide

  12. ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒

    View full-size slide

  13. ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ

    View full-size slide

  14. ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ

    Riakʹ͓͚Δ
    σʔλͷࣗಈϨϓϦέʔγϣϯ
    K V K V
    OPEF OPEF OPEF OPEF

    View full-size slide

  15. ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ

    View full-size slide

  16. ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ

    Riakʹ͓͚Δ
    σʔλͷࣗಈ෮چ
    OPEF OPEF OPEF OPEF
    OPEF OPEF OPEF OPEF

    View full-size slide

  17. ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ

    Riakʹ͓͚Δ
    σʔλͷࣗಈ࠶഑ஔ
    OPEF OPEF OPEF OPEF
    OPEF OPEF OPEF OPEF
    +PJO

    View full-size slide

  18. ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ

    View full-size slide

  19. RiakͷAPI
    •  HTTP  API  or  Protocol  Buffers  
    •  put,  get,  delete,  list_keys,  counter  
    •  PBͳΒ:  http://github.com/basho/riak_pb  
    •  src/riak.proto,  src/riak_kv.proto  
    •  Ruby,  Java,  Python,  Erlang,  PHP,  …  

    View full-size slide

  20. ͭ·Γ… Riak͸
    •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS

    •  ҎԼʹॏ఺Λஔ͍͍ͯΔ

    •  Մ༻ੑʢAvailabilityʣ

    •  ଱ো֐ੑʢFault-toleranceʣ

    •  ӡ༻ͷ༰қੑʢOperational simplicityʣ

    •  εέʔϥϏϦςΟʢScalabilityʣ



    View full-size slide

  21. ͔ࣗ͠͠લͷϋʔυͩͱ…
    •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆

    •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷
    Έʹ࡞ΕΔ

    •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ
    ࠷େ஋ʹ߹Θͤͯௐୡ͠
    ͳ͍ͱ͍͚ͳ͍ͱׂߴ

    •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ
    ߏ੒͠ͳ͍ͱ͍͚ͳ͍

    View full-size slide

  22. ͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋
    •  Մ༻ੑ

    •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍

    •  ଱ো֐ੑ

    •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍

    •  ӡ༻

    •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม

    •  εέʔϥϏϦςΟ

    •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍

    View full-size slide

  23. BUT… YOU KNOW EC2

    View full-size slide

  24. AWS EC2
    •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ
    αʔϏεԽͨ͠΋ͷ

    •  ҎԼͷಛ௃

    •  Մ༻ੑʢAvailabilityʣ

    •  ଱ো֐ੑʢFault-toleranceʣ

    •  ӡ༻ͷ༰қੑʢOperational simplicityʣ

    •  εέʔϥϏϦςΟʢScalabilityʣ



    View full-size slide

  25. AWS EC2͕͋Ε͹…
    •  Մ༻ੑ

    •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ

    •  ଱ো֐ੑ

    •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ

    •  ӡ༻

    •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘

    •  εέʔϥϏϦςΟ

    •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ

    View full-size slide

  26. ͜ΕͰউͭΔ

    View full-size slide

  27. RIAK + EC2 =
    EASY OPERATION AND SCALABLE SYSTEM

    View full-size slide

  28. Riakͷߏங
    •  Bashoͷapt-lineΛ௥Ճ

    •  Πϯετʔϧ

    •  sudo  apt-­‐get  install  riak  
    •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ

    •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ

    •  ެࣜAMI΋͋ΔΑʂami-­‐7d50c77c  
    •  ˠ http://docs.basho.com/riak/latest

    View full-size slide

  29. Riakͷӡ༻ͷૢ࡞ 1/2
    •  ϊʔυΛىಈ

    •  ϊʔυΛΫϥελʹ௥Ճ

    •  riak  start  
    •  riak-­‐admin  cluster  join  \  
    [email protected]  
    •  riak-­‐admin  cluster  plan  
    •  riak-­‐admin  cluster  commit  
    •  σʔλ഑ஔΛܭࢉ

    •  ΞΫηεΛྲྀ࢝͠ΊΔ

    View full-size slide

  30. Riakͷӡ༻ͷૢ࡞ 2/2
    •  ϊʔυΛΫϥελ͔Β֎͢

    •  σʔλ഑ஔΛ࠶ܭࢉ

    •  σʔλΛ࠶഑ஔ

    •  riak-­‐admin  down  \  
    [email protected]  
    •  riak-­‐admin  cluster  plan  
    •  riak-­‐admin  cluster  commit  

    View full-size slide

  31. ͓͢͢Ίπʔϧ
    •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢

    •  ߏ੒ྫ΋͋Γ·͢

    •  https://github.com/ansible/ansible  
    •  https://github.com/ansible/ansible-­‐examples/
    tree/master/riak  

    View full-size slide

  32. ͜Μͳӡ༻͕૊Ί·͢
    ৸Δ

    ໷தʹෛՙ͕
    ্͕Δ

    APIͰϊʔυ
    ࣗಈ௥Ճ

    ໷தʹຊ౰ʹඞ
    ཁͳ਺·Ͱϊʔ
    υΛݮΒ͢

    ى͖Δ

    ʢே͕͖ͨͷͰʣ

    ࡢ໷ͷ࠷େෛՙ
    ΛΈͯɺϊʔυ୆
    ਺ͱൺֱ

    View full-size slide

  33. খ࢝͘͞ΊΔͳΒ
    ˡWebServers

    •  m1.xlarge x3

    •  Elastic Block Storage
    256GB x3, >2000PIOPS

    •  ηΩϡϦςΟάϧʔϓ

    •  TCP 4369, 6000-7999, 8087,
    8098, 8099

    View full-size slide

  34. ;ͭ͏ʹ࢝ΊΔͳΒ
    ˡWebServers

    •  m1.xlarge x5

    •  Elastic Block Storage
    256GB x5, >2000PIOPS

    •  ηΩϡϦςΟάϧʔϓ

    •  TCP 4369, 6000-7999, 8087,
    8098, 8099

    View full-size slide

  35. ˓˓͕૿͖͑ͯͨΒ
    •  ୹ظతʢ1ʙ10िؒʣରࡦ

    •  ϚγϯΛ૿΍͢

    •  ௕ظతରࡦ

    •  ϚγϯεϖοΫΛ্͛Δ

    View full-size slide

  36. ෛՙ͕૿͖͑ͯͨΒ
    ˡWebServers

    •  m3.2xlarge x5

    •  Elastic Block Storage
    512GB x5, >4000PIOPS

    •  ηΩϡϦςΟάϧʔϓ

    •  TCP 4369, 6000-7999, 8087,
    8098, 8099

    View full-size slide

  37. σʔλ͕૿͖͑ͯͨΒ
    ˡWebServers

    •  m1.xlarge x5

    •  Elastic Block Storage
    2TB x5, >2000PIOPS

    •  ηΩϡϦςΟάϧʔϓ

    •  TCP 4369, 6000-7999, 8087,
    8098, 8099

    View full-size slide

  38. ෛՙ͕૿͖͑ͯͨΒ̎
    ˡWebServers

    •  hi1.4xlarge x5

    •  SSD

    •  ηΩϡϦςΟάϧʔϓ

    •  TCP 4369, 6000-7999, 8087,
    8098, 8099

    File

    Archive

    View full-size slide

  39. όοΫΞοϓઓུ
    •  tarball ΛS3ʹ

    •  LVM Snapshot

    •  ͍Ζ͍Ζ͋Γ·͕͢…

    •  MDC Replication

    View full-size slide

  40. Availability Zone Availability Zone
    ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ

    όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ

    View full-size slide

  41. RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠

    View full-size slide

  42. ·ͱΊ
    •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍

    •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ

    •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ

    એ఻

    ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢

    ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ

    View full-size slide

  43. One more thing…
    •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ
    ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢

    •  ϙνϙν͢Δ͚ͩͰOK

    View full-size slide

  44. Go to www.engineyard.com

    View full-size slide

  45. ࣭໰ʁ
    •  Repo: https://github.com/basho/riak

    •  Twitterެࣜ: @BashoJapan or #riakjp

    •  Riak ML: [email protected]

    •  Me: [email protected]

    View full-size slide