Riak on EC2

AWSでRiakを使うときのいろいろお話しました。

Transcript

1. Riak on EC2 JAWS Festa Osaka ্੢߁ଠ 2013/9/28
2. Who are you?? •  ্੢߁ଠ (@kuenishi) •  Bashoδϟύϯ •  ෼ࢄγεςϜྺ6೥
   •  Bashoྺ1೥ɺ΍ͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ •  ޷͖ͳ˓˓
3. Agenda •  What is Riak •  U NO EC2? •  Riak + EC2 = ? •  ߏ੒ྫ •  ࣄྫʢলུ͞Ε·ͨ͠ʣ •  Software Design 10/18 ങͬͯͶʂ
   
4. WAT IS RIAK?
5. ཧ૝ͷσʔλϕʔε
   ʮͲΜͳނো͕ىͬͯ͜΋ σʔλΛৗʹ੔߹ͤͭͭ͞ ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍ ɹσʔλϕʔεʯ ͸ଘࡏ͠·ͤΜ = CAPఆཧ
6. •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  Φʔϓϯιʔε •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
7. Dynamo ࿦จΑΓ
   •  Dynamo: Amazon’s Highly Available Key-value Store •  Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
8. Riak͸ Erlang/OTP্Ͱಈ࡞
   •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/ OTPʹ·͔͍ͤͯΔ •  Erlang/OTPͱ͸ʁ •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ
9. Riak͸෼ࢄKVS
   ɾRiak ͸
   ӬଓੑΛ࣋ͬͨ Key Value Store
10. RiakͷσʔλͱRDBͷσʔλൺֱ
    3JBLͷσʔλߏ଄
    3%#ͷσʔλߏ଄
    PCKFDU
    SFDPSE
    ʢߦʣ
    #VDLFU
    5BCMF
    ද
    K V PK
11. ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ
12. ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒
13. ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ
14. ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ
    K V K V OPEF
    OPEF
    OPEF
    OPEF
    
15. ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ
16. ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    
17. ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶഑ஔ
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    +PJO
    
18. ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ
19. RiakͷAPI •  HTTP  API  or  Protocol  Buffers   •  put,  get,  delete,  list_keys,  counter   •  PBͳΒ:  http://github.com/basho/riak_pb   •  src/riak.proto,  src/riak_kv.proto   •  Ruby,  Java,  Python,  Erlang,  PHP,  …  
20. ͭ·Γ… Riak͸
    •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
21. ͔ࣗ͠͠લͷϋʔυͩͱ… •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷ Έʹ࡞ΕΔ •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ ࠷େ஋ʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ੒͠ͳ͍ͱ͍͚ͳ͍
22. ͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋ •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม •  εέʔϥϏϦςΟ •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍
23. OMG
24. BUT… YOU KNOW EC2
25. AWS EC2
    •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ αʔϏεԽͨ͠΋ͷ •  ҎԼͷಛ௃ •  Մ༻ੑʢAvailabilityʣ •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
26. AWS EC2͕͋Ε͹… •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘ •  εέʔϥϏϦςΟ •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ
27. ͜ΕͰউͭΔ
28. RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM
29. Riakͷߏங •  Bashoͷapt-lineΛ௥Ճ •  Πϯετʔϧ •  sudo  apt-­‐get  install  riak   •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ •  ެࣜAMI΋͋ΔΑʂ
    ami-­‐7d50c77c   •  ˠ http://docs.basho.com/riak/latest
30. Riakͷӡ༻ͷૢ࡞ 1/2 •  ϊʔυΛىಈ •  ϊʔυΛΫϥελʹ௥Ճ •  riak  start   •  riak-­‐admin  cluster  join  \   riak@10.0.0.1   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit   •  σʔλ഑ஔΛܭࢉ •  ΞΫηεΛྲྀ࢝͠ΊΔ
31. Riakͷӡ༻ͷૢ࡞ 2/2 •  ϊʔυΛΫϥελ͔Β֎͢ •  σʔλ഑ஔΛ࠶ܭࢉ •  σʔλΛ࠶഑ஔ •  riak-­‐admin  down  \   riak@10.1.2.3   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit  
32. ͓͢͢Ίπʔϧ •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ •  ߏ੒ྫ΋͋Γ·͢ •  https://github.com/ansible/ansible   •  https://github.com/ansible/ansible-­‐examples/ tree/master/riak  
33. ͜Μͳӡ༻͕૊Ί·͢ ৸Δ ໷தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈ௥Ճ ໷தʹຊ౰ʹඞ ཁͳ਺·Ͱϊʔ υΛݮΒ͢ ى͖Δ ʢே͕͖ͨͷͰʣ ࡢ໷ͷ࠷େෛՙ ΛΈͯɺϊʔυ୆ ਺ͱൺֱ
34. খ࢝͘͞ΊΔͳΒ ˡWebServers •  m1.xlarge x3 •  Elastic Block Storage 256GB x3, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
35. ;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 256GB x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
36. ˓˓͕૿͖͑ͯͨΒ •  ୹ظతʢ1ʙ10िؒʣରࡦ •  ϚγϯΛ૿΍͢ •  ௕ظతରࡦ •  ϚγϯεϖοΫΛ্͛Δ
37. ෛՙ͕૿͖͑ͯͨΒ ˡWebServers •  m3.2xlarge x5 •  Elastic Block Storage 512GB x5, >4000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
38. σʔλ͕૿͖͑ͯͨΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 2TB x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
39. ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers •  hi1.4xlarge x5 •  SSD •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
40. όοΫΞοϓઓུ •  tarball ΛS3ʹ •  LVM Snapshot •  ͍Ζ͍Ζ͋Γ·͕͢… •  MDC Replication
41. Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ
42. RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
43. ·ͱΊ •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍ •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ એ఻ ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ
44. One more thing… •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ •  ϙνϙν͢Δ͚ͩͰOK
45. Go to www.engineyard.com
46. ࣭໰ʁ •  Repo: https://github.com/basho/riak •  Twitterެࣜ: @BashoJapan or #riakjp •  Riak ML: riak-users-jp@lists.basho.com •  Me: kota@basho.com