Riak on EC2

Transcript

1. 1.

   Riak on EC2 JAWS Festa Osaka ্੢߁ଠ 2013/9/28

2. 2.

   Who are you?? •  ্੢߁ଠ (@kuenishi) •  Bashoδϟύϯ •  ෼ࢄγεςϜྺ6೥
   

   •  Bashoྺ1೥ɺ΍ͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ •  ޷͖ͳ˓˓
3. 3.

   Agenda •  What is Riak •  U NO EC2? • 

   Riak + EC2 = ? •  ߏ੒ྫ •  ࣄྫʢলུ͞Ε·ͨ͠ʣ •  Software Design 10/18 ങͬͯͶʂ
   
4. 4.

   WAT IS RIAK?

5. 5.

   ཧ૝ͷσʔλϕʔε
   ʮͲΜͳނো͕ىͬͯ͜΋ σʔλΛৗʹ੔߹ͤͭͭ͞ ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍ ɹσʔλϕʔεʯ ͸ଘࡏ͠·ͤΜ = CAPఆཧ

6. 6.

   •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  Φʔϓϯιʔε •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ • 

   ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
7. 7.

   Dynamo ࿦จΑΓ
   •  Dynamo: Amazon’s Highly Available Key-value Store • 

   Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
8. 8.

   Riak͸ Erlang/OTP্Ͱಈ࡞
   •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/ OTPʹ·͔͍ͤͯΔ •  Erlang/OTPͱ͸ʁ •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ

9. 9.

   Riak͸෼ࢄKVS
   ɾRiak ͸
   ӬଓੑΛ࣋ͬͨ Key Value Store

10. 10.

    RiakͷσʔλͱRDBͷσʔλൺֱ
    3JBLͷσʔλߏ଄
    3%#ͷσʔλߏ଄
    PCKFDU
    SFDPSE
    ʢߦʣ
    #VDLFU
    5BCMF
    ද
    K

    V PK
11. 11.

    ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ

12. 12.

    ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒

13. 13.

    ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ

14. 14.

    ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ
    K V K V OPEF
    OPEF
    OPEF
    

    OPEF
    
15. 15.

    ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ

16. 16.

    ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    

    OPEF
    
17. 17.

    ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶഑ஔ
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    

    OPEF
    +PJO
    
18. 18.

    ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ

19. 19.

    RiakͷAPI •  HTTP  API  or  Protocol  Buffers   •  put,

     get,  delete,  list_keys,  counter   •  PBͳΒ:  http://github.com/basho/riak_pb   •  src/riak.proto,  src/riak_kv.proto   •  Ruby,  Java,  Python,  Erlang,  PHP,  …  
20. 20.

    ͭ·Γ… Riak͸
    •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ • 

    ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
21. 21.

    ͔ࣗ͠͠લͷϋʔυͩͱ… •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷ Έʹ࡞ΕΔ •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ ࠷େ஋ʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ

    •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ੒͠ͳ͍ͱ͍͚ͳ͍
22. 22.

    ͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋ •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ • 

    ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม •  εέʔϥϏϦςΟ •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍
23. 23.

    OMG

24. 24.

    BUT… YOU KNOW EC2

25. 25.

    AWS EC2
    •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ αʔϏεԽͨ͠΋ͷ •  ҎԼͷಛ௃ •  Մ༻ੑʢAvailabilityʣ

    •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ
26. 26.

    AWS EC2͕͋Ε͹… •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ

    •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘ •  εέʔϥϏϦςΟ •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ
27. 27.

    ͜ΕͰউͭΔ

28. 28.

    RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM

29. 29.

    Riakͷߏங •  Bashoͷapt-lineΛ௥Ճ •  Πϯετʔϧ •  sudo  apt-­‐get  install  riak

      •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ •  ެࣜAMI΋͋ΔΑʂ
    ami-­‐7d50c77c   •  ˠ http://docs.basho.com/riak/latest
30. 30.

    Riakͷӡ༻ͷૢ࡞ 1/2 •  ϊʔυΛىಈ •  ϊʔυΛΫϥελʹ௥Ճ •  riak  start  

    •  riak-­‐admin  cluster  join  \   riak@10.0.0.1   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit   •  σʔλ഑ஔΛܭࢉ •  ΞΫηεΛྲྀ࢝͠ΊΔ
31. 31.

    Riakͷӡ༻ͷૢ࡞ 2/2 •  ϊʔυΛΫϥελ͔Β֎͢ •  σʔλ഑ஔΛ࠶ܭࢉ •  σʔλΛ࠶഑ஔ •  riak-­‐admin

     down  \   riak@10.1.2.3   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit  
32. 32.

    ͓͢͢Ίπʔϧ •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ •  ߏ੒ྫ΋͋Γ·͢ •  https://github.com/ansible/ansible   •  https://github.com/ansible/ansible-­‐examples/

    tree/master/riak  
33. 33.

    ͜Μͳӡ༻͕૊Ί·͢ ৸Δ ໷தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈ௥Ճ ໷தʹຊ౰ʹඞ ཁͳ਺·Ͱϊʔ υΛݮΒ͢ ى͖Δ

    ʢே͕͖ͨͷͰʣ ࡢ໷ͷ࠷େෛՙ ΛΈͯɺϊʔυ୆ ਺ͱൺֱ
34. 34.

    খ࢝͘͞ΊΔͳΒ ˡWebServers •  m1.xlarge x3 •  Elastic Block Storage 256GB

    x3, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
35. 35.

    ;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 256GB

    x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
36. 36.

    ˓˓͕૿͖͑ͯͨΒ •  ୹ظతʢ1ʙ10िؒʣରࡦ •  ϚγϯΛ૿΍͢ •  ௕ظతରࡦ •  ϚγϯεϖοΫΛ্͛Δ

37. 37.

    ෛՙ͕૿͖͑ͯͨΒ ˡWebServers •  m3.2xlarge x5 •  Elastic Block Storage 512GB

    x5, >4000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
38. 38.

    σʔλ͕૿͖͑ͯͨΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 2TB

    x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099
39. 39.

    ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers •  hi1.4xlarge x5 •  SSD •  ηΩϡϦςΟάϧʔϓ • 

    TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
40. 40.

    όοΫΞοϓઓུ •  tarball ΛS3ʹ •  LVM Snapshot •  ͍Ζ͍Ζ͋Γ·͕͢… • 

    MDC Replication
41. 41.

    Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ

42. 42.

    RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠

43. 43.

    ·ͱΊ •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍ •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ

    એ఻ ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ
44. 44.

    One more thing… •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ •  ϙνϙν͢Δ͚ͩͰOK

45. 45.

    Go to www.engineyard.com

46. 46.

    ࣭໰ʁ •  Repo: https://github.com/basho/riak •  Twitterެࣜ: @BashoJapan or #riakjp • 

    Riak ML: riak-users-jp@lists.basho.com •  Me: kota@basho.com