Riak on EC2

Transcript

1. Riak on EC2 JAWS Festa Osaka ্੢߁ଠ 2013/9/28

2. Who are you?? •  ্੢߁ଠ (@kuenishi) •  Bashoδϟύϯ •  ෼ࢄγεςϜྺ6೥
   

   •  Bashoྺ1೥ɺ΍ͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ •  ޷͖ͳ˓˓

3. Agenda •  What is Riak •  U NO EC2? • 

   Riak + EC2 = ? •  ߏ੒ྫ •  ࣄྫʢলུ͞Ε·ͨ͠ʣ •  Software Design 10/18 ങͬͯͶʂ
   

4. WAT IS RIAK?

5. ཧ૝ͷσʔλϕʔε
   ʮͲΜͳނো͕ىͬͯ͜΋ σʔλΛৗʹ੔߹ͤͭͭ͞ ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍ ɹσʔλϕʔεʯ ͸ଘࡏ͠·ͤΜ = CAPఆཧ

6. •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  Φʔϓϯιʔε •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ • 

   ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ

7. Dynamo ࿦จΑΓ
   •  Dynamo: Amazon’s Highly Available Key-value Store • 

   Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ

8. Riak͸ Erlang/OTP্Ͱಈ࡞
   •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/ OTPʹ·͔͍ͤͯΔ •  Erlang/OTPͱ͸ʁ •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ

9. Riak͸෼ࢄKVS
   ɾRiak ͸
   ӬଓੑΛ࣋ͬͨ Key Value Store

10. RiakͷσʔλͱRDBͷσʔλൺֱ
    3JBLͷσʔλߏ଄
    3%#ͷσʔλߏ଄
    PCKFDU
    SFDPSE
    ʢߦʣ
    #VDLFU
    5BCMF
    ද
    K

    V PK

11. ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ

12. ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒

13. ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ

14. ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ
    K V K V OPEF
    OPEF
    OPEF
    

    OPEF
    

15. ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ

16. ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    

    OPEF
    

17. ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶഑ஔ
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    OPEF
    

    OPEF
    +PJO
    

18. ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ

19. RiakͷAPI •  HTTP  API  or  Protocol  Buffers   •  put,

     get,  delete,  list_keys,  counter   •  PBͳΒ:  http://github.com/basho/riak_pb   •  src/riak.proto,  src/riak_kv.proto   •  Ruby,  Java,  Python,  Erlang,  PHP,  …  

20. ͭ·Γ… Riak͸
    •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS •  ҎԼʹॏ఺Λஔ͍͍ͯΔ •  Մ༻ੑʢAvailabilityʣ • 

    ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ

21. ͔ࣗ͠͠લͷϋʔυͩͱ… •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷ Έʹ࡞ΕΔ •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ ࠷େ஋ʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ

    •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ੒͠ͳ͍ͱ͍͚ͳ͍

22. ͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋ •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ • 

    ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม •  εέʔϥϏϦςΟ •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍

23. OMG

24. BUT… YOU KNOW EC2

25. AWS EC2
    •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ αʔϏεԽͨ͠΋ͷ •  ҎԼͷಛ௃ •  Մ༻ੑʢAvailabilityʣ

    •  ଱ো֐ੑʢFault-toleranceʣ •  ӡ༻ͷ༰қੑʢOperational simplicityʣ •  εέʔϥϏϦςΟʢScalabilityʣ

26. AWS EC2͕͋Ε͹… •  Մ༻ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ •  ଱ো֐ੑ •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ

    •  ӡ༻ •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘ •  εέʔϥϏϦςΟ •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ

27. ͜ΕͰউͭΔ

28. RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM

29. Riakͷߏங •  Bashoͷapt-lineΛ௥Ճ •  Πϯετʔϧ •  sudo  apt-­‐get  install  riak

      •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ •  ެࣜAMI΋͋ΔΑʂ
    ami-­‐7d50c77c   •  ˠ http://docs.basho.com/riak/latest

30. Riakͷӡ༻ͷૢ࡞ 1/2 •  ϊʔυΛىಈ •  ϊʔυΛΫϥελʹ௥Ճ •  riak  start  

    •  riak-­‐admin  cluster  join  \   riak@10.0.0.1   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit   •  σʔλ഑ஔΛܭࢉ •  ΞΫηεΛྲྀ࢝͠ΊΔ

31. Riakͷӡ༻ͷૢ࡞ 2/2 •  ϊʔυΛΫϥελ͔Β֎͢ •  σʔλ഑ஔΛ࠶ܭࢉ •  σʔλΛ࠶഑ஔ •  riak-­‐admin

     down  \   riak@10.1.2.3   •  riak-­‐admin  cluster  plan   •  riak-­‐admin  cluster  commit  

32. ͓͢͢Ίπʔϧ •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ •  ߏ੒ྫ΋͋Γ·͢ •  https://github.com/ansible/ansible   •  https://github.com/ansible/ansible-­‐examples/

    tree/master/riak  

33. ͜Μͳӡ༻͕૊Ί·͢ ৸Δ ໷தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈ௥Ճ ໷தʹຊ౰ʹඞ ཁͳ਺·Ͱϊʔ υΛݮΒ͢ ى͖Δ

    ʢே͕͖ͨͷͰʣ ࡢ໷ͷ࠷େෛՙ ΛΈͯɺϊʔυ୆ ਺ͱൺֱ

34. খ࢝͘͞ΊΔͳΒ ˡWebServers •  m1.xlarge x3 •  Elastic Block Storage 256GB

    x3, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

35. ;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 256GB

    x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

36. ˓˓͕૿͖͑ͯͨΒ •  ୹ظతʢ1ʙ10िؒʣରࡦ •  ϚγϯΛ૿΍͢ •  ௕ظతରࡦ •  ϚγϯεϖοΫΛ্͛Δ

37. ෛՙ͕૿͖͑ͯͨΒ ˡWebServers •  m3.2xlarge x5 •  Elastic Block Storage 512GB

    x5, >4000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

38. σʔλ͕૿͖͑ͯͨΒ ˡWebServers •  m1.xlarge x5 •  Elastic Block Storage 2TB

    x5, >2000PIOPS •  ηΩϡϦςΟάϧʔϓ •  TCP 4369, 6000-7999, 8087, 8098, 8099

39. ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers •  hi1.4xlarge x5 •  SSD •  ηΩϡϦςΟάϧʔϓ • 

    TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive

40. όοΫΞοϓઓུ •  tarball ΛS3ʹ •  LVM Snapshot •  ͍Ζ͍Ζ͋Γ·͕͢… • 

    MDC Replication

41. Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ

42. RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠

43. ·ͱΊ •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍ •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ

    એ఻ ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ

44. One more thing… •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ •  ϙνϙν͢Δ͚ͩͰOK

45. Go to www.engineyard.com

46. ࣭໰ʁ •  Repo: https://github.com/basho/riak •  Twitterެࣜ: @BashoJapan or #riakjp • 

    Riak ML: riak-users-jp@lists.basho.com •  Me: kota@basho.com