Riak on EC2

AWSでRiakを使うときのいろいろお話しました。

Transcript

1. Riak on EC2
   JAWS Festa Osaka
   ্੢߁ଠ
   2013/9/28
   

   View Slide

2. Who are you??
   •  ্੢߁ଠ (@kuenishi)
   
   •  Bashoδϟύϯ
   
   •  ෼ࢄγεςϜྺ6೥
   •  Bashoྺ1೥ɺ΍ͬͱίʔ
   υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ
   
   •  ޷͖ͳ˓˓
   
   

   View Slide

3. Agenda
   •  What is Riak
   
   •  U NO EC2?
   
   •  Riak + EC2 = ?
   
   •  ߏ੒ྫ
   
   •  ࣄྫʢলུ͞Ε·ͨ͠ʣ
   
   •  Software Design 10/18 ങͬͯͶʂ
   

   View Slide

4. WAT IS RIAK?
   

   View Slide

5. ཧ૝ͷσʔλϕʔε
   ʮͲΜͳނো͕ىͬͯ͜΋
   
   σʔλΛৗʹ੔߹ͤͭͭ͞
   
   ɹαʔϏε΋ࢭ·Δ͜ͱ͸ͳ͍
   
   ɹσʔλϕʔεʯ
   
   
   ͸ଘࡏ͠·ͤΜ = CAPఆཧ
   
   

   View Slide

6. •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS
   
   •  Φʔϓϯιʔε
   
   •  ҎԼʹॏ఺Λஔ͍͍ͯΔ
   
   •  Մ༻ੑʢAvailabilityʣ
   
   •  ଱ো֐ੑʢFault-toleranceʣ
   
   •  ӡ༻ͷ༰қੑʢOperational simplicityʣ
   
   •  εέʔϥϏϦςΟʢScalabilityʣ
   
   
   
   

   View Slide

7. Dynamo ࿦จΑΓ
   •  Dynamo: Amazon’s Highly Available Key-value Store
   
   •  Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ

   View Slide

8. Riak͸ Erlang/OTP্Ͱಈ࡞
   •  ΫϥελϦϯά΍ϊʔυؒͷϝοηʔδϯάͳͲ͸ɺErlang/
   OTPʹ·͔͍ͤͯΔ
   
   •  Erlang/OTPͱ͸ʁ
   
   •  ݩʑ͸ΤϦΫιϯ͕ࣾి࿩ަ׵ػϓϩάϥϜͷͨΊʹ։ൃɻ

   View Slide

9. Riak͸෼ࢄKVS
   ɾRiak ͸ӬଓੑΛ࣋ͬͨ Key Value Store

   View Slide

10. RiakͷσʔλͱRDBͷσʔλൺֱ
    3JBLͷσʔλߏ଄ 3%#ͷσʔλߏ଄
    PCKFDU SFDPSEʢߦʣ
    #VDLFU 5BCMF ද
    
    K V PK
    

    View Slide

11. ɾ1ͭͷRiakϊʔυ͸ɺOSϓϩηε্͸1ͭͷErlangΞϓϦ

    View Slide

12. ɾ෼ࢄσʔλϕʔεͰ͋ΔRiak͸ෳ਺ϊʔυͰߏ੒

    View Slide

13. ɾBucket͸ɺKeyΛ෼ྨ͢ΔͨΊͷ໊લۭؒ

    View Slide

14. ɾσʔλ͸ҟͳΔRiakϊʔυʹࣗಈෳ੡͞ΕΔ
    
    Riakʹ͓͚Δ
    σʔλͷࣗಈϨϓϦέʔγϣϯ
    K V K V
    OPEF OPEF OPEF OPEF
    

    View Slide

15. ɾΞΫηεͰ͖ͳ͍ϊʔυ΁ͷΞΫηε͸εΩοϓɻ

    View Slide

16. ɾো֐ͳͲͰࣦΘΕͨσʔλͷෳ੡͸ࣗಈ෮چ͞ΕΔ
    
    Riakʹ͓͚Δ
    σʔλͷࣗಈ෮چ
    OPEF OPEF OPEF OPEF
    OPEF OPEF OPEF OPEF
    

    View Slide

17. ɾϊʔυ௥Ճ࣌ͳͲɺඞཁʹԠͯ͡࠶഑ஔ͕ߦΘΕΔ
    
    Riakʹ͓͚Δ
    σʔλͷࣗಈ࠶഑ஔ
    OPEF OPEF OPEF OPEF
    OPEF OPEF OPEF OPEF
    +PJO
    

    View Slide

18. ɾෳ਺ͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ

    View Slide

19. RiakͷAPI
    •  HTTP  API  or  Protocol  Buffers  
    •  put,  get,  delete,  list_keys,  counter  
    •  PBͳΒ:  http://github.com/basho/riak_pb  
    •  src/riak.proto,  src/riak_kv.proto  
    •  Ruby,  Java,  Python,  Erlang,  PHP,  …  
    

    View Slide

20. ͭ·Γ… Riak͸
    •  Amazon Dynamo࿦จΛϕʔεʹઃܭ͞Εͨ෼ࢄKVS
    
    •  ҎԼʹॏ఺Λஔ͍͍ͯΔ
    
    •  Մ༻ੑʢAvailabilityʣ
    
    •  ଱ো֐ੑʢFault-toleranceʣ
    
    •  ӡ༻ͷ༰қੑʢOperational simplicityʣ
    
    •  εέʔϥϏϦςΟʢScalabilityʣ
    
    
    
    

    View Slide

21. ͔ࣗ͠͠લͷϋʔυͩͱ…
    •  εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆
    
    •  ϋʔυ΢ΣΞ΍ωοτϫʔΫ͔Βࣗ෼޷
    Έʹ࡞ΕΔ
    
    •  ෛՙ΍ྔʹ೾͕͋ͬͯɺ
    ࠷େ஋ʹ߹Θͤͯௐୡ͠
    ͳ͍ͱ͍͚ͳ͍ͱׂߴ
    
    •  ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ
    ߏ੒͠ͳ͍ͱ͍͚ͳ͍
    
    

    View Slide

22. ͍͘Βιϑτ΢ΣΞͰؤுͬͯ΋
    •  Մ༻ੑ
    
    •  ωοτϫʔΫ΍ిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍
    
    •  ଱ো֐ੑ
    
    •  ωοτϫʔΫ΍ిݯ͕୔ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍
    
    •  ӡ༻
    
    •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸େม
    
    •  εέʔϥϏϦςΟ
    
    •  ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Ε͹ҙຯ͕ͳ͍
    
    

    View Slide

23. OMG
    

    View Slide

24. BUT… YOU KNOW EC2
    

    View Slide

25. AWS EC2
    •  Amazon ͕ࣗ෼ͨͪͷԾ૝Ϛγϯ؅ཧͷΠϯϑϥΛ
    αʔϏεԽͨ͠΋ͷ
    
    •  ҎԼͷಛ௃
    
    •  Մ༻ੑʢAvailabilityʣ
    
    •  ଱ো֐ੑʢFault-toleranceʣ
    
    •  ӡ༻ͷ༰қੑʢOperational simplicityʣ
    
    •  εέʔϥϏϦςΟʢScalabilityʣ
    
    
    
    

    View Slide

26. AWS EC2͕͋Ε͹…
    •  Մ༻ੑ
    
    •  ωοτϫʔΫ΍ిݯ͸جຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ
    
    •  ଱ো֐ੑ
    
    •  ωοτϫʔΫ΍ిݯ͸جຊతʹ৑௕Խ͞Ε͍ͯΔ
    
    •  ӡ༻
    
    •  ϋʔυ΢ΣΞ΍ωοτϫʔΫͷӡ༻͸΍Βͳ͍͍ͯ͘
    
    •  εέʔϥϏϦςΟ
    
    •  ৽͍͠Ϛγϯ΋σΟεΫ΋APIͰҰൃήοτʂ
    
    

    View Slide

27. ͜ΕͰউͭΔ
    

    View Slide

28. RIAK + EC2 =
    EASY OPERATION AND SCALABLE SYSTEM
    

    View Slide

29. Riakͷߏங
    •  Bashoͷapt-lineΛ௥Ճ
    
    •  Πϯετʔϧ
    
    •  sudo  apt-­‐get  install  riak  
    •  ઃఆϑΝΠϧΛ2ͭࣗ෼޲͚ʹมߋ
    
    •  ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ
    
    •  ެࣜAMI΋͋ΔΑʂami-­‐7d50c77c  
    •  ˠ http://docs.basho.com/riak/latest
    
    

    View Slide

30. Riakͷӡ༻ͷૢ࡞ 1/2
    •  ϊʔυΛىಈ
    
    •  ϊʔυΛΫϥελʹ௥Ճ
    
    •  riak  start  
    •  riak-­‐admin  cluster  join  \  
    riak@10.0.0.1  
    •  riak-­‐admin  cluster  plan  
    •  riak-­‐admin  cluster  commit  
    •  σʔλ഑ஔΛܭࢉ
    
    •  ΞΫηεΛྲྀ࢝͠ΊΔ
    
    

    View Slide

31. Riakͷӡ༻ͷૢ࡞ 2/2
    •  ϊʔυΛΫϥελ͔Β֎͢
    
    •  σʔλ഑ஔΛ࠶ܭࢉ
    
    •  σʔλΛ࠶഑ஔ
    
    •  riak-­‐admin  down  \  
    riak@10.1.2.3  
    •  riak-­‐admin  cluster  plan  
    •  riak-­‐admin  cluster  commit  
    

    View Slide

32. ͓͢͢Ίπʔϧ
    •  AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢
    
    •  ߏ੒ྫ΋͋Γ·͢
    
    •  https://github.com/ansible/ansible  
    •  https://github.com/ansible/ansible-­‐examples/
    tree/master/riak  
    

    View Slide

33. ͜Μͳӡ༻͕૊Ί·͢
    ৸Δ
    
    ໷தʹෛՙ͕
    ্͕Δ
    
    APIͰϊʔυ
    ࣗಈ௥Ճ
    
    ໷தʹຊ౰ʹඞ
    ཁͳ਺·Ͱϊʔ
    υΛݮΒ͢
    
    ى͖Δ
    
    ʢே͕͖ͨͷͰʣ
    
    ࡢ໷ͷ࠷େෛՙ
    ΛΈͯɺϊʔυ୆
    ਺ͱൺֱ
    
    

    View Slide

34. খ࢝͘͞ΊΔͳΒ
    ˡWebServers
    
    •  m1.xlarge x3
    
    •  Elastic Block Storage
    256GB x3, >2000PIOPS
    
    •  ηΩϡϦςΟάϧʔϓ
    
    •  TCP 4369, 6000-7999, 8087,
    8098, 8099
    
    

    View Slide

35. ;ͭ͏ʹ࢝ΊΔͳΒ
    ˡWebServers
    
    •  m1.xlarge x5
    
    •  Elastic Block Storage
    256GB x5, >2000PIOPS
    
    •  ηΩϡϦςΟάϧʔϓ
    
    •  TCP 4369, 6000-7999, 8087,
    8098, 8099
    
    

    View Slide

36. ˓˓͕૿͖͑ͯͨΒ
    •  ୹ظతʢ1ʙ10िؒʣରࡦ
    
    •  ϚγϯΛ૿΍͢
    
    •  ௕ظతରࡦ
    
    •  ϚγϯεϖοΫΛ্͛Δ
    
    

    View Slide

37. ෛՙ͕૿͖͑ͯͨΒ
    ˡWebServers
    
    •  m3.2xlarge x5
    
    •  Elastic Block Storage
    512GB x5, >4000PIOPS
    
    •  ηΩϡϦςΟάϧʔϓ
    
    •  TCP 4369, 6000-7999, 8087,
    8098, 8099
    
    

    View Slide

38. σʔλ͕૿͖͑ͯͨΒ
    ˡWebServers
    
    •  m1.xlarge x5
    
    •  Elastic Block Storage
    2TB x5, >2000PIOPS
    
    •  ηΩϡϦςΟάϧʔϓ
    
    •  TCP 4369, 6000-7999, 8087,
    8098, 8099
    
    

    View Slide

39. ෛՙ͕૿͖͑ͯͨΒ̎
    ˡWebServers
    
    •  hi1.4xlarge x5
    
    •  SSD
    
    •  ηΩϡϦςΟάϧʔϓ
    
    •  TCP 4369, 6000-7999, 8087,
    8098, 8099
    
    File
    
    Archive
    
    

    View Slide

40. όοΫΞοϓઓུ
    •  tarball ΛS3ʹ
    
    •  LVM Snapshot
    
    •  ͍Ζ͍Ζ͋Γ·͕͢…
    
    •  MDC Replication
    
    

    View Slide

41. Availability Zone Availability Zone
    ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ
    
    όοΫΞοϓ΍ෛՙ෼ࢄͳͲʹ
    
    

    View Slide

42. RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
    

    View Slide

43. ·ͱΊ
    •  Riak ͱ EC2 ͷ૊Έ߹Θͤ͸ͱͯ΋Α͍
    
    •  ؆୯ʹ࢖͑Δˍ҆ఆ͍ͯ͠Δ
    
    •  ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ࢖͍ͬͯΔ
    
    એ఻
    
    ɾιϑτ΢ΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢
    
    ɾ11݄߸ʢ10/18ൃച͸ͪΐ͏ͲRiak on EC2ͷࡉ࿦͕ʣ
    
    

    View Slide

44. One more thing…
    •  ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗ෼ͰؤுΔΑΓ΋100ഒָ
    ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢
    
    •  ϙνϙν͢Δ͚ͩͰOK
    
    

    View Slide

45. Go to www.engineyard.com
    
    

    View Slide

46. ࣭໰ʁ
    •  Repo: https://github.com/basho/riak
    
    •  Twitterެࣜ: @BashoJapan or #riakjp
    
    •  Riak ML: riak-users-jp@lists.basho.com
    
    •  Me: kota@basho.com
    
    

    View Slide