Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Riak on EC2
UENISHI Kota
September 28, 2013
Technology
6
2.4k
Riak on EC2
AWSでRiakを使うときのいろいろお話しました。
UENISHI Kota
September 28, 2013
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
kuenishi
3
600
kuenishi
0
770
kuenishi
5
580
kuenishi
2
310
kuenishi
10
5.5k
kuenishi
7
1.9k
kuenishi
2
1.1k
kuenishi
1
210
kuenishi
3
640
Other Decks in Technology
See All in Technology
oracle4engineer
9
6.2k
sei88888
5
480
yashi8484
0
150
ks91
PRO
0
360
kawaguti
0
500
yushiku
4
810
smzksts
0
120
line_developers
PRO
2
400
finengine
0
280
udzura
0
180
michimani
0
150
techharmony
0
210
Featured
See All Featured
lauravandoore
12
1.7k
eileencodes
114
25k
addyosmani
312
21k
vanstee
118
4.9k
dougneiner
119
8k
3n
163
22k
jacobian
257
20k
akmur
252
19k
jakevdp
776
200k
sachag
267
17k
morganepeng
95
14k
robhawkes
53
2.9k
Transcript
Riak on EC2 JAWS Festa Osaka ্߁ଠ 2013/9/28
Who are you?? • ্߁ଠ (@kuenishi) • Bashoδϟύϯ • ࢄγεςϜྺ6
• Bashoྺ1ɺͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ • ͖ͳ˓˓
Agenda • What is Riak • U NO EC2? •
Riak + EC2 = ? • ߏྫ • ࣄྫʢলུ͞Ε·ͨ͠ʣ • Software Design 10/18 ങͬͯͶʂ
WAT IS RIAK?
ཧͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜ σʔλΛৗʹ߹ͤͭͭ͞ ɹαʔϏεࢭ·Δ͜ͱͳ͍ ɹσʔλϕʔεʯ ଘࡏ͠·ͤΜ = CAPఆཧ
• Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • Φʔϓϯιʔε • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
Dynamo จΑΓ • Dynamo: Amazon’s Highly Available Key-value Store •
Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
Riak Erlang/OTP্Ͱಈ࡞ • ΫϥελϦϯάϊʔυؒͷϝοηʔδϯάͳͲɺErlang/ OTPʹ·͔͍ͤͯΔ • Erlang/OTPͱʁ • ݩʑΤϦΫιϯ͕ࣾిަػϓϩάϥϜͷͨΊʹ։ൃɻ
RiakࢄKVS ɾRiak ӬଓੑΛ࣋ͬͨ Key Value Store
RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ 3%#ͷσʔλߏ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද K
V PK
ɾ1ͭͷRiakϊʔυɺOSϓϩηε্1ͭͷErlangΞϓϦ
ɾࢄσʔλϕʔεͰ͋ΔRiakෳϊʔυͰߏ
ɾBucketɺKeyΛྨ͢ΔͨΊͷ໊લۭؒ
ɾσʔλҟͳΔRiakϊʔυʹࣗಈෳ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF
OPEF
ɾΞΫηεͰ͖ͳ͍ϊʔυͷΞΫηεεΩοϓɻ
ɾোͳͲͰࣦΘΕͨσʔλͷෳࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF
ɾϊʔυՃ࣌ͳͲɺඞཁʹԠͯ͡࠶ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF +PJO
ɾෳͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ
RiakͷAPI • HTTP API or Protocol Buffers • put,
get, delete, list_keys, counter • PBͳΒ: http://github.com/basho/riak_pb • src/riak.proto, src/riak_kv.proto • Ruby, Java, Python, Erlang, PHP, …
ͭ·Γ… Riak • Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
͔ࣗ͠͠લͷϋʔυͩͱ… • εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ • ϋʔυΣΞωοτϫʔΫ͔Βࣗ Έʹ࡞ΕΔ • ෛՙྔʹ͕͋ͬͯɺ ࠷େʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ
• ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ͠ͳ͍ͱ͍͚ͳ͍
͍͘ΒιϑτΣΞͰؤுͬͯ • Մ༻ੑ • ωοτϫʔΫిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ • োੑ • ωοτϫʔΫిݯ͕ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •
ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻େม • εέʔϥϏϦςΟ • ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Εҙຯ͕ͳ͍
OMG
BUT… YOU KNOW EC2
AWS EC2 • Amazon ͕ࣗͨͪͷԾϚγϯཧͷΠϯϑϥΛ αʔϏεԽͨ͠ͷ • ҎԼͷಛ • Մ༻ੑʢAvailabilityʣ
• োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
AWS EC2͕͋Ε… • Մ༻ੑ • ωοτϫʔΫిݯجຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ • োੑ • ωοτϫʔΫిݯجຊతʹԽ͞Ε͍ͯΔ
• ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻Βͳ͍͍ͯ͘ • εέʔϥϏϦςΟ • ৽͍͠ϚγϯσΟεΫAPIͰҰൃήοτʂ
͜ΕͰউͭΔ
RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM
Riakͷߏங • Bashoͷapt-lineΛՃ • Πϯετʔϧ • sudo apt-‐get install riak
• ઃఆϑΝΠϧΛ2͚ͭࣗʹมߋ • ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ • ެࣜAMI͋ΔΑʂami-‐7d50c77c • ˠ http://docs.basho.com/riak/latest
Riakͷӡ༻ͷૢ࡞ 1/2 • ϊʔυΛىಈ • ϊʔυΛΫϥελʹՃ • riak start
• riak-‐admin cluster join \ riak@10.0.0.1 • riak-‐admin cluster plan • riak-‐admin cluster commit • σʔλஔΛܭࢉ • ΞΫηεΛྲྀ࢝͠ΊΔ
Riakͷӡ༻ͷૢ࡞ 2/2 • ϊʔυΛΫϥελ͔Β֎͢ • σʔλஔΛ࠶ܭࢉ • σʔλΛ࠶ஔ • riak-‐admin
down \ riak@10.1.2.3 • riak-‐admin cluster plan • riak-‐admin cluster commit
͓͢͢Ίπʔϧ • AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ • ߏྫ͋Γ·͢ • https://github.com/ansible/ansible • https://github.com/ansible/ansible-‐examples/
tree/master/riak
͜Μͳӡ༻͕Ί·͢ ৸Δ தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈՃ தʹຊʹඞ ཁͳ·Ͱϊʔ υΛݮΒ͢ ى͖Δ
ʢே͕͖ͨͷͰʣ ࡢͷ࠷େෛՙ ΛΈͯɺϊʔυ ͱൺֱ
খ࢝͘͞ΊΔͳΒ ˡWebServers • m1.xlarge x3 • Elastic Block Storage 256GB
x3, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 256GB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
˓˓͕૿͖͑ͯͨΒ • ظతʢ1ʙ10िؒʣରࡦ • ϚγϯΛ૿͢ • ظతରࡦ • ϚγϯεϖοΫΛ্͛Δ
ෛՙ͕૿͖͑ͯͨΒ ˡWebServers • m3.2xlarge x5 • Elastic Block Storage 512GB
x5, >4000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
σʔλ͕૿͖͑ͯͨΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 2TB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers • hi1.4xlarge x5 • SSD • ηΩϡϦςΟάϧʔϓ •
TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
όοΫΞοϓઓུ • tarball ΛS3ʹ • LVM Snapshot • ͍Ζ͍Ζ͋Γ·͕͢… •
MDC Replication
Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓෛՙࢄͳͲʹ
RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
·ͱΊ • Riak ͱ EC2 ͷΈ߹ΘͤͱͯΑ͍ • ؆୯ʹ͑Δˍ҆ఆ͍ͯ͠Δ • ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ͍ͬͯΔ
એ ɾιϑτΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃചͪΐ͏ͲRiak on EC2ͷࡉ͕ʣ
One more thing… • ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗͰؤுΔΑΓ100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ • ϙνϙν͢Δ͚ͩͰOK
Go to www.engineyard.com
࣭ʁ • Repo: https://github.com/basho/riak • Twitterެࣜ: @BashoJapan or #riakjp •
Riak ML: riak-users-jp@lists.basho.com • Me: kota@basho.com