Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Riak on EC2
Search
UENISHI Kota
September 28, 2013
Technology
6
2.7k
Riak on EC2
AWSでRiakを使うときのいろいろお話しました。
UENISHI Kota
September 28, 2013
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Metadata Management in Distributed File Systems
kuenishi
2
470
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
340
Apache Ozone behind Simulation and AI Industries
kuenishi
0
310
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.1k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1k
Introducing Retz
kuenishi
5
1.1k
Introducing Retz and how to develop practical frameworks
kuenishi
3
700
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.3k
Mesos Frameworkの作り方 (How to Make Mesos Framework)
kuenishi
7
2.3k
Other Decks in Technology
See All in Technology
2025年の挑戦 コーポレートエンジニアの技術広報/techpr5
nishiuma
0
120
AWS Community Builderのススメ - みんなもCommunity Builderに応募しよう! -
smt7174
0
120
rootful・rootless・privilegedコンテナの違い/rootful_rootless_privileged_container_difference
moz_sec_
0
130
Building Scalable Backend Services with Firebase
wisdommatt
0
110
チームが毎日小さな変化と適応を続けたら1年間でスケール可能なアジャイルチームができた話 / Building a Scalable Agile Team
kakehashi
1
200
12 Days of OpenAIから読み解く、生成AI 2025年のトレンド
shunsukeono_am
0
1.2k
効率的な技術組織が作れる!書籍『チームトポロジー』要点まとめ
iwamot
2
200
The future we create with our own MVV
matsukurou
0
1.8k
🌏丸い地球を効率的に平たくする 〜🗺️地図の幾何学とWeb地図技術〜
syotasasaki593876
0
120
30分でわかる「リスクから学ぶKubernetesコンテナセキュリティ」/30min-k8s-container-sec
mochizuki875
3
410
2025年のARグラスの潮流
kotauchisunsun
0
740
Unlearn Product Development - Unleashed Edition
lemiorhan
PRO
2
170
Featured
See All Featured
Build your cross-platform service in a week with App Engine
jlugia
229
18k
How GitHub (no longer) Works
holman
312
140k
Unsuck your backbone
ammeep
669
57k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Site-Speed That Sticks
csswizardry
2
240
Testing 201, or: Great Expectations
jmmastey
41
7.2k
Building a Scalable Design System with Sketch
lauravandoore
460
33k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
3
350
Embracing the Ebb and Flow
colly
84
4.5k
The Invisible Side of Design
smashingmag
299
50k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Transcript
Riak on EC2 JAWS Festa Osaka ্߁ଠ 2013/9/28
Who are you?? • ্߁ଠ (@kuenishi) • Bashoδϟύϯ • ࢄγεςϜྺ6
• Bashoྺ1ɺͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ • ͖ͳ˓˓
Agenda • What is Riak • U NO EC2? •
Riak + EC2 = ? • ߏྫ • ࣄྫʢলུ͞Ε·ͨ͠ʣ • Software Design 10/18 ങͬͯͶʂ
WAT IS RIAK?
ཧͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜ σʔλΛৗʹ߹ͤͭͭ͞ ɹαʔϏεࢭ·Δ͜ͱͳ͍ ɹσʔλϕʔεʯ ଘࡏ͠·ͤΜ = CAPఆཧ
• Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • Φʔϓϯιʔε • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
Dynamo จΑΓ • Dynamo: Amazon’s Highly Available Key-value Store •
Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
Riak Erlang/OTP্Ͱಈ࡞ • ΫϥελϦϯάϊʔυؒͷϝοηʔδϯάͳͲɺErlang/ OTPʹ·͔͍ͤͯΔ • Erlang/OTPͱʁ • ݩʑΤϦΫιϯ͕ࣾిަػϓϩάϥϜͷͨΊʹ։ൃɻ
RiakࢄKVS ɾRiak ӬଓੑΛ࣋ͬͨ Key Value Store
RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ 3%#ͷσʔλߏ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද K
V PK
ɾ1ͭͷRiakϊʔυɺOSϓϩηε্1ͭͷErlangΞϓϦ
ɾࢄσʔλϕʔεͰ͋ΔRiakෳϊʔυͰߏ
ɾBucketɺKeyΛྨ͢ΔͨΊͷ໊લۭؒ
ɾσʔλҟͳΔRiakϊʔυʹࣗಈෳ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF
OPEF
ɾΞΫηεͰ͖ͳ͍ϊʔυͷΞΫηεεΩοϓɻ
ɾোͳͲͰࣦΘΕͨσʔλͷෳࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF
ɾϊʔυՃ࣌ͳͲɺඞཁʹԠͯ͡࠶ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF +PJO
ɾෳͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ
RiakͷAPI • HTTP API or Protocol Buffers • put,
get, delete, list_keys, counter • PBͳΒ: http://github.com/basho/riak_pb • src/riak.proto, src/riak_kv.proto • Ruby, Java, Python, Erlang, PHP, …
ͭ·Γ… Riak • Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
͔ࣗ͠͠લͷϋʔυͩͱ… • εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ • ϋʔυΣΞωοτϫʔΫ͔Βࣗ Έʹ࡞ΕΔ • ෛՙྔʹ͕͋ͬͯɺ ࠷େʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ
• ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ͠ͳ͍ͱ͍͚ͳ͍
͍͘ΒιϑτΣΞͰؤுͬͯ • Մ༻ੑ • ωοτϫʔΫిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ • োੑ • ωοτϫʔΫిݯ͕ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •
ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻େม • εέʔϥϏϦςΟ • ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Εҙຯ͕ͳ͍
OMG
BUT… YOU KNOW EC2
AWS EC2 • Amazon ͕ࣗͨͪͷԾϚγϯཧͷΠϯϑϥΛ αʔϏεԽͨ͠ͷ • ҎԼͷಛ • Մ༻ੑʢAvailabilityʣ
• োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
AWS EC2͕͋Ε… • Մ༻ੑ • ωοτϫʔΫిݯجຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ • োੑ • ωοτϫʔΫిݯجຊతʹԽ͞Ε͍ͯΔ
• ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻Βͳ͍͍ͯ͘ • εέʔϥϏϦςΟ • ৽͍͠ϚγϯσΟεΫAPIͰҰൃήοτʂ
͜ΕͰউͭΔ
RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM
Riakͷߏங • Bashoͷapt-lineΛՃ • Πϯετʔϧ • sudo apt-‐get install riak
• ઃఆϑΝΠϧΛ2͚ͭࣗʹมߋ • ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ • ެࣜAMI͋ΔΑʂami-‐7d50c77c • ˠ http://docs.basho.com/riak/latest
Riakͷӡ༻ͷૢ࡞ 1/2 • ϊʔυΛىಈ • ϊʔυΛΫϥελʹՃ • riak start
• riak-‐admin cluster join \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit • σʔλஔΛܭࢉ • ΞΫηεΛྲྀ࢝͠ΊΔ
Riakͷӡ༻ͷૢ࡞ 2/2 • ϊʔυΛΫϥελ͔Β֎͢ • σʔλஔΛ࠶ܭࢉ • σʔλΛ࠶ஔ • riak-‐admin
down \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit
͓͢͢Ίπʔϧ • AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ • ߏྫ͋Γ·͢ • https://github.com/ansible/ansible • https://github.com/ansible/ansible-‐examples/
tree/master/riak
͜Μͳӡ༻͕Ί·͢ ৸Δ தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈՃ தʹຊʹඞ ཁͳ·Ͱϊʔ υΛݮΒ͢ ى͖Δ
ʢே͕͖ͨͷͰʣ ࡢͷ࠷େෛՙ ΛΈͯɺϊʔυ ͱൺֱ
খ࢝͘͞ΊΔͳΒ ˡWebServers • m1.xlarge x3 • Elastic Block Storage 256GB
x3, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 256GB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
˓˓͕૿͖͑ͯͨΒ • ظతʢ1ʙ10िؒʣରࡦ • ϚγϯΛ૿͢ • ظతରࡦ • ϚγϯεϖοΫΛ্͛Δ
ෛՙ͕૿͖͑ͯͨΒ ˡWebServers • m3.2xlarge x5 • Elastic Block Storage 512GB
x5, >4000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
σʔλ͕૿͖͑ͯͨΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 2TB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers • hi1.4xlarge x5 • SSD • ηΩϡϦςΟάϧʔϓ •
TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
όοΫΞοϓઓུ • tarball ΛS3ʹ • LVM Snapshot • ͍Ζ͍Ζ͋Γ·͕͢… •
MDC Replication
Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓෛՙࢄͳͲʹ
RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
·ͱΊ • Riak ͱ EC2 ͷΈ߹ΘͤͱͯΑ͍ • ؆୯ʹ͑Δˍ҆ఆ͍ͯ͠Δ • ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ͍ͬͯΔ
એ ɾιϑτΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃചͪΐ͏ͲRiak on EC2ͷࡉ͕ʣ
One more thing… • ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗͰؤுΔΑΓ100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ • ϙνϙν͢Δ͚ͩͰOK
Go to www.engineyard.com
࣭ʁ • Repo: https://github.com/basho/riak • Twitterެࣜ: @BashoJapan or #riakjp •
Riak ML:
[email protected]
• Me:
[email protected]