Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Riak on EC2
Search
UENISHI Kota
September 28, 2013
Technology
6
2.8k
Riak on EC2
AWSでRiakを使うときのいろいろお話しました。
UENISHI Kota
September 28, 2013
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Storage Systems in Preferred Networks
kuenishi
0
48
Metadata Management in Distributed File Systems
kuenishi
2
520
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
410
Apache Ozone behind Simulation and AI Industries
kuenishi
0
400
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.2k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1.1k
Introducing Retz
kuenishi
5
1.2k
Introducing Retz and how to develop practical frameworks
kuenishi
3
750
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.4k
Other Decks in Technology
See All in Technology
OPENLOGI Company Profile for engineer
hr01
1
45k
プロダクト開発と社内データ活用での、BI×AIの現在地 / Data_Findy
sansan_randd
1
550
会社を支える Pythonという言語戦略 ~なぜPythonを主要言語にしているのか?~
curekoshimizu
3
880
SRE × マネジメントレイヤーが挑戦した組織・会社のオブザーバビリティ改革 ― ビジネス価値と信頼性を両立するリアルな挑戦
coconala_engineer
0
280
IoTLT@ストラタシスジャパン_20251021
norioikedo
0
140
Dify on AWS 環境構築手順
yosse95ai
0
150
dbtとAIエージェントを組み合わせて見えたデータ調査の新しい形
10xinc
6
1.2k
re:Invent 2025の見どころと便利アイテムをご紹介 / Highlights and Useful Items for re:Invent 2025
yuj1osm
0
230
ざっくり学ぶ 『エンジニアリングリーダー 技術組織を育てるリーダーシップと セルフマネジメント』 / 50 minute Engineering Leader
iwashi86
3
1.1k
知覚とデザイン
rinchoku
1
610
.NET 10のBlazorの期待の新機能
htkym
0
150
マルチエージェントのチームビルディング_2025-10-25
shinoyamada
0
200
Featured
See All Featured
Become a Pro
speakerdeck
PRO
29
5.6k
Side Projects
sachag
455
43k
Designing Experiences People Love
moore
142
24k
Bash Introduction
62gerente
615
210k
Fireside Chat
paigeccino
41
3.7k
Building a Modern Day E-commerce SEO Strategy
aleyda
44
7.8k
How STYLIGHT went responsive
nonsquared
100
5.9k
Designing for Performance
lara
610
69k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
Embracing the Ebb and Flow
colly
88
4.9k
Making the Leap to Tech Lead
cromwellryan
135
9.6k
Building an army of robots
kneath
305
46k
Transcript
Riak on EC2 JAWS Festa Osaka ্߁ଠ 2013/9/28
Who are you?? • ্߁ଠ (@kuenishi) • Bashoδϟύϯ • ࢄγεςϜྺ6
• Bashoྺ1ɺͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ • ͖ͳ˓˓
Agenda • What is Riak • U NO EC2? •
Riak + EC2 = ? • ߏྫ • ࣄྫʢলུ͞Ε·ͨ͠ʣ • Software Design 10/18 ങͬͯͶʂ
WAT IS RIAK?
ཧͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜ σʔλΛৗʹ߹ͤͭͭ͞ ɹαʔϏεࢭ·Δ͜ͱͳ͍ ɹσʔλϕʔεʯ ଘࡏ͠·ͤΜ = CAPఆཧ
• Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • Φʔϓϯιʔε • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
Dynamo จΑΓ • Dynamo: Amazon’s Highly Available Key-value Store •
Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
Riak Erlang/OTP্Ͱಈ࡞ • ΫϥελϦϯάϊʔυؒͷϝοηʔδϯάͳͲɺErlang/ OTPʹ·͔͍ͤͯΔ • Erlang/OTPͱʁ • ݩʑΤϦΫιϯ͕ࣾిަػϓϩάϥϜͷͨΊʹ։ൃɻ
RiakࢄKVS ɾRiak ӬଓੑΛ࣋ͬͨ Key Value Store
RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ 3%#ͷσʔλߏ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද K
V PK
ɾ1ͭͷRiakϊʔυɺOSϓϩηε্1ͭͷErlangΞϓϦ
ɾࢄσʔλϕʔεͰ͋ΔRiakෳϊʔυͰߏ
ɾBucketɺKeyΛྨ͢ΔͨΊͷ໊લۭؒ
ɾσʔλҟͳΔRiakϊʔυʹࣗಈෳ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF
OPEF
ɾΞΫηεͰ͖ͳ͍ϊʔυͷΞΫηεεΩοϓɻ
ɾোͳͲͰࣦΘΕͨσʔλͷෳࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF
ɾϊʔυՃ࣌ͳͲɺඞཁʹԠͯ͡࠶ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF +PJO
ɾෳͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ
RiakͷAPI • HTTP API or Protocol Buffers • put,
get, delete, list_keys, counter • PBͳΒ: http://github.com/basho/riak_pb • src/riak.proto, src/riak_kv.proto • Ruby, Java, Python, Erlang, PHP, …
ͭ·Γ… Riak • Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
͔ࣗ͠͠લͷϋʔυͩͱ… • εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ • ϋʔυΣΞωοτϫʔΫ͔Βࣗ Έʹ࡞ΕΔ • ෛՙྔʹ͕͋ͬͯɺ ࠷େʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ
• ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ͠ͳ͍ͱ͍͚ͳ͍
͍͘ΒιϑτΣΞͰؤுͬͯ • Մ༻ੑ • ωοτϫʔΫిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ • োੑ • ωοτϫʔΫిݯ͕ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •
ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻େม • εέʔϥϏϦςΟ • ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Εҙຯ͕ͳ͍
OMG
BUT… YOU KNOW EC2
AWS EC2 • Amazon ͕ࣗͨͪͷԾϚγϯཧͷΠϯϑϥΛ αʔϏεԽͨ͠ͷ • ҎԼͷಛ • Մ༻ੑʢAvailabilityʣ
• োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
AWS EC2͕͋Ε… • Մ༻ੑ • ωοτϫʔΫిݯجຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ • োੑ • ωοτϫʔΫిݯجຊతʹԽ͞Ε͍ͯΔ
• ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻Βͳ͍͍ͯ͘ • εέʔϥϏϦςΟ • ৽͍͠ϚγϯσΟεΫAPIͰҰൃήοτʂ
͜ΕͰউͭΔ
RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM
Riakͷߏங • Bashoͷapt-lineΛՃ • Πϯετʔϧ • sudo apt-‐get install riak
• ઃఆϑΝΠϧΛ2͚ͭࣗʹมߋ • ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ • ެࣜAMI͋ΔΑʂami-‐7d50c77c • ˠ http://docs.basho.com/riak/latest
Riakͷӡ༻ͷૢ࡞ 1/2 • ϊʔυΛىಈ • ϊʔυΛΫϥελʹՃ • riak start
• riak-‐admin cluster join \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit • σʔλஔΛܭࢉ • ΞΫηεΛྲྀ࢝͠ΊΔ
Riakͷӡ༻ͷૢ࡞ 2/2 • ϊʔυΛΫϥελ͔Β֎͢ • σʔλஔΛ࠶ܭࢉ • σʔλΛ࠶ஔ • riak-‐admin
down \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit
͓͢͢Ίπʔϧ • AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ • ߏྫ͋Γ·͢ • https://github.com/ansible/ansible • https://github.com/ansible/ansible-‐examples/
tree/master/riak
͜Μͳӡ༻͕Ί·͢ ৸Δ தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈՃ தʹຊʹඞ ཁͳ·Ͱϊʔ υΛݮΒ͢ ى͖Δ
ʢே͕͖ͨͷͰʣ ࡢͷ࠷େෛՙ ΛΈͯɺϊʔυ ͱൺֱ
খ࢝͘͞ΊΔͳΒ ˡWebServers • m1.xlarge x3 • Elastic Block Storage 256GB
x3, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 256GB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
˓˓͕૿͖͑ͯͨΒ • ظతʢ1ʙ10िؒʣରࡦ • ϚγϯΛ૿͢ • ظతରࡦ • ϚγϯεϖοΫΛ্͛Δ
ෛՙ͕૿͖͑ͯͨΒ ˡWebServers • m3.2xlarge x5 • Elastic Block Storage 512GB
x5, >4000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
σʔλ͕૿͖͑ͯͨΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 2TB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers • hi1.4xlarge x5 • SSD • ηΩϡϦςΟάϧʔϓ •
TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
όοΫΞοϓઓུ • tarball ΛS3ʹ • LVM Snapshot • ͍Ζ͍Ζ͋Γ·͕͢… •
MDC Replication
Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓෛՙࢄͳͲʹ
RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
·ͱΊ • Riak ͱ EC2 ͷΈ߹ΘͤͱͯΑ͍ • ؆୯ʹ͑Δˍ҆ఆ͍ͯ͠Δ • ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ͍ͬͯΔ
એ ɾιϑτΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃചͪΐ͏ͲRiak on EC2ͷࡉ͕ʣ
One more thing… • ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗͰؤுΔΑΓ100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ • ϙνϙν͢Δ͚ͩͰOK
Go to www.engineyard.com
࣭ʁ • Repo: https://github.com/basho/riak • Twitterެࣜ: @BashoJapan or #riakjp •
Riak ML:
[email protected]
• Me:
[email protected]