Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Riak on EC2
Search
UENISHI Kota
September 28, 2013
Technology
6
2.8k
Riak on EC2
AWSでRiakを使うときのいろいろお話しました。
UENISHI Kota
September 28, 2013
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Storage Systems in Preferred Networks
kuenishi
0
41
Metadata Management in Distributed File Systems
kuenishi
2
520
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
410
Apache Ozone behind Simulation and AI Industries
kuenishi
0
380
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.2k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1.1k
Introducing Retz
kuenishi
5
1.2k
Introducing Retz and how to develop practical frameworks
kuenishi
3
740
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.4k
Other Decks in Technology
See All in Technology
Django's GeneratedField by example - DjangoCon US 2025
pauloxnet
0
160
2025/09/16 仕様駆動開発とAI-DLCが導くAI駆動開発の新フェーズ
masahiro_okamura
0
140
いま注目のAIエージェントを作ってみよう
supermarimobros
0
360
はじめてのOSS開発からみえたGo言語の強み
shibukazu
4
1k
JTCにおける内製×スクラム開発への挑戦〜内製化率95%達成の舞台裏/JTC's challenge of in-house development with Scrum
aeonpeople
0
270
Snowflake Intelligenceにはこうやって立ち向かう!クラシルが考えるAI Readyなデータ基盤と活用のためのDataOps
gappy50
0
280
Modern Linux
oracle4engineer
PRO
0
160
Firestore → Spanner 移行 を成功させた段階的移行プロセス
athug
1
500
プラットフォーム転換期におけるGitHub Copilot活用〜Coding agentがそれを加速するか〜 / Leveraging GitHub Copilot During Platform Transition Periods
aeonpeople
1
240
スマートファクトリーの第一歩 〜AWSマネージドサービスで 実現する予知保全と生成AI活用まで
ganota
2
320
slog.Handlerのよくある実装ミス
sakiengineer
4
480
要件定義・デザインフェーズでもAIを活用して、コミュニケーションの密度を高める
kazukihayase
0
120
Featured
See All Featured
Become a Pro
speakerdeck
PRO
29
5.5k
Mobile First: as difficult as doing things right
swwweet
224
9.9k
Writing Fast Ruby
sferik
628
62k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
The Cult of Friendly URLs
andyhume
79
6.6k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
810
Side Projects
sachag
455
43k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
127
53k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Java REST API Framework Comparison - PWX 2021
mraible
33
8.8k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
Transcript
Riak on EC2 JAWS Festa Osaka ্߁ଠ 2013/9/28
Who are you?? • ্߁ଠ (@kuenishi) • Bashoδϟύϯ • ࢄγεςϜྺ6
• Bashoྺ1ɺͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ • ͖ͳ˓˓
Agenda • What is Riak • U NO EC2? •
Riak + EC2 = ? • ߏྫ • ࣄྫʢলུ͞Ε·ͨ͠ʣ • Software Design 10/18 ങͬͯͶʂ
WAT IS RIAK?
ཧͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜ σʔλΛৗʹ߹ͤͭͭ͞ ɹαʔϏεࢭ·Δ͜ͱͳ͍ ɹσʔλϕʔεʯ ଘࡏ͠·ͤΜ = CAPఆཧ
• Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • Φʔϓϯιʔε • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
Dynamo จΑΓ • Dynamo: Amazon’s Highly Available Key-value Store •
Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
Riak Erlang/OTP্Ͱಈ࡞ • ΫϥελϦϯάϊʔυؒͷϝοηʔδϯάͳͲɺErlang/ OTPʹ·͔͍ͤͯΔ • Erlang/OTPͱʁ • ݩʑΤϦΫιϯ͕ࣾిަػϓϩάϥϜͷͨΊʹ։ൃɻ
RiakࢄKVS ɾRiak ӬଓੑΛ࣋ͬͨ Key Value Store
RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ 3%#ͷσʔλߏ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද K
V PK
ɾ1ͭͷRiakϊʔυɺOSϓϩηε্1ͭͷErlangΞϓϦ
ɾࢄσʔλϕʔεͰ͋ΔRiakෳϊʔυͰߏ
ɾBucketɺKeyΛྨ͢ΔͨΊͷ໊લۭؒ
ɾσʔλҟͳΔRiakϊʔυʹࣗಈෳ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF
OPEF
ɾΞΫηεͰ͖ͳ͍ϊʔυͷΞΫηεεΩοϓɻ
ɾোͳͲͰࣦΘΕͨσʔλͷෳࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF
ɾϊʔυՃ࣌ͳͲɺඞཁʹԠͯ͡࠶ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF +PJO
ɾෳͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ
RiakͷAPI • HTTP API or Protocol Buffers • put,
get, delete, list_keys, counter • PBͳΒ: http://github.com/basho/riak_pb • src/riak.proto, src/riak_kv.proto • Ruby, Java, Python, Erlang, PHP, …
ͭ·Γ… Riak • Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
͔ࣗ͠͠લͷϋʔυͩͱ… • εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ • ϋʔυΣΞωοτϫʔΫ͔Βࣗ Έʹ࡞ΕΔ • ෛՙྔʹ͕͋ͬͯɺ ࠷େʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ
• ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ͠ͳ͍ͱ͍͚ͳ͍
͍͘ΒιϑτΣΞͰؤுͬͯ • Մ༻ੑ • ωοτϫʔΫిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ • োੑ • ωοτϫʔΫిݯ͕ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •
ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻େม • εέʔϥϏϦςΟ • ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Εҙຯ͕ͳ͍
OMG
BUT… YOU KNOW EC2
AWS EC2 • Amazon ͕ࣗͨͪͷԾϚγϯཧͷΠϯϑϥΛ αʔϏεԽͨ͠ͷ • ҎԼͷಛ • Մ༻ੑʢAvailabilityʣ
• োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
AWS EC2͕͋Ε… • Մ༻ੑ • ωοτϫʔΫిݯجຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ • োੑ • ωοτϫʔΫిݯجຊతʹԽ͞Ε͍ͯΔ
• ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻Βͳ͍͍ͯ͘ • εέʔϥϏϦςΟ • ৽͍͠ϚγϯσΟεΫAPIͰҰൃήοτʂ
͜ΕͰউͭΔ
RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM
Riakͷߏங • Bashoͷapt-lineΛՃ • Πϯετʔϧ • sudo apt-‐get install riak
• ઃఆϑΝΠϧΛ2͚ͭࣗʹมߋ • ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ • ެࣜAMI͋ΔΑʂami-‐7d50c77c • ˠ http://docs.basho.com/riak/latest
Riakͷӡ༻ͷૢ࡞ 1/2 • ϊʔυΛىಈ • ϊʔυΛΫϥελʹՃ • riak start
• riak-‐admin cluster join \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit • σʔλஔΛܭࢉ • ΞΫηεΛྲྀ࢝͠ΊΔ
Riakͷӡ༻ͷૢ࡞ 2/2 • ϊʔυΛΫϥελ͔Β֎͢ • σʔλஔΛ࠶ܭࢉ • σʔλΛ࠶ஔ • riak-‐admin
down \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit
͓͢͢Ίπʔϧ • AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ • ߏྫ͋Γ·͢ • https://github.com/ansible/ansible • https://github.com/ansible/ansible-‐examples/
tree/master/riak
͜Μͳӡ༻͕Ί·͢ ৸Δ தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈՃ தʹຊʹඞ ཁͳ·Ͱϊʔ υΛݮΒ͢ ى͖Δ
ʢே͕͖ͨͷͰʣ ࡢͷ࠷େෛՙ ΛΈͯɺϊʔυ ͱൺֱ
খ࢝͘͞ΊΔͳΒ ˡWebServers • m1.xlarge x3 • Elastic Block Storage 256GB
x3, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 256GB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
˓˓͕૿͖͑ͯͨΒ • ظతʢ1ʙ10िؒʣରࡦ • ϚγϯΛ૿͢ • ظతରࡦ • ϚγϯεϖοΫΛ্͛Δ
ෛՙ͕૿͖͑ͯͨΒ ˡWebServers • m3.2xlarge x5 • Elastic Block Storage 512GB
x5, >4000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
σʔλ͕૿͖͑ͯͨΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 2TB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers • hi1.4xlarge x5 • SSD • ηΩϡϦςΟάϧʔϓ •
TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
όοΫΞοϓઓུ • tarball ΛS3ʹ • LVM Snapshot • ͍Ζ͍Ζ͋Γ·͕͢… •
MDC Replication
Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓෛՙࢄͳͲʹ
RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
·ͱΊ • Riak ͱ EC2 ͷΈ߹ΘͤͱͯΑ͍ • ؆୯ʹ͑Δˍ҆ఆ͍ͯ͠Δ • ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ͍ͬͯΔ
એ ɾιϑτΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃചͪΐ͏ͲRiak on EC2ͷࡉ͕ʣ
One more thing… • ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗͰؤுΔΑΓ100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ • ϙνϙν͢Δ͚ͩͰOK
Go to www.engineyard.com
࣭ʁ • Repo: https://github.com/basho/riak • Twitterެࣜ: @BashoJapan or #riakjp •
Riak ML:
[email protected]
• Me:
[email protected]