Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Riak on EC2
Search
UENISHI Kota
September 28, 2013
Technology
6
2.7k
Riak on EC2
AWSでRiakを使うときのいろいろお話しました。
UENISHI Kota
September 28, 2013
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Metadata Management in Distributed File Systems
kuenishi
2
470
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
330
Apache Ozone behind Simulation and AI Industries
kuenishi
0
300
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.1k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1k
Introducing Retz
kuenishi
5
1.1k
Introducing Retz and how to develop practical frameworks
kuenishi
3
680
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.3k
Mesos Frameworkの作り方 (How to Make Mesos Framework)
kuenishi
7
2.3k
Other Decks in Technology
See All in Technology
Google Cloud で始める Cloud Run 〜AWSとの比較と実例デモで解説〜
risatube
PRO
0
100
PHP ユーザのための OpenTelemetry 入門 / phpcon2024-opentelemetry
shin1x1
1
210
株式会社ログラス − エンジニア向け会社説明資料 / Loglass Comapany Deck for Engineer
loglass2019
3
32k
プロダクト開発を加速させるためのQA文化の築き方 / How to build QA culture to accelerate product development
mii3king
1
260
AWS re:Invent 2024で発表された コードを書く開発者向け機能について
maruto
0
190
KubeCon NA 2024 Recap: How to Move from Ingress to Gateway API with Minimal Hassle
ysakotch
0
200
UI State設計とテスト方針
rmakiyama
2
580
組織に自動テストを書く文化を根付かせる戦略(2024冬版) / Building Automated Test Culture 2024 Winter Edition
twada
PRO
13
3.8k
フロントエンド設計にモブ設計を導入してみた / 20241212_cloudsign_TechFrontMeetup
bengo4com
0
1.9k
マイクロサービスにおける容易なトランザクション管理に向けて
scalar
0
130
日本版とグローバル版のモバイルアプリ統合の開発の裏側と今後の展望
miichan
1
130
オプトインカメラ:UWB測位を応用したオプトイン型のカメラ計測
matthewlujp
0
170
Featured
See All Featured
Rails Girls Zürich Keynote
gr2m
94
13k
Building Adaptive Systems
keathley
38
2.3k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.3k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
5
450
Testing 201, or: Great Expectations
jmmastey
40
7.1k
Large-scale JavaScript Application Architecture
addyosmani
510
110k
Java REST API Framework Comparison - PWX 2021
mraible
28
8.3k
Bash Introduction
62gerente
608
210k
Optimizing for Happiness
mojombo
376
70k
VelocityConf: Rendering Performance Case Studies
addyosmani
326
24k
Transcript
Riak on EC2 JAWS Festa Osaka ্߁ଠ 2013/9/28
Who are you?? • ্߁ଠ (@kuenishi) • Bashoδϟύϯ • ࢄγεςϜྺ6
• Bashoྺ1ɺͬͱίʔ υॻ͕࣌ؒ͘Ͱ͖͖ͯͨ • ͖ͳ˓˓
Agenda • What is Riak • U NO EC2? •
Riak + EC2 = ? • ߏྫ • ࣄྫʢলུ͞Ε·ͨ͠ʣ • Software Design 10/18 ങͬͯͶʂ
WAT IS RIAK?
ཧͷσʔλϕʔε ʮͲΜͳނো͕ىͬͯ͜ σʔλΛৗʹ߹ͤͭͭ͞ ɹαʔϏεࢭ·Δ͜ͱͳ͍ ɹσʔλϕʔεʯ ଘࡏ͠·ͤΜ = CAPఆཧ
• Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • Φʔϓϯιʔε • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
Dynamo จΑΓ • Dynamo: Amazon’s Highly Available Key-value Store •
Մ༻ੑୈҰɿৗʹॻ͖ࠐΈ͕Ͱ͖Δ͜ͱ
Riak Erlang/OTP্Ͱಈ࡞ • ΫϥελϦϯάϊʔυؒͷϝοηʔδϯάͳͲɺErlang/ OTPʹ·͔͍ͤͯΔ • Erlang/OTPͱʁ • ݩʑΤϦΫιϯ͕ࣾిަػϓϩάϥϜͷͨΊʹ։ൃɻ
RiakࢄKVS ɾRiak ӬଓੑΛ࣋ͬͨ Key Value Store
RiakͷσʔλͱRDBͷσʔλൺֱ 3JBLͷσʔλߏ 3%#ͷσʔλߏ PCKFDU SFDPSEʢߦʣ #VDLFU 5BCMF ද K
V PK
ɾ1ͭͷRiakϊʔυɺOSϓϩηε্1ͭͷErlangΞϓϦ
ɾࢄσʔλϕʔεͰ͋ΔRiakෳϊʔυͰߏ
ɾBucketɺKeyΛྨ͢ΔͨΊͷ໊લۭؒ
ɾσʔλҟͳΔRiakϊʔυʹࣗಈෳ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈϨϓϦέʔγϣϯ K V K V OPEF OPEF OPEF
OPEF
ɾΞΫηεͰ͖ͳ͍ϊʔυͷΞΫηεεΩοϓɻ
ɾোͳͲͰࣦΘΕͨσʔλͷෳࣗಈ෮چ͞ΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ෮چ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF
ɾϊʔυՃ࣌ͳͲɺඞཁʹԠͯ͡࠶ஔ͕ߦΘΕΔ Riakʹ͓͚Δ σʔλͷࣗಈ࠶ஔ OPEF OPEF OPEF OPEF OPEF OPEF OPEF
OPEF +PJO
ɾෳͷRiakΫϥελʔؒͰɺσʔλͷϨϓϦέʔγϣϯ
RiakͷAPI • HTTP API or Protocol Buffers • put,
get, delete, list_keys, counter • PBͳΒ: http://github.com/basho/riak_pb • src/riak.proto, src/riak_kv.proto • Ruby, Java, Python, Erlang, PHP, …
ͭ·Γ… Riak • Amazon DynamoจΛϕʔεʹઃܭ͞ΕͨࢄKVS • ҎԼʹॏΛஔ͍͍ͯΔ • Մ༻ੑʢAvailabilityʣ •
োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
͔ࣗ͠͠લͷϋʔυͩͱ… • εέʔϧ͕େ͖͍ˍܭըՄೳͳΒ͍҆ • ϋʔυΣΞωοτϫʔΫ͔Βࣗ Έʹ࡞ΕΔ • ෛՙྔʹ͕͋ͬͯɺ ࠷େʹ߹Θͤͯௐୡ͠ ͳ͍ͱ͍͚ͳ͍ͱׂߴ
• ࡉ͔͍ͱ͜Ζ·Ͱߟ͑ͯ ߏ͠ͳ͍ͱ͍͚ͳ͍
͍͘ΒιϑτΣΞͰؤுͬͯ • Մ༻ੑ • ωοτϫʔΫిݯ͕མ͍ͪͯͨΒҙຯ͕ͳ͍ • োੑ • ωοτϫʔΫిݯ͕ࢁյΕ͍ͯͨΒҙຯ͕ͳ͍ •
ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻େม • εέʔϥϏϦςΟ • ؊৺ͳͱ͖ʹαʔόʔΛௐୡͰ͖ͳ͚Εҙຯ͕ͳ͍
OMG
BUT… YOU KNOW EC2
AWS EC2 • Amazon ͕ࣗͨͪͷԾϚγϯཧͷΠϯϑϥΛ αʔϏεԽͨ͠ͷ • ҎԼͷಛ • Մ༻ੑʢAvailabilityʣ
• োੑʢFault-toleranceʣ • ӡ༻ͷ༰қੑʢOperational simplicityʣ • εέʔϥϏϦςΟʢScalabilityʣ
AWS EC2͕͋Ε… • Մ༻ੑ • ωοτϫʔΫిݯجຊతʹམͪͳ͍ͱࢥ͍͍ͬͯ • োੑ • ωοτϫʔΫిݯجຊతʹԽ͞Ε͍ͯΔ
• ӡ༻ • ϋʔυΣΞωοτϫʔΫͷӡ༻Βͳ͍͍ͯ͘ • εέʔϥϏϦςΟ • ৽͍͠ϚγϯσΟεΫAPIͰҰൃήοτʂ
͜ΕͰউͭΔ
RIAK + EC2 = EASY OPERATION AND SCALABLE SYSTEM
Riakͷߏங • Bashoͷapt-lineΛՃ • Πϯετʔϧ • sudo apt-‐get install riak
• ઃఆϑΝΠϧΛ2͚ͭࣗʹมߋ • ΧʔωϧपΓΛ͍͔ͭ͘ઃఆʢεέδϡʔϥͳͲͳͲʣ • ެࣜAMI͋ΔΑʂami-‐7d50c77c • ˠ http://docs.basho.com/riak/latest
Riakͷӡ༻ͷૢ࡞ 1/2 • ϊʔυΛىಈ • ϊʔυΛΫϥελʹՃ • riak start
• riak-‐admin cluster join \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit • σʔλஔΛܭࢉ • ΞΫηεΛྲྀ࢝͠ΊΔ
Riakͷӡ༻ͷૢ࡞ 2/2 • ϊʔυΛΫϥελ͔Β֎͢ • σʔλஔΛ࠶ܭࢉ • σʔλΛ࠶ஔ • riak-‐admin
down \
[email protected]
• riak-‐admin cluster plan • riak-‐admin cluster commit
͓͢͢Ίπʔϧ • AnsibleͷެࣜPlaybookϥΠϒϥϦʹ͋Γ·͢ • ߏྫ͋Γ·͢ • https://github.com/ansible/ansible • https://github.com/ansible/ansible-‐examples/
tree/master/riak
͜Μͳӡ༻͕Ί·͢ ৸Δ தʹෛՙ͕ ্͕Δ APIͰϊʔυ ࣗಈՃ தʹຊʹඞ ཁͳ·Ͱϊʔ υΛݮΒ͢ ى͖Δ
ʢே͕͖ͨͷͰʣ ࡢͷ࠷େෛՙ ΛΈͯɺϊʔυ ͱൺֱ
খ࢝͘͞ΊΔͳΒ ˡWebServers • m1.xlarge x3 • Elastic Block Storage 256GB
x3, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
;ͭ͏ʹ࢝ΊΔͳΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 256GB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
˓˓͕૿͖͑ͯͨΒ • ظతʢ1ʙ10िؒʣରࡦ • ϚγϯΛ૿͢ • ظతରࡦ • ϚγϯεϖοΫΛ্͛Δ
ෛՙ͕૿͖͑ͯͨΒ ˡWebServers • m3.2xlarge x5 • Elastic Block Storage 512GB
x5, >4000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
σʔλ͕૿͖͑ͯͨΒ ˡWebServers • m1.xlarge x5 • Elastic Block Storage 2TB
x5, >2000PIOPS • ηΩϡϦςΟάϧʔϓ • TCP 4369, 6000-7999, 8087, 8098, 8099
ෛՙ͕૿͖͑ͯͨΒ̎ ˡWebServers • hi1.4xlarge x5 • SSD • ηΩϡϦςΟάϧʔϓ •
TCP 4369, 6000-7999, 8087, 8098, 8099 File Archive
όοΫΞοϓઓུ • tarball ΛS3ʹ • LVM Snapshot • ͍Ζ͍Ζ͋Γ·͕͢… •
MDC Replication
Availability Zone Availability Zone ;ͨͭͷΫϥελΛϨϓϦέʔγϣϯͰಉظ͢Δ͜ͱ͕Ͱ͖Δ όοΫΞοϓෛՙࢄͳͲʹ
RIAK + EC2 USECASES (ࣄྫ)ɹলུ͞Ε·ͨ͠
·ͱΊ • Riak ͱ EC2 ͷΈ߹ΘͤͱͯΑ͍ • ؆୯ʹ͑Δˍ҆ఆ͍ͯ͠Δ • ׂͱଟ͘ͷϢʔβʔ͕EC2্ͰRiakΛ͍ͬͯΔ
એ ɾιϑτΣΞσβΠϯͱ͍͏ࡶࢽʹ࿈ࡌ͍ͯ͠·͢ ɾ11݄߸ʢ10/18ൃചͪΐ͏ͲRiak on EC2ͷࡉ͕ʣ
One more thing… • ͍Ζ͍Ζݴ͍·͕ͨ͠ɺࣗͰؤுΔΑΓ100ഒָ ʹAWS্Ͱεέʔϧ͢ΔDBΛ࡞Δํ๏͕͋Γ·͢ • ϙνϙν͢Δ͚ͩͰOK
Go to www.engineyard.com
࣭ʁ • Repo: https://github.com/basho/riak • Twitterެࣜ: @BashoJapan or #riakjp •
Riak ML:
[email protected]
• Me:
[email protected]