Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Serverless??そんなことよりDBだ!! / We should know Datab...
Search
Masashi Terui
November 03, 2018
Technology
3
3.6k
Serverless??そんなことよりDBだ!! / We should know Databases before Serverless
JAWS Festa 2018の資料です
https://jft2018.jaws-ug.jp/session/729/
Masashi Terui
November 03, 2018
Tweet
Share
More Decks by Masashi Terui
See All by Masashi Terui
What is Jeffy ?? - Why did we create a new framework
marcyterui
2
840
Serverlessの今とこれから / Everything will be Serverless
marcyterui
7
3.4k
見せてやろう…!! Serverlessの本当の力を…!! / Let’s show you the real power of serverless…!!
marcyterui
6
2.5k
re:Invent 2018でServerlessの世界は何が変わったか / What has changed in the world of Serverless at re:Invent 2018??
marcyterui
4
2.9k
実践サーバレス + α / Serverless Practices
marcyterui
2
590
Serverlessを極めるためにDynamoDBデータモデリングを極めよう / Let’s become the master of DynamoDB Data Modeling to become the master of Serverless
marcyterui
12
17k
マイクロサービスを成功させるためのサーバーレスアーキテクチャ設計とNoSQLデータモデリング / Serverless Architecting and NoSQL Data Modeling for Successful Microservices
marcyterui
13
6.7k
何故私達はサーバレスアプリケーションをイベントドリブンで作るべきなのか / Why should we make serverless application as event driven?
marcyterui
8
1.2k
The theory of data modeling to become the friend of DynamoDB
marcyterui
2
210
Other Decks in Technology
See All in Technology
Cloudflareで実現する AIエージェント ワークフロー基盤
kmd09
0
270
信頼されるためにやったこと、 やらなかったこと。/What we did to be trusted, What we did not do.
bitkey
PRO
0
2.1k
東京Ruby会議12 Ruby と Rust と私 / Tokyo RubyKaigi 12 Ruby, Rust and me
eagletmt
3
830
AWSマルチアカウント統制環境のすゝめ / 20250115 Mitsutoshi Matsuo
shift_evolve
0
100
ゼロからわかる!!AWSの構成図を書いてみようワークショップ 問題&解答解説 #デッカイギ #羽田デッカイギおつ
_mossann_t
0
1.5k
comilioとCloudflare、そして未来へと向けて
oliver_diary
6
430
カップ麺の待ち時間(3分)でわかるPartyRockアップデート
ryutakondo
0
130
いま現場PMのあなたが、 経営と向き合うPMになるために 必要なこと、腹をくくること
hiro93n
9
7k
深層学習と3Dキャプチャ・3Dモデル生成(土木学会応用力学委員会 応用数理・AIセミナー)
pfn
PRO
0
450
テストを書かないためのテスト/ Tests for not writing tests
sinsoku
1
170
Alignment and Autonomy in Cybozu - 300人の開発組織でアラインメントと自律性を両立させるアジャイルな組織運営 / RSGT2025
ama_ch
1
2.3k
Accessibility Inspectorを活用した アプリのアクセシビリティ向上方法
hinakko
0
170
Featured
See All Featured
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
98
18k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
GraphQLの誤解/rethinking-graphql
sonatard
68
10k
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.1k
Typedesign – Prime Four
hannesfritz
40
2.5k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
It's Worth the Effort
3n
183
28k
The World Runs on Bad Software
bkeepers
PRO
66
11k
Product Roadmaps are Hard
iamctodd
PRO
50
11k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
What's in a price? How to price your products and services
michaelherold
244
12k
Transcript
SERVERLESSʁʁ ͦΜͳ͜ͱΑΓDBͩʂʂ MASASHI TERUI @ JAWS FESTA OSAKA 2018
SERVERWORKS CO.,LTD. + FREELANCER • Serverless Oji-san • Serverless Framework
Plugin Developer • Serverlessconf Tokyo 2016,2017,2018 speaker • Remote worker (in Sapporo-shi, Hokkaido) • http://marcy.hatenablog.com/entry/2018/08/04/231241 MASASHI TERUI ARCHITECT / DEVELOPER
SERVERLESSͬͯ·͔͢ʁ DynamoDBͬͯ·͔͢ʁ
• ͖ • ී௨ • ݏ͍ • LambdaͱRDSͷ૬ੑ͕ѱ͍͔Βํͳ͖͘߹͍ͬͯΔ DynamoDB͖Ͱ͔͢ʁ
ํ͕ͳ͍͔ΒDynamoDB…… ຊRDBΛ͍͍͚ͨͲ
RDBʹ͑ͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ NoSQL͕͑ͳ͍Μ͡Όͳͯ͘
RDBͷ͜ͱͪΌΜͱ͔ͬͯ·͔͢ʁʁ ͦͦ
• ίωΫγϣϯϞσϧ • Ӭଓత • ίωΫγϣϯϓʔϦϯά • ηΩϡϦςΟ • ΈࠐΈͷऑ͍ೝূ
• VPCͷΞΫηείετʢENIੜॲཧʣ LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ ͱͯOLTPͰ͑ͳ͍(ඵʙेඵ)
DynamoDBͷ͜ͱ͔ͬͯ·͔͢ʁʁ ͡Ό͋
σʔλϕʔεͷ͜ͱ͔ͬͯ·͔͢ʁʁ ͱ͍͏͔
• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎͳͷʁ • ݁Ռ߹ੑͬͯେৎͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹͲ͏͢Εྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆
• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎͳͷʁ • ݁Ռ߹ੑͬͯେৎͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹͲ͏͢Εྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆ ͔Βͳ͍͔Βෆ҆ʹͳΔ
• AWS͕ఏڙ͢ΔNoSQLαʔϏε • KeyʹΑΔશҰகͱIndex୳ࡧ͕Մೳ • ϑϧϚωʔδυ • ແ੍ݶͷࣗಈ֦ுετϨʔδʢ͚ͬͨͩʣ • ΠϯελϯεͷཧෆཁɺWrite/Readͷੑೳ͚ͩΛࢦఆ͢Δ
• ͰɺΩϟύγςΟͷࣄલ༧͕ඞཁͳαʔϏεServerless͡Όͳ͍ͬͯCNCF͞Μ͕ݴͬɾɾɾ͋ɺ୭͔དྷͨΈ͍ͨ DynamoDBͷجຊ
DynamoDBͲΜͳ ”AWSαʔϏε” ͔ Ͱͳ͘
DynamoDBͲΜͳ ”σʔλϕʔε” ͔ ͬͯ·͔͢ʁ
• Consistent Hashing • Vector clocks with reconciliation during reads
• Sloppy Quorum and hinted handoff • Anti-entropy using Merkle trees • Gossip-based membership protocol and failure detection Amazon’s Dynamo ৄ͘͠Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰͳ͍ʣ https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
• ΩʔͷϋογϡͰࢄ → Partition Key • ͦͷࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key
• όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ • ݁Ռ߹ੑϕʔε • ॻ͖ࠐΈɾಡΈࠐΈͷQuorumͰ߹ੑ͕ܾ·Δ • ΩʔΛࢦఆͨ͠ಡΈࠐΈڧ߹ੑ͕બՄೳ ͑Δ͖ϙΠϯτ
Partition KeyͷϋογϡͰࢄ ͋ɺ͜ΕRDB(Hash Index)Ͱʢ͈́
• Partition Keyʹ࿈൪Λ͏ͷԿͷҙຯͳ͍͜ͱ͕͔Δ • ࿈൪ΧϯλʔΛ࡞ΔͱPartition͕ภΔ • Max + 1ͳΜͯҎͯͷ֎ •
ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ͏ • ͳ͚ΕUUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜͰྑ͍ • SortͰ͖ͳ͍ͷͰSnowflakeͷΑ͏ͳॱংੑཁΒͳ͍ • User IDCognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ͏ • ΞΫηε੍ޚʹ͑Δ ͜ͷલఏΛ౿·͑Δͱ
• 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ) • ͲΕ͔ҰͭͰա͢ΔͱPartitionׂ͕͞ΕΔ •
ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉʹׂΓৼΒΕΔ • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ • ݶఆతͳόʔετ • ༨Λճ͢ (Adaptive Capacity) Partition͝ͱͷΩϟύγςΟ
Partition KeyͰࢄɺSort Keyʁ
B+TREE INDEX B-treeͳͷ͔B+treeͳͷ͔ެද͞Ε͍ͯͳ͍͚Ͳ Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ ͋ɺ͜ΕRDBͰʢ͈́
TARGETING QUERY B+treeͷߏΛ͍ͬͯΕ Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘ ͜͏Εྑ͍͜ͱ͕͔Δ
Partition KeyͰࢄͨ͠தͰSort KeyͰྻ ͜ͷΠϝʔδΛ࣋ͭ͜ͱ͕ͱͯॏཁ
• Partition KeyͰࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ • ҰͭͷΠϯελϯεͰಉظతʹIndexߋ৽͞ΕΔ • ڧ߹ੑ͕બՄೳ • Primary Sort
KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕ ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ • LSI࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ Primary Sort Key, LSI
GSI࣮ςʔϒϧͷࣹӨ
• ू߹ʹ͓͚ΔࣹӨ • GSIͷPartition KeyΛݩʹࢄ ʢߋ৽ඇಉظͰద༻ʣ • ϦʔϑϊʔυʹଐੑͱPrimary Key ʢৄ͘͠ޙड़ʣ
• ιʔτࡁΈͷσʔλͷ෦ෳ ʢҟೝΊΔʣ ࣹӨ ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825 https://www.slideshare.net/yoku0825/mysql-57449062/62
• KEYS_ONLY: Primary KeyͷͷΈ • INCLUDE: Primary Keyͷ + બͨ͠ଐੑ
• ALL: શͯͷଐੑ • GSIʹؚΊͨଐੑPrimary KeyΛҾ͖͢ඞཁ͕ͳ͘ͳΔ • ಡΈࠐΈͷޮԽ • ߋ৽ίετͱͷτϨʔυΦϑ • MySQLPostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ
ࣹӨͱ͍͏͜ͱ 1:Nͷ͕ؔ͜ͷΑ͏ʹղܾͰ͖Δ
• ϝΠϯςʔϒϧͱผͷPartitionͰࢄ͞ΕΔ • ߋ৽ඇಉظͰө • ݁Ռ߹ੑͬͯେৎͳͷʁ • RDSͩͬͯRead ReplicaಡΜͩΒ݁Ռ߹ੑͰ͢Αʁʁ •
GSIͷΫΤϦ͍͍ͩͨRead Replicaʹ͛ΔΑ͏ͳΫΤϦ GSIʹର͢ΔΫΤϦ݁Ռ߹ੑ
͋ɺ͜ΕRDBͰʢԿճʁ
ΈΛΕ࣮ʹγϯϓϧ શ෦RDBͰطʹ͋ͬͨΈͰ͢Αʁʁ
☓ DynamoDBบ͕͋Δ ◦ RDBͷบ͕͋Γ͗͢Δ →γϯϓϧ͗͢ΔDynamoDBͱͷΪϟοϓ͕ڧͯͦ͘͏ײ͡Δ͚ͩ
RDBʹચ͞ΕͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ ͱ͍͏͔ͦͷRDBͷ͜ͱ͢ΒͪΌΜͱ͔ͬͯ͋͛ͯ·͔ͨ͠ʁ
RDBͷ͜ͱ͔ͬͯ·͔ͨ͠ʁ ͱ͍͏͔ɺͦͷେ͖ͳ
DynamoDBͷσʔλϞσϦϯά ͰͦΕΛ౿·͑ͯ
ͦΕσʔλͷஔΛܾΊΔ͜ͱ ҟೝΊΔʂʂ
σʔλϞσϦϯάͷߟ͑ํͷϙΠϯτ ஔΛܾΊΔͨΊͷ
• εΩʔϚϨεͷҙຯΛߟ͑Δ • List, Map͕࣋ͯΔ͜ͱʁ • ޙ͔ΒଐੑΛγʔϜϨεʹՃͰ͖Δ͜ͱʁ • A. ͳΜͰಥͬࠐΊΔ͜ͱ
• Partition KeyͰࢄ • ςʔϒϧΛ͚ͳͯ͘ෛՙࢄ͢Δ • ۃ͢Δͱ͚Δҙຯ͕ͳ͍ɺ͚ͳ͍ํ͕ΩϟύγςΟ͕ཧ͍͢͠ • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰɺඞཁͳςʔϒϧ 1 ͭͷΈͰ͢” • GSIͷݸ੍ݶ(5ݸ)ͳͲ͋ΔͷͰۀυϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ ςʔϒϧͷ͚ํ
• RDB • ·ͣεΩʔϚΛߟ͑Δ • ਖ਼نԽ • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ • DynamoDB
• εΩʔϚϨε • ඇਖ਼نԽ • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ • ΞϓϦέʔγϣϯͱσʔλϞσϧಉ࣌ʹઃܭ͢Δ RDBͱΞϓϩʔν͕શ͘ҟͳΔ
ACIDτϥϯβΫγϣϯ͕ແ͍ͷͲ͏͢Δͷʁ ͱ͍͑
• ݪࢠੑ (Atomicity) • Ұ؏(߹)ੑ (Consistency) • ಠཱੑ (Isolation) •
Ӭଓੑ (Durability) ͦͦACIDͬͯͳΜ͚ͩͬʁ ͜ΕΒΛ୲อ͢Δ͜ͱ͕తͰ͋ͬͯ τϥϯβΫγϣϯγεςϜͰ͋Δඞཁແ͍
• RDB • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷجຊతʹ૿͑Δ • ཧతʹ͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ߹ੑΛऔΔΈ͕͋Δ • ϦϨʔγϣϯγοϓ • ֎෦Ωʔ੍
• ACIDτϥϯβΫγϣϯ • DynamoDB • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ • 1ΞΠςϜͷߋ৽ΞτϛοΫ • ͖݅ॻ͖ࠐΈΛָͬͨ؍తϩοΫ ACIDͷ୲อ
• PutItem • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ • Get -> ॻ͖͑ -> Putͩͱ߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ
• UpdateItem • ΞΠςϜΛ෦ߋ৽ɺແ͔ͬͨΒ࡞Δ • List, Mapͷ෦ՃɺআͳͲՄೳ • ଐੑͷΛར༻ͨ͠ߋ৽Մೳʢ͋Δଐੑʹ+1͢Δ) • ଐੑͷΛར༻͖ͨ݅͠ॻ͖ࠐΈ DynamoDBͷߋ৽ૢ࡞
ඇਖ਼نԽ͢ΔͱಡΈࠐΈ͕ͭΒ͍ ϑϧεΩϟϯ͢Δ͔͠ͳ͍ શ෦GSIషΔͷʁ
CQRSͰ͢Α ͦ͜Ͱ
• ॻ͖ࠐΉσʔλͱಡΈࠐΉσʔλಉ͡Ͱ͋Δඞཁແ͍ • ݁Ռ߹ੑΛड͚ೖΕͯඇಉظͰಡΈࠐΈσʔλΛ࡞ΕΔ • CommandͷྃΛͬͯQuery༻σʔλΛ࡞ΕΔ • DynamoDB Streams͕Ϛον͢Δ •
CQRSΛૉʹ࣮͢Δͱ·ͣKinesis Streams͚ͩͲ • Materialized View CQRS (Command Query Responsibility Segregation)
ARCHITECTURE PATTERN AppSync͕͜Ε͔ΒॏཁͳҐஔΛΊΔͣ
DATA ATOMICITY MicroserviesతͳΠϝʔδͰ͜ͷݪ୯Ґ͕ॏཁ σʔλετΞɾςʔϒϧͷ͚ํͰͳ͍ ލ͕͍ͤͨͳΒMaterialized ViewΛ࡞Δ User Profile User Status
Name Email Read/Write Search Read/Write
͜ΜͳΠϝʔδ 1ͭͷGSIͰ༷ʑͳݕࡧ͕Ͱ͖Δ
• ෳͷ࣮ମʹލΔτϥϯβΫγϣϯ • DynamoDB୯ମͩͱ͜͜Ͳ͏ͯͭ͠Β͍ • τϥϯβΫγϣϯϨϕϧͷ࠷ߴSelializable • ͭ·ΓྻԽ • Kinesis
StreamsͰྻԽ • ίέΔ͔͠Εͳ͍ͷͰႈʹ࣮͢Δ • ݅ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷႈ ͱ͍͑ݶք͋Δ
• Kinesis StreamsͰಉ࣌ଓΛίϯτϩʔϧ͢Ε ผʹRDSͬͯԿͷͳ͍Θ͚Ͱ • ಉ࣌ଓ → ShardͱಉͰݻఆ • LambdaجຊWarm
Start • ͭ·ΓɺENIͷԆͱ΄΅ແԑ • σϓϩΠShardม͑ͨλΠϛϯάͰى͖͏Δ͚ͲͦͦඇಉظͳͷͰʢ͈́ ͱ͍͏͔
ADVANCED TIPS ͕࣌ؒ͋Ε͍͔ͭ͘͝հ
• DynamoDBͷIndexSparse • Indexଐੑ͕ଘࡏ͠ͳ͍߹ɺIndexʹͦͷσʔλؚ·ͳ͍ • ಛఆͷΞΠςϜͷΈʹଘࡏ͢ΔଐੑΛࢦఆ͢Δ͜ͱͰ Index༰ྔ͕େ͖͘ѹॖͰ͖Δ Sparse Index
SortཁΒͳ͍ GSIʹ҉ͰPrimary Keyؚ͕·ΕΔ
• Α͋͘Δ֊ߏΛ࣋ͭσʔλ • ৫ߏ • ωετͨ͠ΧςΰϥΠζ • ͳͲͳͲ • ݕࡧΛߟ͑ͳ͚Ε֊ߏΛ·Δͬͱ1ΞΠςϜʹԡ͠ࠐΉ
• Sort KeyͰ֊Λදݱ͢Δ Hierarchical data
Sort KeyͰલํҰகݕࡧ ͋ɺ͜ΕSQLΞϯνύλʔϯͰʢ͈́ ઃܭ͕ࡶͳͷ͝Ѫᇷ
• Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ • ଐੑͷΛར༻͖ͨ݅͠ॻ͖ࠐΈ • ConditionExpression=“Version = 1” • UpdateExpression=“ADD
Version 1” • Version = 1Ͱͳ͚ΕΤϥʔ͕ฦΔ • Version = 1ͳΒ2ʹߋ৽͞ΕΔ όʔδϣχϯάͱָ؍తϩοΫ
• తʹݴͬͯόΠϒϧ • ͜ΕΛಡΜͰશͯཧղͰ͖ͨΒҰਓલʂʂ • https://docs.aws.amazon.com/ja_jp/amazondynamodb/latest/ developerguide/best-practices.html DynamoDB Best Practices
“DynamoDB ދͷר” [ݕࡧ] http://marcy.hatenablog.com/entry/2018/07/31/213705 ҰԠະͳͷͰ·ͩॻ͔͘
“LET’S BECOME THE FRIEND OF RDB” THANKS!! “LET’S BECOME THE
FRIEND OF DYNAMODB” “LET’S BECOME THE FRIEND OF SERVERLESS”