Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Event Sourcing+CQRS を活用するスケーラブルアプリケーション開発 / event-sourcing-cqrs-v2
yuuki takezawa
November 02, 2018
Technology
7
10k
Event Sourcing+CQRS を活用するスケーラブルアプリケーション開発 / event-sourcing-cqrs-v2
AWS Dev Day 2018
マイクロサービスアーキテクチャの裏側の巻
yuuki takezawa
November 02, 2018
Tweet
Share
More Decks by yuuki takezawa
See All by yuuki takezawa
ytake
4
1.1k
ytake
0
5.4k
ytake
3
6.9k
ytake
2
2.6k
ytake
3
270
ytake
1
1.7k
ytake
4
1.8k
ytake
1
3.4k
ytake
0
280
Other Decks in Technology
See All in Technology
mamix1116
3
390
manuelmeyer
0
120
conciergeu
0
120
soracom
0
500
myajiri
0
300
1stship
0
230
adhorn
0
340
ryusa
1
190
yoshiori
1
1.1k
cfisch3r
1
310
oliva
7
950
chaspy
6
1.2k
Featured
See All Featured
imathis
479
150k
shlominoach
176
7.5k
deanohume
295
28k
shpigford
165
19k
sachag
267
17k
paulrobertlloyd
71
3.6k
morganepeng
93
14k
sferik
610
54k
cromwellryan
103
6.1k
addyosmani
311
21k
jcasabona
8
550
swwweet
206
6.9k
Transcript
Event Sourcing+CQRSΛ׆༻͢Δ εέʔϥϒϧΞϓϦέʔγϣϯ։ൃ yuuki takezawa AWS Dev Day Tokyo 2018
Profile • ᖒ ༗و / ytake • גࣜձࣾΞΠελΠϧ CTO •
PHP, Hack, Go, Scala • Apache Hadoop, Apache Spark, Apache Kafka • twitter https://twitter.com/ex_takezawa • facebook https://www.facebook.com/yuuki.takezawa • github https://github.com/ytake
ΞϓϦέʔγϣϯͷ
ΞϓϦέʔγϣϯྫ • ϩάΠϯϢʔβʔίϝϯτ͕ߘͰ͖Δ • ະϩάΠϯϢʔβʔӾཡͷΈ͕Ͱ͖Δ • ͓ؾʹೖΓͨ͠ϢʔβʔͷίϝϯτΛӾཡͰ͖Δ • ࣌ܥྻͰίϝϯτΛӾཡͰ͖Δ •
ίϝϯτͷআ͕Ͱ͖Δ • ίϝϯτͷฤूͰ͖ͳ͍ ͳͲ
ΞϓϦέʔγϣϯ࣮ • ೝূػೳͷ࣮ • ίϝϯτߘɺฤूɺӾཡػೳ࣮ • ݕࡧػೳͷ࣮
γϯϓϧͳΞϓϦέʔγϣϯߏ
ෳࡶԽ͢ΔΞϓϦέʔγϣϯ • ػೳՃ • ίϝϯτใΛଞͷػೳͰར༻͍ͨ͠ • ݅ʹΑΔҰཡใͷιʔτมߋ ϢʔβʔใଐੑʹΑΔιʔτॱมߋ
ෳࡶԽ͢ΔΞϓϦέʔγϣϯ
ංେԽ͢ΔΞϓϦέʔγϣϯ • ऩӹԽͷͨΊͷػೳ • རศੑͷͨΊͷػೳ • طଘػೳͷ֦ு
ංେԽ͢ΔΞϓϦέʔγϣϯ
ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά • ݁߹ͨ͠ػೳΛίϯϙʔωϯτఏڙ • APIʹΑΔσʔλૢ࡞ͷநԽ • σʔλϕʔεύϑΥʔϚϯεվળ
ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά
ڊେͳΞϓϦέʔγϣϯ
ਂࠁͳ • ༷มߋʹ͏ίϯϙʔωϯτमਖ਼ͱϦϦʔεෳࡶԽ ϥΠϒϥϦͷόʔδϣϯ͋͛ͳ͖Όɾɾʂ ͋ͷػೳΛऔΓࠐΉʹ͋Εͱ͜Εͱɾɾɾ • ΞϓϦέʔγϣϯؒ࿈ܞͷෳࡶԽ APIमਖ਼ʹ͏Өڹൣғௐࠪ • σʔλಉظͷෳࡶԽ
ϦΞϧλΠϜͰσʔλ͕ཉ͍͠ ΠϯσοΫεߋ৽͕ఆظ࣮ߦ
͍ͯ͘͠ࣄۀʹ߹Θͤͯɺ ΞϓϦέʔγϣϯΛ Ͳ͏εέʔϧ͍͔ͤͯ͘͞
มΘΓΏ͘ڥ • σʔλΛऔಘ͢Δʹʁ APIίʔϧʹΑΔ֬ೝʁ ΩϟογϡԿʁ • ͦΕͧΕͷαʔϏεͰͦΕͧΕͷݴޠ ͋ΔαʔϏεPHPɺ͋ΔαʔϏεͰGo ScalaͳͲ
σʔλॲཧ Ϣʔβʔใ͍߹Θͤ ใΛߋ৽ͨ͠߹Ͱ ґଘαʔϏε͕ͳ͍ͨΊͳ͠
σʔλॲཧͷෳࡶԽ Ϣʔβʔใมߋ Ϣʔβʔใআ Ϣʔβʔใ͕ݟ͔ͭΒͳ͍ͨΊ σʔλॲཧ͝ͱʹ"1*ΞΫηε ͋ͦ͜ͱͦ͜Ͱใ͕ҧ͏ʂ Ͳͷσʔλ͕ຊʁ શͯͷΞϓϦέʔγϣϯ͔Β Ωϟογϡͳ͠ͰΞΫηε ৗʹߴෛՙঢ়ଶʹ
Event / Application Side
Event? • Observerύλʔϯ • ʙͨ࣌͠ʹɺʙ͢Δ Λදݱ͢Δ • ༷ʑͳΞϓϦέʔγϣϯͰར༻͞ΕΔ
Event / Application • γεςϜཁ݅ɾػೳͱͯ͠औΓೖΕΔέʔεଟ͍ • Request͕དྷͨΒɺRouteΛىಈͤ͞Δ • ʙͷॲཧ͕ྃͨ͠Βɺϩάʹॻ͖ग़͢
public function __invoke( FulltextRequest $request ): RedirectResponse { // ొॲཧޙʹevent
try { $entity = new Entity(1, ‘testing’); $this->dispatcher->dispatch( new SinkConnect($this->usecase->run($entity)) ); } catch(\RuntimeException $e) { // Կ͔errorॲཧ } return $this->redirector->route('fulltext.index'); }
ҰͭͷΞϓϦέʔγϣϯͷ߹ • ΞϓϦέʔγϣϯͷ༷ʑͳग़དྷࣄΛ ObserverΛ࣮ͬͯ͢Δ • ΞϓϦέʔγϣϯͷΫϥε͕ංେԽͨ࣌͠ খ͞ͳΫϥεʹׂ͢Δ߹ɺ ϦϑΝΫλϦϯάʹར༻͢Δ͜ͱ • ୯ҰͷΞϓϦέʔγϣϯͰ࣮͢Δ߹ʹ༗༻
ߟྀ͢Δ͜ͱ • ΞϓϦέʔγϣϯؒͰ࿈ܞ͢Δ߹Ͳ͏͖͔͢ • ʙͨ࣌͠ʹɺʙ͢Δ ଞͷΞϓϦέʔγϣϯɾݴޠͰ࣮ߦ͞ΕΔ͔ Γग़͢ඞཁ͕͋Δ͔Ͳ͏͔
Event Sourcing
Event Sourcing • ΠϕϯτΛهͯ͠࠶ݱ͢Δύλʔϯ • ΠϕϯτΛΠϕϯτετΞʹอଘ • ه͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩ • ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτෆม
• ൃੜͨ͠ࣄ࣮Λ͑Δ
Event SourcingΛಋೖ͖͢Ͱͳ͍߹ • ΠϕϯτΛهͯ͠࠶ݱ͢Δඞཁ͕ͳ͍߹ • ه͞ΕͨΠϕϯτ͔Β ΦϒδΣΫτΛ෮ݩ͢Δ͜ͱ͕ͳ͍߹ • খنͳΞϓϦέʔγϣϯɺ γϯϓϧͳCRUD͔͠ͳ͍ΞϓϦέʔγϣϯ
Event Sourcing • Ͳ͏࣮ͬͯ͢Δ͔ʁ • RDBMSͱΈ߹ΘͤͯશͯͷΠϕϯτใΛอʁ • େنͳσʔλʹରԠͰ͖Δ͔ʁ • ר͖͠ͷॲཧͲ͏͢Δ͔ʁ
Asynchronous / Synchronous Event
ಉظΠϕϯτ • ΠϕϯτಉظॲཧͰ͋Δ͖͔Ͳ͏͔ Πϕϯτൃੜ࣌ʹଈ࠲ʹଞΞϓϦέʔγϣϯʹө͢Δ ඞཁ͕͋Δ͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋ɺ αϒεΫϥΠόʹΑΔσʔλӬଓԽΛߦ͏
ඇಉظΠϕϯτ • ΠϕϯτඇಉظʹͰ͖Δ͔Ͳ͏͔ Πϕϯτ͕ൃੜͨ͜͠ͱΛ୲อ͠ɺ өԆͤͯ͞Α͍͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋·Ͱ σʔλӬଓԽԆͰ࣮ߦ͞ΕΔ
CQRS
ΞϓϦέʔγϣϯͷͰൃੜ͢Δ • σʔλͱύϑΥʔϚϯε • αʔϏεͱڞʹ૿͑ΔෳࡶͳΫΤϦͱ༰ྔ • σʔλͷਖ਼͠͞(ਖ਼نԽ)͔ޮԽ͔(ඇਖ਼نԽ) • RDBMSͷෆಘҙ +
NoSQLʹΑΔΧόʔ ಉظํ๏ͷෳࡶԽ
σʔλϕʔεઃܭͷΈ • ਖ਼نԽͱநग़͍ͨ͠σʔλͷΪϟοϓ • நग़͍ͨ͠σʔλʹ߹ΘͤͨઃܭΛ͢Δ͜ͱʹΑΔ ੍͕͔͚ΒΕͳ͍δϨϯϚ • ࣮ίʔυͰ୲อ͢ΔͨΊʹෳࡶԽ
ཁٻ༷ʹରԠ͢ΔͨΊʹ σʔλϕʔεͰશͯରԠ͢Δͷਖ਼͍͔͠ ਖ਼نԽɺඇਖ਼نԽ ಉ࣠͡Ͱߟ͑Δ͖͔
CQRS "A few myths about CQRS". Ouarzy's Blog. http://www.ouarzy.com/2016/10/02/a-few-myths-about-cqrs/
ࢀর
CQRSͱԿ͔ • ॻ͖ࠐΈ(෭࡞༻͋Γ)ͱ ಡΈࠐΈ(෭࡞༻ͳ͠)Λ͢Δ • ෭࡞༻͋ΓσʔλͳͲʹԿΒ͔ͷมߋΛ༩͑Δͷ • ॻ͖ࠐΈ => Command
• ಡΈࠐΈ => Query
CQRS • ͋ΒΏΔϝιουɺΞΫγϣϯΛ࣮ߦ͢ΔίϚϯυ ͔ɺݺͼग़͠ݩʹσʔλΛฦ͢ΫΤϦ͔ͷ͍ͣΕ͔Ͱ ͋ͬͯɺ྆ํΛߦͬͯͳΒͳ͍ɻ͜Εɺ࣭Λ͢Δ ͜ͱͰճΛมԽͤͯ͞ͳΒͳ͍ͱ͍͏͜ͱͩɻ "Greg YoungྲྀCQRS -
Mark Nijhof". Digital Romanticism. http://d.hatena.ne.jp/digitalsoul/20100712/1278886009 ࢀর
Command • ߹ੑ ੍Λ͔͚ͯਖ਼͘͠ɺݎ͘ • ॻ͖ࠐΈɺߋ৽ɺআॲཧ ಡΈࠐΈʹൺͯޮੑɺ εέʔϥϏϦςΟΛҙࣝ͢Δ͜ͱ͋·Γͳ͍
Query • ߹ੑΑΓݕࡧޮ • ΄ͱΜͲͷΞϓϦέʔγϣϯʹ͓͍ͯɺ ॻ͖ࠐΈΑΓݕࡧͷൺॏ͕େ͖͍͜ͱ͕΄ͱΜͲ • εέʔϧ͕༰қͰ͋Δ͜ͱɺ ߴͳύϑΥʔϚϯεͰ͋Δ͜ͱ
Command, Query࣌ͷσʔλϕʔε
Command
Command
Query
CQRS Application Application
ཧ۶Θ͔Δ͚Ͳɺ Ͳ͏࣮͢Ε͍͍ͷ͔ʁ ಉظͲ͏͢Δͷʁ
Event Sourcing + CQRS
Event Sourcing / Command ྫ
Event Sourcing / Command ྫ ΞϓϦέʔγϣϯ͔Β Πϕϯτૹ৴
Event Sourcing / Command ྫ ϝοηʔδΛड͚औΓɺ ͷՃͳͲ
Event Sourcing / Command ྫ RDBMSɺNoSQLͳͲ ༻్ʹ߹Θͤͯॻ͖ࠐΈ
Event Sourcing / Command ྫ ଞͷΞϓϦέʔγϣϯ͔Β σʔλϕʔεͷΈΛࢀর Ճʹ͍ͭͯͷࣝΒͳ͍
Cache System σʔλϕʔεߋ৽ $PNNBOE σʔλϕʔεߋ৽ྃΠϕϯτ &WFOU4PVSDJOH σʔλϕʔεߋ৽ྃݕ &WFOU-JTUFOFS ΩϟογϡআͳͲ
Event Sourcing • CommandɺQueryɺEvent Sourcing ผΞϓϦέʔγϣϯɾݴޠͰ࣮Ͱ͖Δ͘Β͍ͷ γϯϓϧ͞ • EventอଘͱΞϓϦέʔγϣϯʹରԠͨ͠ QueryΛͲ͏ͬͯ
ޮతʹ࣮͢Δ͔
ΞϓϦέʔγϣϯׂྫ
ΞϓϦέʔγϣϯׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ
ΞϓϦέʔγϣϯׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH
etc
ΞϓϦέʔγϣϯׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH
etc ࠂ৴ܥAPI ࠂ৴։ൃνʔϜ
ΞϓϦέʔγϣϯׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH
etc ࠂ৴ܥAPI ࠂ৴։ൃνʔϜ ϢʔβʔͷӾཡใʹج͍ͮͨ ใ͕ཉ͍͠Ͱ͢ ϦΞϧλΠϜͰ
Event Sourcing / Message Broker
Event SourcingͱMiddlewareબఆ • PubSubͰشൃੑͷͳ͍ͷΛબ͢Δ • ϝοηʔδอ͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳͷ Λ࠾༻͢Δͷ͕·͍͠ • εέʔϥϒϧͰ͋Δ͜ͱ
Event Sourcing / RDBMS • PubSubͰشൃੑͷͳ͍ͷΛબ͢Δ ػೳͳ͍ͷͰόονॲཧͳͲͰ ఆظతʹσʔλՃ͕͋Δ͔Ͳ͏͔Λݕࡧ͢Δ • ϝοηʔδอ͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕ՄೳͳͷΛ࠾༻͢Δͷ
͕·͍͠ ಘҙ • εέʔϥϒϧͰ͋Δ͜ͱ Slaveɺmulti masterͳͲͰରԠͰ͖Δ
Event Sourcing / Redis • PubSubͰشൃੑͷͳ͍ͷΛબ͢Δ PUBLISH SUBSCRIBEͳͲ͕͋ΔͷͰ࣮ݱͰ͖Δ͕ɺ شൃੑʹ͍ͭͯઃఆͰؤுΔ •
ϝοηʔδอ͕ՄೳͰɺ ࠶ݱ͢Δ͜ͱ͕ՄೳͳͷΛ࠾༻͢Δͷ͕·͍͠ τϥϯβΫγϣϯ͋Δ • εέʔϥϒϧͰ͋Δ͜ͱ ༰қʹͰ͖Δ
None
None
Message Broker
None
Apache Kafka • ZookeeperΛར༻ͨ͠ΫϥελϦϯάʹΑΔߴՄ༻ੑ • ϝοηʔδͷӬଓԽɺϨϓϦέʔγϣϯɺ࠶औಘՄ • ϏοάσʔλରԠ • ετϦʔϜରԠͷϝοηʔδϯάϛυϧΣΞ
• Kafka ConnectʹΑΔपลγεςϜͱͷߴ͍ੑ
Event Sourcing • ΠϕϯτΛΠϕϯτετΞʹอଘ Kafkaࣗମʹอଘ͢Δ͜ͱ͕Ͱ͖ɺ σʔλϕʔεʹసૹՄೳ • ه͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩͰ͖Δ ॲཧ͞ΕͨΠϕϯτ࠶ൃߦՄೳ •
ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτෆม ൃߦޙͷΠϕϯτΛॻ͖͑Δ͜ͱͰ͖ͳ͍ • ൃੜͨ͠ࣄ࣮Λ͑Δ τϥϯβΫγϣϯ͋Γ
Apache Kafka֓ཁ • Partition ෛՙࢄ༻్ʹར༻ ෳͷConsumer͕ͦΕͧΕͷPartitionΛࢀর͠ɺ ͦΕͧΕ͕ॲཧΛߦ͏ ॲཧϑϩʔͷσβΠϯʹΑͬͯଟ༷ͳར༻ํ๏ • librdkafka͕ඇৗʹߴ
Example Partition
Consumer
Kafka Connect • Kafka Connectͱɺ पลγεςϜ͔ΒͷσʔλΛऔΓࠐΈ(Source)ɺ σʔλૹ৴(Sink)ͷೋछྨΛαϙʔτ͢Δػೳ • Amazon SQSMongoDBͷσʔλΛKafkaͰऔࠐΉɺ
ϝοηʔδΛͦͷ··ElasticsearchRDBMSʹ֨ೲɺ ͕ߦ͑Δ • Connectࣗ༝ʹ֦ுͯ͠ಠࣗConnectΛ࣮Մೳ
Kafka Connect • γϯϓϧͳέʔεͰ͋Εɺ ಛʹ࣮͢Δ͜ͱͳ͘QueryΛαϙʔτ • ෳࡶͳσʔλΛ࡞͢ΔͷͰ͋Ε Kafka StreamsɺSpark StreamingͰσʔλՃ
None
Amazon Kinesis • εέʔϥϒϧ • ϝοηʔδ 7ؒอ(࠷େ) • ετϦʔϜରԠͷKinesis Stream
• Kinesis FirehoseΛͬͨσʔλసૹ Amazon S3ɺAmazon Redshiftɺ Amazon Elasticsearch Service etc
Event Sourcing+ CQRS / Kafka • Command QueryΛ୲͢ΔWebΞϓϦέʔγϣϯΛ γϯϓϧʹ •
Event SourcingʹΑͬͯɺΞϓϦέʔγϣϯʹՁΛ Ճ͢Δ͜ͱ͕Ͱ͖Δ(ੳͳͲ) • Command QueryͷಉظΛαϙʔτ͠ɺ ΞϓϦέʔγϣϯʹ߹Θͤͯෛՙࢄ͕༰қ
ఆظߋ৽ͤ͞Δόονॲཧͷߟ͑ํมΘΔ
࣮ྫ
ྫ
ྫ ༷ʑͳΞϓϦέʔγϣϯ͔Β σʔλૹ৴
ྫ "QBDIF,BGLB͕ ϝοηʔδड৴
ྫ ,BGLB 4QBSL4USFBNJOH ΞϓϦέʔγϣϯ͔Βૹ৴͞Εͨσʔλͱɺ 3%#.4ʹ֨ೲ͞ΕͨσʔλΛ݁߹͠ɺ ूܭɾूΛߦ͏
ྫ ूܭɾू͞ΕͨσʔλΛɺ ಡΈࠐΈͰར༻͢ΔΞϓϦέʔγϣϯʹ ߹Θͤͯอ $BTTBOESBͱ4QBSL4USFBNJOHͷΈͰ ೖग़ྗΛߦ͏έʔε
ྫ ूܭɾू͞Εͨσʔλͷ͏ͪ ༷ʑͳՕॴͰར༻͞ΕΔͷɺ)%'4 ࠶ܭࢉɺোൃੜ࣌ʹ෮چͤ͞ΔͳͲ
ྫ ΞϓϦέʔγϣϯଆ͔Β $BTTBOESBͷΈʹ͍߹ΘͤΛߦ͏
ྫ ͦΕͧΕͷΞϓϦέʔγϣϯͷ $POTVNFSͰͦΕͧΕͷॲཧ
ྫ 3BCCJU.2ϝοηʔδૹ৴ ଞαʔϏε͕ར༻
ྫ ֤αʔϏεͷίϯγϡʔϚʔ͕ ଞͷαϒγεςϜͰར༻͢ΔσʔλΛੜ
·ͱΊ • େنΞϓϦέʔγϣϯʹͳΔ͜ͱ͕༧ଌͰ͖ͨΒ • ϚΠΫϩαʔϏεΞʔΩςΫνϟΛࢧ͑Δ • ϛυϧΣΞͷࣝඞཁʹ • ࢄॲཧෳࡶͳσʔλॲཧͷ ϦϑΝΫλϦϯάʹ׆͖Δύλʔϯ