AWS Dev Day 2018 マイクロサービスアーキテクチャの裏側の巻
Event Sourcing+CQRSΛ׆༻͢ΔεέʔϥϒϧΞϓϦέʔγϣϯ։ൃyuuki takezawaAWS Dev Day Tokyo 2018
View Slide
Profile• ᖒ ༗و / ytake• גࣜձࣾΞΠελΠϧ CTO• PHP, Hack, Go, Scala• Apache Hadoop, Apache Spark, Apache Kafka • twitter https://twitter.com/ex_takezawa• facebook https://www.facebook.com/yuuki.takezawa• github https://github.com/ytake
ΞϓϦέʔγϣϯͷ
ΞϓϦέʔγϣϯྫ• ϩάΠϯϢʔβʔίϝϯτ͕ߘͰ͖Δ• ະϩάΠϯϢʔβʔӾཡͷΈ͕Ͱ͖Δ• ͓ؾʹೖΓͨ͠ϢʔβʔͷίϝϯτΛӾཡͰ͖Δ• ࣌ܥྻͰίϝϯτΛӾཡͰ͖Δ• ίϝϯτͷআ͕Ͱ͖Δ• ίϝϯτͷฤूͰ͖ͳ͍ ͳͲ
ΞϓϦέʔγϣϯ࣮• ೝূػೳͷ࣮ • ίϝϯτߘɺฤूɺӾཡػೳ࣮ • ݕࡧػೳͷ࣮
γϯϓϧͳΞϓϦέʔγϣϯߏ
ෳࡶԽ͢ΔΞϓϦέʔγϣϯ• ػೳՃ • ίϝϯτใΛଞͷػೳͰར༻͍ͨ͠ • ݅ʹΑΔҰཡใͷιʔτมߋ ϢʔβʔใଐੑʹΑΔιʔτॱมߋ
ෳࡶԽ͢ΔΞϓϦέʔγϣϯ
ංେԽ͢ΔΞϓϦέʔγϣϯ• ऩӹԽͷͨΊͷػೳ • རศੑͷͨΊͷػೳ• طଘػೳͷ֦ு
ංେԽ͢ΔΞϓϦέʔγϣϯ
ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά• ݁߹ͨ͠ػೳΛίϯϙʔωϯτఏڙ • APIʹΑΔσʔλૢ࡞ͷநԽ • σʔλϕʔεύϑΥʔϚϯεվળ
ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά
ڊେͳΞϓϦέʔγϣϯ
ਂࠁͳ• ༷มߋʹ͏ίϯϙʔωϯτमਖ਼ͱϦϦʔεෳࡶԽ ϥΠϒϥϦͷόʔδϣϯ͋͛ͳ͖Όɾɾʂ ͋ͷػೳΛऔΓࠐΉʹ͋Εͱ͜Εͱɾɾɾ• ΞϓϦέʔγϣϯؒ࿈ܞͷෳࡶԽ APIमਖ਼ʹ͏Өڹൣғௐࠪ• σʔλಉظͷෳࡶԽ ϦΞϧλΠϜͰσʔλ͕ཉ͍͠ ΠϯσοΫεߋ৽͕ఆظ࣮ߦ
͍ͯ͘͠ࣄۀʹ߹ΘͤͯɺΞϓϦέʔγϣϯΛͲ͏εέʔϧ͍͔ͤͯ͘͞
มΘΓΏ͘ڥ• σʔλΛऔಘ͢Δʹʁ APIίʔϧʹΑΔ֬ೝʁ ΩϟογϡԿʁ • ͦΕͧΕͷαʔϏεͰͦΕͧΕͷݴޠ ͋ΔαʔϏεPHPɺ͋ΔαʔϏεͰGo ScalaͳͲ
σʔλॲཧϢʔβʔใ͍߹ΘͤใΛߋ৽ͨ͠߹Ͱ ґଘαʔϏε͕ͳ͍ͨΊͳ͠
σʔλॲཧͷෳࡶԽϢʔβʔใมߋ ϢʔβʔใআϢʔβʔใ͕ݟ͔ͭΒͳ͍ͨΊ σʔλॲཧ͝ͱʹ"1*ΞΫηε͋ͦ͜ͱͦ͜Ͱใ͕ҧ͏ʂ Ͳͷσʔλ͕ຊʁશͯͷΞϓϦέʔγϣϯ͔Β Ωϟογϡͳ͠ͰΞΫηε ৗʹߴෛՙঢ়ଶʹ
Event / Application Side
Event?• Observerύλʔϯ• ʙͨ࣌͠ʹɺʙ͢Δ Λදݱ͢Δ• ༷ʑͳΞϓϦέʔγϣϯͰར༻͞ΕΔ
Event / Application• γεςϜཁ݅ɾػೳͱͯ͠औΓೖΕΔέʔεଟ͍• Request͕དྷͨΒɺRouteΛىಈͤ͞Δ• ʙͷॲཧ͕ྃͨ͠Βɺϩάʹॻ͖ग़͢
public function __invoke(FulltextRequest $request): RedirectResponse {// ొॲཧޙʹeventtry {$entity = new Entity(1, ‘testing’);$this->dispatcher->dispatch(new SinkConnect($this->usecase->run($entity)));} catch(\RuntimeException $e) {// Կ͔errorॲཧ}return $this->redirector->route('fulltext.index');}
ҰͭͷΞϓϦέʔγϣϯͷ߹• ΞϓϦέʔγϣϯͷ༷ʑͳग़དྷࣄΛ ObserverΛ࣮ͬͯ͢Δ• ΞϓϦέʔγϣϯͷΫϥε͕ංେԽͨ࣌͠ খ͞ͳΫϥεʹׂ͢Δ߹ɺ ϦϑΝΫλϦϯάʹར༻͢Δ͜ͱ • ୯ҰͷΞϓϦέʔγϣϯͰ࣮͢Δ߹ʹ༗༻
ߟྀ͢Δ͜ͱ• ΞϓϦέʔγϣϯؒͰ࿈ܞ͢Δ߹Ͳ͏͖͔͢• ʙͨ࣌͠ʹɺʙ͢Δ ଞͷΞϓϦέʔγϣϯɾݴޠͰ࣮ߦ͞ΕΔ͔ Γग़͢ඞཁ͕͋Δ͔Ͳ͏͔
Event Sourcing
Event Sourcing• ΠϕϯτΛهͯ͠࠶ݱ͢Δύλʔϯ• ΠϕϯτΛΠϕϯτετΞʹอଘ• ه͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩ• ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτෆม• ൃੜͨ͠ࣄ࣮Λ͑Δ
Event SourcingΛಋೖ͖͢Ͱͳ͍߹• ΠϕϯτΛهͯ͠࠶ݱ͢Δඞཁ͕ͳ͍߹• ه͞ΕͨΠϕϯτ͔Β ΦϒδΣΫτΛ෮ݩ͢Δ͜ͱ͕ͳ͍߹• খنͳΞϓϦέʔγϣϯɺ γϯϓϧͳCRUD͔͠ͳ͍ΞϓϦέʔγϣϯ
Event Sourcing• Ͳ͏࣮ͬͯ͢Δ͔ʁ• RDBMSͱΈ߹ΘͤͯશͯͷΠϕϯτใΛอʁ• େنͳσʔλʹରԠͰ͖Δ͔ʁ• ר͖͠ͷॲཧͲ͏͢Δ͔ʁ
Asynchronous / Synchronous Event
ಉظΠϕϯτ• ΠϕϯτಉظॲཧͰ͋Δ͖͔Ͳ͏͔ Πϕϯτൃੜ࣌ʹଈ࠲ʹଞΞϓϦέʔγϣϯʹө͢Δඞཁ͕͋Δ͔• ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋ɺ αϒεΫϥΠόʹΑΔσʔλӬଓԽΛߦ͏
ඇಉظΠϕϯτ• ΠϕϯτඇಉظʹͰ͖Δ͔Ͳ͏͔ Πϕϯτ͕ൃੜͨ͜͠ͱΛ୲อ͠ɺ өԆͤͯ͞Α͍͔• ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋·Ͱ σʔλӬଓԽԆͰ࣮ߦ͞ΕΔ
CQRS
ΞϓϦέʔγϣϯͷͰൃੜ͢Δ• σʔλͱύϑΥʔϚϯε• αʔϏεͱڞʹ૿͑ΔෳࡶͳΫΤϦͱ༰ྔ• σʔλͷਖ਼͠͞(ਖ਼نԽ)͔ޮԽ͔(ඇਖ਼نԽ)• RDBMSͷෆಘҙ + NoSQLʹΑΔΧόʔ ಉظํ๏ͷෳࡶԽ
σʔλϕʔεઃܭͷΈ• ਖ਼نԽͱநग़͍ͨ͠σʔλͷΪϟοϓ• நग़͍ͨ͠σʔλʹ߹ΘͤͨઃܭΛ͢Δ͜ͱʹΑΔ ੍͕͔͚ΒΕͳ͍δϨϯϚ• ࣮ίʔυͰ୲อ͢ΔͨΊʹෳࡶԽ
ཁٻ༷ʹରԠ͢ΔͨΊʹ σʔλϕʔεͰશͯରԠ͢Δͷਖ਼͍͔͠ਖ਼نԽɺඇਖ਼نԽ ಉ࣠͡Ͱߟ͑Δ͖͔
CQRS"A few myths about CQRS". Ouarzy's Blog. http://www.ouarzy.com/2016/10/02/a-few-myths-about-cqrs/ ࢀর
CQRSͱԿ͔• ॻ͖ࠐΈ(෭࡞༻͋Γ)ͱ ಡΈࠐΈ(෭࡞༻ͳ͠)Λ͢Δ• ෭࡞༻͋ΓσʔλͳͲʹԿΒ͔ͷมߋΛ༩͑Δͷ • ॻ͖ࠐΈ => Command • ಡΈࠐΈ => Query
CQRS• ͋ΒΏΔϝιουɺΞΫγϣϯΛ࣮ߦ͢ΔίϚϯυ͔ɺݺͼग़͠ݩʹσʔλΛฦ͢ΫΤϦ͔ͷ͍ͣΕ͔Ͱ͋ͬͯɺ྆ํΛߦͬͯͳΒͳ͍ɻ͜Εɺ࣭Λ͢Δ͜ͱͰճΛมԽͤͯ͞ͳΒͳ͍ͱ͍͏͜ͱͩɻ "Greg YoungྲྀCQRS - Mark Nijhof". Digital Romanticism. http://d.hatena.ne.jp/digitalsoul/20100712/1278886009 ࢀর
Command• ߹ੑ ੍Λ͔͚ͯਖ਼͘͠ɺݎ͘• ॻ͖ࠐΈɺߋ৽ɺআॲཧ ಡΈࠐΈʹൺͯޮੑɺ εέʔϥϏϦςΟΛҙࣝ͢Δ͜ͱ͋·Γͳ͍
Query• ߹ੑΑΓݕࡧޮ• ΄ͱΜͲͷΞϓϦέʔγϣϯʹ͓͍ͯɺ ॻ͖ࠐΈΑΓݕࡧͷൺॏ͕େ͖͍͜ͱ͕΄ͱΜͲ• εέʔϧ͕༰қͰ͋Δ͜ͱɺ ߴͳύϑΥʔϚϯεͰ͋Δ͜ͱ
Command, Query࣌ͷσʔλϕʔε
Command
Query
CQRS ApplicationApplication
ཧ۶Θ͔Δ͚ͲɺͲ͏࣮͢Ε͍͍ͷ͔ʁಉظͲ͏͢Δͷʁ
Event Sourcing + CQRS
Event Sourcing / Command ྫ
Event Sourcing / Command ྫΞϓϦέʔγϣϯ͔ΒΠϕϯτૹ৴
Event Sourcing / Command ྫϝοηʔδΛड͚औΓɺͷՃͳͲ
Event Sourcing / Command ྫRDBMSɺNoSQLͳͲ༻్ʹ߹Θͤͯॻ͖ࠐΈ
Event Sourcing / Command ྫଞͷΞϓϦέʔγϣϯ͔ΒσʔλϕʔεͷΈΛࢀরՃʹ͍ͭͯͷࣝΒͳ͍
Cache Systemσʔλϕʔεߋ৽ $PNNBOE σʔλϕʔεߋ৽ྃΠϕϯτ &WFOU4PVSDJOHσʔλϕʔεߋ৽ྃݕ &WFOU-JTUFOFSΩϟογϡআͳͲ
Event Sourcing• CommandɺQueryɺEvent Sourcing ผΞϓϦέʔγϣϯɾݴޠͰ࣮Ͱ͖Δ͘Β͍ͷ γϯϓϧ͞• EventอଘͱΞϓϦέʔγϣϯʹରԠͨ͠ QueryΛͲ͏ͬͯ ޮతʹ࣮͢Δ͔
ΞϓϦέʔγϣϯׂྫ
ΞϓϦέʔγϣϯׂྫPHP WebΞϓϦέʔγϣϯwebΞϓϦ։ൃνʔϜ
ΞϓϦέʔγϣϯׂྫPHP WebΞϓϦέʔγϣϯwebΞϓϦ։ൃνʔϜूܭॲཧ push etcCLI։ൃνʔϜ / DWH etc
ΞϓϦέʔγϣϯׂྫPHP WebΞϓϦέʔγϣϯwebΞϓϦ։ൃνʔϜूܭॲཧ push etcCLI։ൃνʔϜ / DWH etcࠂ৴ܥAPIࠂ৴։ൃνʔϜ
ΞϓϦέʔγϣϯׂྫPHP WebΞϓϦέʔγϣϯwebΞϓϦ։ൃνʔϜूܭॲཧ push etcCLI։ൃνʔϜ / DWH etcࠂ৴ܥAPIࠂ৴։ൃνʔϜϢʔβʔͷӾཡใʹج͍ͮͨใ͕ཉ͍͠Ͱ͢ϦΞϧλΠϜͰ
Event Sourcing / Message Broker
Event SourcingͱMiddlewareબఆ• PubSubͰشൃੑͷͳ͍ͷΛબ͢Δ• ϝοηʔδอ͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕ՄೳͳͷΛ࠾༻͢Δͷ͕·͍͠• εέʔϥϒϧͰ͋Δ͜ͱ
Event Sourcing / RDBMS• PubSubͰشൃੑͷͳ͍ͷΛબ͢Δ ػೳͳ͍ͷͰόονॲཧͳͲͰ ఆظతʹσʔλՃ͕͋Δ͔Ͳ͏͔Λݕࡧ͢Δ • ϝοηʔδอ͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕ՄೳͳͷΛ࠾༻͢Δͷ͕·͍͠ ಘҙ• εέʔϥϒϧͰ͋Δ͜ͱ Slaveɺmulti masterͳͲͰରԠͰ͖Δ
Event Sourcing / Redis• PubSubͰشൃੑͷͳ͍ͷΛબ͢Δ PUBLISH SUBSCRIBEͳͲ͕͋ΔͷͰ࣮ݱͰ͖Δ͕ɺ شൃੑʹ͍ͭͯઃఆͰؤுΔ • ϝοηʔδอ͕ՄೳͰɺ ࠶ݱ͢Δ͜ͱ͕ՄೳͳͷΛ࠾༻͢Δͷ͕·͍͠ τϥϯβΫγϣϯ͋Δ• εέʔϥϒϧͰ͋Δ͜ͱ ༰қʹͰ͖Δ
Message Broker
Apache Kafka• ZookeeperΛར༻ͨ͠ΫϥελϦϯάʹΑΔߴՄ༻ੑ• ϝοηʔδͷӬଓԽɺϨϓϦέʔγϣϯɺ࠶औಘՄ• ϏοάσʔλରԠ• ετϦʔϜରԠͷϝοηʔδϯάϛυϧΣΞ• Kafka ConnectʹΑΔपลγεςϜͱͷߴ͍ੑ
Event Sourcing• ΠϕϯτΛΠϕϯτετΞʹอଘ Kafkaࣗମʹอଘ͢Δ͜ͱ͕Ͱ͖ɺ σʔλϕʔεʹసૹՄೳ• ه͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩͰ͖Δ ॲཧ͞ΕͨΠϕϯτ࠶ൃߦՄೳ• ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτෆม ൃߦޙͷΠϕϯτΛॻ͖͑Δ͜ͱͰ͖ͳ͍• ൃੜͨ͠ࣄ࣮Λ͑Δ τϥϯβΫγϣϯ͋Γ
Apache Kafka֓ཁ• Partition ෛՙࢄ༻్ʹར༻ ෳͷConsumer͕ͦΕͧΕͷPartitionΛࢀর͠ɺ ͦΕͧΕ͕ॲཧΛߦ͏ ॲཧϑϩʔͷσβΠϯʹΑͬͯଟ༷ͳར༻ํ๏• librdkafka͕ඇৗʹߴ
Example Partition
Consumer
Kafka Connect• Kafka Connectͱɺ पลγεςϜ͔ΒͷσʔλΛऔΓࠐΈ(Source)ɺ σʔλૹ৴(Sink)ͷೋछྨΛαϙʔτ͢Δػೳ• Amazon SQSMongoDBͷσʔλΛKafkaͰऔࠐΉɺ ϝοηʔδΛͦͷ··ElasticsearchRDBMSʹ֨ೲɺ ͕ߦ͑Δ• Connectࣗ༝ʹ֦ுͯ͠ಠࣗConnectΛ࣮Մೳ
Kafka Connect• γϯϓϧͳέʔεͰ͋Εɺ ಛʹ࣮͢Δ͜ͱͳ͘QueryΛαϙʔτ• ෳࡶͳσʔλΛ࡞͢ΔͷͰ͋Ε Kafka StreamsɺSpark StreamingͰσʔλՃ
Amazon Kinesis• εέʔϥϒϧ• ϝοηʔδ 7ؒอ(࠷େ)• ετϦʔϜରԠͷKinesis Stream• Kinesis FirehoseΛͬͨσʔλసૹ Amazon S3ɺAmazon Redshiftɺ Amazon Elasticsearch Service etc
Event Sourcing+ CQRS / Kafka• Command QueryΛ୲͢ΔWebΞϓϦέʔγϣϯΛ γϯϓϧʹ• Event SourcingʹΑͬͯɺΞϓϦέʔγϣϯʹՁΛՃ͢Δ͜ͱ͕Ͱ͖Δ(ੳͳͲ)• Command QueryͷಉظΛαϙʔτ͠ɺ ΞϓϦέʔγϣϯʹ߹Θͤͯෛՙࢄ͕༰қ
ఆظߋ৽ͤ͞Δόονॲཧͷߟ͑ํมΘΔ
࣮ྫ
ྫ
ྫ༷ʑͳΞϓϦέʔγϣϯ͔Β σʔλૹ৴
ྫ"QBDIF,BGLB͕ ϝοηʔδड৴
ྫ,BGLB4QBSL4USFBNJOHΞϓϦέʔγϣϯ͔Βૹ৴͞Εͨσʔλͱɺ3%#.4ʹ֨ೲ͞ΕͨσʔλΛ݁߹͠ɺूܭɾूΛߦ͏
ྫूܭɾू͞ΕͨσʔλΛɺ ಡΈࠐΈͰར༻͢ΔΞϓϦέʔγϣϯʹ ߹Θͤͯอ$BTTBOESBͱ4QBSL4USFBNJOHͷΈͰ ೖग़ྗΛߦ͏έʔε
ྫूܭɾू͞Εͨσʔλͷ͏༷ͪʑͳՕॴͰར༻͞ΕΔͷɺ)%'4࠶ܭࢉɺোൃੜ࣌ʹ෮چͤ͞ΔͳͲ
ྫΞϓϦέʔγϣϯଆ͔Β$BTTBOESBͷΈʹ͍߹ΘͤΛߦ͏
ྫͦΕͧΕͷΞϓϦέʔγϣϯͷ$POTVNFSͰͦΕͧΕͷॲཧ
ྫ3BCCJU.2ϝοηʔδૹ৴ ଞαʔϏε͕ར༻
ྫ֤αʔϏεͷίϯγϡʔϚʔ͕ଞͷαϒγεςϜͰར༻͢ΔσʔλΛੜ
·ͱΊ• େنΞϓϦέʔγϣϯʹͳΔ͜ͱ͕༧ଌͰ͖ͨΒ• ϚΠΫϩαʔϏεΞʔΩςΫνϟΛࢧ͑Δ• ϛυϧΣΞͷࣝඞཁʹ• ࢄॲཧෳࡶͳσʔλॲཧͷ ϦϑΝΫλϦϯάʹ׆͖Δύλʔϯ
ytake
chiroruxx
odasho
carta_engineering
nikkei_engineer_recruiting
line_developers
aiji42
tsubakimoto_s
devops_vtj
regonn
ikuyamada
seike460
you
chrislema
tanoku
erikaheidi
sonatard
bkeepers
garrettdimon
deanohume
mza
kastner
danielanewman
morganepeng
destraynor