Event Sourcing+CQRS を活用するスケーラブルアプリケーション開発 / event-sourcing-cqrs-v2

Event Sourcing+CQRS を活用するスケーラブルアプリケーション開発 / event-sourcing-cqrs-v2

AWS Dev Day 2018
マイクロサービスアーキテクチャの裏側の巻

17d4ef53b432ebf7c566fd6a11345570?s=128

yuuki takezawa

November 02, 2018
Tweet

Transcript

  1. Event Sourcing+CQRSΛ׆༻͢Δ εέʔϥϒϧΞϓϦέʔγϣϯ։ൃ yuuki takezawa AWS Dev Day Tokyo 2018

  2. Profile • ஛ᖒ ༗و / ytake • גࣜձࣾΞΠελΠϧ CTO •

    PHP, Hack, Go, Scala • Apache Hadoop, Apache Spark, Apache Kafka
 • twitter https://twitter.com/ex_takezawa • facebook https://www.facebook.com/yuuki.takezawa • github https://github.com/ytake
  3. ΞϓϦέʔγϣϯͷ੒௕

  4. ΞϓϦέʔγϣϯྫ • ϩάΠϯϢʔβʔ͸ίϝϯτ͕౤ߘͰ͖Δ • ະϩάΠϯϢʔβʔ͸ӾཡͷΈ͕Ͱ͖Δ • ͓ؾʹೖΓͨ͠ϢʔβʔͷίϝϯτΛӾཡͰ͖Δ • ࣌ܥྻͰίϝϯτΛӾཡͰ͖Δ •

    ίϝϯτͷ࡟আ͕Ͱ͖Δ • ίϝϯτͷฤू͸Ͱ͖ͳ͍ 
 ͳͲ
  5. ΞϓϦέʔγϣϯ࣮૷ • ೝূػೳͷ࣮૷
 • ίϝϯτ౤ߘɺฤूɺӾཡػೳ࣮૷
 • ݕࡧػೳͷ࣮૷

  6. γϯϓϧͳΞϓϦέʔγϣϯߏ੒

  7. ෳࡶԽ͢ΔΞϓϦέʔγϣϯ • ػೳ௥Ճ
 • ίϝϯτ৘ใΛଞͷػೳͰར༻͍ͨ͠
 • ৚݅ʹΑΔҰཡ৘ใͷιʔτมߋ
 Ϣʔβʔ৘ใଐੑʹΑΔιʔτॱมߋ

  8. ෳࡶԽ͢ΔΞϓϦέʔγϣϯ

  9. ංେԽ͢ΔΞϓϦέʔγϣϯ • ऩӹԽͷͨΊͷػೳ
 • རศੑͷͨΊͷػೳ • طଘػೳͷ֦ு

  10. ංେԽ͢ΔΞϓϦέʔγϣϯ

  11. ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά • ݁߹ͨ͠ػೳΛίϯϙʔωϯτఏڙ
 • APIʹΑΔσʔλૢ࡞ͷந৅Խ
 • σʔλϕʔεύϑΥʔϚϯεվળ


  12. ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά

  13. ڊେͳΞϓϦέʔγϣϯ΁

  14. ਂࠁͳ໰୊ • ࢓༷มߋʹ൐͏ίϯϙʔωϯτमਖ਼ͱϦϦʔεෳࡶԽ
 ϥΠϒϥϦͷόʔδϣϯ͋͛ͳ͖Όɾɾʂ
 ͋ͷػೳΛऔΓࠐΉʹ͸͋Εͱ͜Εͱɾɾɾ • ΞϓϦέʔγϣϯؒ࿈ܞͷෳࡶԽ
 APIमਖ਼ʹ൐͏Өڹൣғௐࠪ • σʔλಉظͷෳࡶԽ


    ϦΞϧλΠϜͰσʔλ͕ཉ͍͠
 ΠϯσοΫεߋ৽͕ఆظ࣮ߦ
  15. ੒௕͍ͯ͘͠ࣄۀʹ߹Θͤͯɺ ΞϓϦέʔγϣϯΛ Ͳ͏εέʔϧ͍͔ͤͯ͘͞

  16. มΘΓΏ͘؀ڥ • σʔλΛऔಘ͢Δʹ͸ʁ
 APIίʔϧʹΑΔ౎౓֬ೝʁ
 Ωϟογϡ͸Կ෼ʁ
 • ͦΕͧΕͷαʔϏεͰͦΕͧΕͷݴޠ΁
 ͋ΔαʔϏε͸PHPɺ͋ΔαʔϏεͰ͸Go ScalaͳͲ


  17. σʔλॲཧ Ϣʔβʔ৘ใ͸౎౓໰͍߹Θͤ ৘ใΛߋ৽ͨ͠৔߹Ͱ΋
 ґଘαʔϏε͕ͳ͍ͨΊ໰୊ͳ͠

  18. σʔλॲཧͷෳࡶԽ Ϣʔβʔ৘ใมߋ Ϣʔβʔ৘ใ࡟আ Ϣʔβʔ৘ใ͕ݟ͔ͭΒͳ͍ͨΊ
 σʔλॲཧ͝ͱʹ"1*ΞΫηε ͋ͦ͜ͱͦ͜Ͱ৘ใ͕ҧ͏ʂ
 Ͳͷσʔλ͕ຊ෺ʁ શͯͷΞϓϦέʔγϣϯ͔Β
 Ωϟογϡͳ͠ͰΞΫηε
 ৗʹߴෛՙঢ়ଶʹ

  19. Event / Application Side

  20. Event? • Observerύλʔϯ • ʙͨ࣌͠ʹɺʙ͢Δ Λදݱ͢Δ • ༷ʑͳΞϓϦέʔγϣϯͰར༻͞ΕΔ

  21. Event / Application • γεςϜཁ݅ɾػೳͱͯ͠औΓೖΕΔέʔε΋ଟ͍ • Request͕དྷͨΒɺRouteΛىಈͤ͞Δ • ʙͷॲཧ͕׬ྃͨ͠Βɺϩάʹॻ͖ग़͢

  22. public function __invoke( FulltextRequest $request ): RedirectResponse { // ొ࿥ॲཧޙʹevent

    try { $entity = new Entity(1, ‘testing’); $this->dispatcher->dispatch( new SinkConnect($this->usecase->run($entity)) ); } catch(\RuntimeException $e) { // Կ͔errorॲཧ } return $this->redirector->route('fulltext.index'); }
  23. ҰͭͷΞϓϦέʔγϣϯͷ৔߹ • ΞϓϦέʔγϣϯ಺ͷ༷ʑͳग़དྷࣄΛ
 ObserverΛ࢖࣮ͬͯ૷͢Δ • ΞϓϦέʔγϣϯͷΫϥε͕ංେԽͨ࣌͠
 খ͞ͳΫϥεʹ෼ׂ͢Δ৔߹΍ɺ
 ϦϑΝΫλϦϯάʹར༻͢Δ͜ͱ΋
 • ୯ҰͷΞϓϦέʔγϣϯ಺Ͱ࣮૷͢Δ৔߹ʹ༗༻

  24. ߟྀ͢Δ͜ͱ • ΞϓϦέʔγϣϯؒͰ࿈ܞ͢Δ৔߹͸Ͳ͏͢΂͖͔ • ʙͨ࣌͠ʹɺʙ͢Δ
 ଞͷΞϓϦέʔγϣϯɾݴޠͰ࣮ߦ͞ΕΔ͔
 ੾Γग़͢ඞཁ͕͋Δ͔Ͳ͏͔

  25. Event Sourcing

  26. Event Sourcing • ΠϕϯτΛه࿥ͯ͠࠶ݱ͢Δύλʔϯ • ΠϕϯτΛΠϕϯτετΞʹอଘ • ه࿥͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩ • ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτ͸ෆม

    • ൃੜͨ͠ࣄ࣮Λ఻͑Δ
  27. Event SourcingΛಋೖ͢΂͖Ͱ͸ͳ͍৔߹ • ΠϕϯτΛه࿥ͯ͠࠶ݱ͢Δඞཁ͕ͳ͍৔߹ • ه࿥͞ΕͨΠϕϯτ͔Β
 ΦϒδΣΫτΛ෮ݩ͢Δ͜ͱ͕ͳ͍৔߹ • খن໛ͳΞϓϦέʔγϣϯɺ
 γϯϓϧͳCRUD͔͠ͳ͍ΞϓϦέʔγϣϯ

  28. Event Sourcing • Ͳ͏΍࣮ͬͯ૷͢Δ͔ʁ • RDBMSͱ૊Έ߹ΘͤͯશͯͷΠϕϯτ৘ใΛอ؅ʁ • େن໛ͳσʔλʹ͸ରԠͰ͖Δ͔ʁ • ר͖໭͠ͷॲཧ͸Ͳ͏͢Δ͔ʁ

  29. Asynchronous / Synchronous Event

  30. ಉظΠϕϯτ • Πϕϯτ͸ಉظॲཧͰ͋Δ΂͖͔Ͳ͏͔
 Πϕϯτൃੜ࣌ʹଈ࠲ʹଞΞϓϦέʔγϣϯʹ൓ө͢Δ ඞཁ͕͋Δ͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋ɺ
 αϒεΫϥΠόʹΑΔσʔλӬଓԽΛߦ͏

  31. ඇಉظΠϕϯτ • Πϕϯτ͸ඇಉظʹͰ͖Δ͔Ͳ͏͔
 Πϕϯτ͕ൃੜͨ͜͠ͱΛ୲อ͠ɺ
 ൓ө͸஗Ԇͤͯ͞΋Α͍͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋·Ͱ
 σʔλӬଓԽ͸஗ԆͰ࣮ߦ͞ΕΔ

  32. CQRS

  33. ΞϓϦέʔγϣϯͷ੒௕Ͱൃੜ͢Δ໰୊ • σʔλͱύϑΥʔϚϯε • αʔϏε੒௕ͱڞʹ૿͑ΔෳࡶͳΫΤϦͱ༰ྔ • σʔλͷਖ਼͠͞(ਖ਼نԽ)͔ޮ཰Խ͔(ඇਖ਼نԽ) • RDBMSͷෆಘҙ෼໺ +

    NoSQLʹΑΔΧόʔ
 ಉظํ๏ͷෳࡶԽ
  34. σʔλϕʔεઃܭͷ೰Έ • ਖ਼نԽͱநग़͍ͨ͠σʔλͷΪϟοϓ • நग़͍ͨ͠σʔλʹ߹ΘͤͨઃܭΛ͢Δ͜ͱʹΑΔ
 ੍໿͕͔͚ΒΕͳ͍δϨϯϚ • ࣮૷ίʔυͰ୲อ͢ΔͨΊʹෳࡶԽ

  35. ཁٻ࢓༷ʹରԠ͢ΔͨΊʹ
 σʔλϕʔεͰશͯରԠ͢Δͷ͸ਖ਼͍͔͠ ਖ਼نԽɺඇਖ਼نԽ͸
 ಉ࣠͡Ͱߟ͑Δ΂͖͔

  36. CQRS "A few myths about CQRS". Ouarzy's Blog. 
 http://www.ouarzy.com/2016/10/02/a-few-myths-about-cqrs/

    ࢀর
  37. CQRSͱ͸Կ͔ • ॻ͖ࠐΈ(෭࡞༻͋Γ)ͱ
 ಡΈࠐΈ(෭࡞༻ͳ͠)Λ෼཭͢Δ • ෭࡞༻͋Γ͸σʔλͳͲʹԿΒ͔ͷมߋΛ༩͑Δ΋ͷ
 • ॻ͖ࠐΈ => Command


    • ಡΈࠐΈ => Query
  38. CQRS • ͋ΒΏΔϝιου͸ɺΞΫγϣϯΛ࣮ߦ͢ΔίϚϯυ ͔ɺݺͼग़͠ݩʹσʔλΛฦ͢ΫΤϦ͔ͷ͍ͣΕ͔Ͱ ͋ͬͯɺ྆ํΛߦͬͯ͸ͳΒͳ͍ɻ͜Ε͸ɺ࣭໰Λ͢Δ ͜ͱͰճ౴ΛมԽͤͯ͞͸ͳΒͳ͍ͱ͍͏͜ͱͩɻ
 
 "Greg YoungྲྀCQRS -

    Mark Nijhof". Digital Romanticism. 
 http://d.hatena.ne.jp/digitalsoul/20100712/1278886009 ࢀর 

  39. Command • ੔߹ੑ
 ੍໿Λ͔͚ͯਖ਼͘͠ɺݎ͘ • ॻ͖ࠐΈɺߋ৽ɺ࡟আॲཧ͸
 ಡΈࠐΈʹൺ΂ͯޮ཰ੑ΍଎౓ɺ
 εέʔϥϏϦςΟΛҙࣝ͢Δ͜ͱ͸͋·Γͳ͍

  40. Query • ੔߹ੑΑΓ΋ݕࡧޮ཰ • ΄ͱΜͲͷΞϓϦέʔγϣϯʹ͓͍ͯɺ
 ॻ͖ࠐΈΑΓ΋ݕࡧͷൺॏ͕େ͖͍͜ͱ͕΄ͱΜͲ • εέʔϧ͕༰қͰ͋Δ͜ͱɺ
 ߴ଎ͳύϑΥʔϚϯεͰ͋Δ͜ͱ

  41. Command, Query෼཭࣌ͷσʔλϕʔε

  42. Command

  43. Command

  44. Query

  45. CQRS Application Application

  46. ཧ۶͸Θ͔Δ͚Ͳɺ Ͳ͏࣮૷͢Ε͹͍͍ͷ͔ʁ ಉظ͸Ͳ͏͢Δͷʁ

  47. Event Sourcing + CQRS

  48. Event Sourcing / Command ྫ

  49. Event Sourcing / Command ྫ ΞϓϦέʔγϣϯ͔Β Πϕϯτૹ৴

  50. Event Sourcing / Command ྫ ϝοηʔδΛड͚औΓɺ ஋ͷՃ޻ͳͲ

  51. Event Sourcing / Command ྫ RDBMS΍ɺNoSQLͳͲ ༻్ʹ߹Θͤͯॻ͖ࠐΈ

  52. Event Sourcing / Command ྫ ଞͷΞϓϦέʔγϣϯ͔Β σʔλϕʔεͷΈΛࢀর Ճ޻ʹ͍ͭͯͷ஌ࣝ͸஌Βͳ͍

  53. Cache System σʔλϕʔεߋ৽
 $PNNBOE σʔλϕʔεߋ৽׬ྃΠϕϯτ
 &WFOU4PVSDJOH σʔλϕʔεߋ৽׬ྃݕ஌
 &WFOU-JTUFOFS Ωϟογϡ࡟আͳͲ

  54. Event Sourcing • CommandɺQueryɺEvent Sourcing͸
 ผΞϓϦέʔγϣϯɾݴޠͰ࣮૷Ͱ͖Δ͘Β͍ͷ
 γϯϓϧ͞ • EventอଘͱΞϓϦέʔγϣϯʹରԠͨ͠
 QueryΛͲ͏΍ͬͯ


    ޮ཰తʹ࣮૷͢Δ͔
  55. ΞϓϦέʔγϣϯ෼ׂྫ

  56. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ

  57. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH

    etc
  58. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH

    etc ޿ࠂ഑৴ܥAPI ޿ࠂ഑৴։ൃνʔϜ
  59. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH

    etc ޿ࠂ഑৴ܥAPI ޿ࠂ഑৴։ൃνʔϜ ϢʔβʔͷӾཡ৘ใʹج͍ͮͨ ৘ใ͕ཉ͍͠Ͱ͢ ϦΞϧλΠϜͰ
  60. Event Sourcing / Message Broker

  61. Event SourcingͱMiddlewareબఆ • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ • ϝοηʔδอ؅͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷ Λ࠾༻͢Δͷ͕๬·͍͠ • εέʔϥϒϧͰ͋Δ͜ͱ

  62. Event Sourcing / RDBMS • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ
 ػೳ͸ͳ͍ͷͰόονॲཧͳͲͰ
 ఆظతʹσʔλ௥Ճ͕͋Δ͔Ͳ͏͔Λݕࡧ͢Δ
 • ϝοηʔδอ؅͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷΛ࠾༻͢Δͷ

    ͕๬·͍͠
 ಘҙ෼໺ • εέʔϥϒϧͰ͋Δ͜ͱ
 Slaveɺmulti masterͳͲͰରԠ͸Ͱ͖Δ
  63. Event Sourcing / Redis • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ
 PUBLISH SUBSCRIBEͳͲ͕͋ΔͷͰ࣮ݱ͸Ͱ͖Δ͕ɺ
 شൃੑʹ͍ͭͯ͸ઃఆͰؤுΔ
 •

    ϝοηʔδอ؅͕ՄೳͰɺ
 ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷΛ࠾༻͢Δͷ͕๬·͍͠
 τϥϯβΫγϣϯ͸͋Δ • εέʔϥϒϧͰ͋Δ͜ͱ
 ༰қʹͰ͖Δ
  64. None
  65. None
  66. Message Broker

  67. None
  68. Apache Kafka • ZookeeperΛར༻ͨ͠ΫϥελϦϯάʹΑΔߴՄ༻ੑ • ϝοηʔδͷӬଓԽɺϨϓϦέʔγϣϯɺ࠶औಘՄ • ϏοάσʔλରԠ • ετϦʔϜରԠͷϝοηʔδϯάϛυϧ΢ΣΞ

    • Kafka ConnectʹΑΔपลγεςϜͱͷߴ͍਌࿨ੑ
  69. Event Sourcing • ΠϕϯτΛΠϕϯτετΞʹอଘ
 Kafkaࣗମʹอଘ͢Δ͜ͱ͕Ͱ͖ɺ
 σʔλϕʔεʹసૹՄೳ • ه࿥͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩͰ͖Δ
 ॲཧ͞ΕͨΠϕϯτ͸࠶ൃߦՄೳ •

    ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτ͸ෆม
 ൃߦޙͷΠϕϯτΛॻ͖׵͑Δ͜ͱ͸Ͱ͖ͳ͍ • ൃੜͨ͠ࣄ࣮Λ఻͑Δ
 τϥϯβΫγϣϯ͋Γ
  70. Apache Kafka֓ཁ • Partition
 ෛՙ෼ࢄ༻్ʹར༻
 ෳ਺ͷConsumer͕ͦΕͧΕͷPartitionΛࢀর͠ɺ
 ͦΕͧΕ͕ॲཧΛߦ͏
 ॲཧϑϩʔͷσβΠϯʹΑͬͯଟ༷ͳར༻ํ๏ • librdkafka͕ඇৗʹߴ଎

  71. Example Partition

  72. Consumer

  73. Kafka Connect • Kafka Connectͱ͸ɺ
 पลγεςϜ͔ΒͷσʔλΛऔΓࠐΈ(Source)ɺ
 σʔλૹ৴(Sink)ͷೋछྨΛαϙʔτ͢Δػೳ • Amazon SQS΍MongoDBͷσʔλΛKafkaͰऔࠐΉɺ


    ϝοηʔδΛͦͷ··Elasticsearch΍RDBMSʹ֨ೲɺ
 ͕ߦ͑Δ • Connect͸ࣗ༝ʹ֦ுͯ͠ಠࣗConnectΛ࣮૷Մೳ
  74. Kafka Connect • γϯϓϧͳέʔεͰ͋Ε͹ɺ
 ಛʹ࣮૷͢Δ͜ͱͳ͘QueryΛαϙʔτ • ෳࡶͳσʔλΛ࡞੒͢ΔͷͰ͋Ε͹
 Kafka StreamsɺSpark StreamingͰσʔλՃ޻

  75. None
  76. Amazon Kinesis • εέʔϥϒϧ • ϝοηʔδ 7೔ؒอ؅(࠷େ) • ετϦʔϜରԠͷKinesis Stream

    • Kinesis FirehoseΛ࢖ͬͨσʔλసૹ
 Amazon S3ɺAmazon Redshiftɺ
 Amazon Elasticsearch Service etc
  77. Event Sourcing+ CQRS / Kafka • Command QueryΛ୲౰͢ΔWebΞϓϦέʔγϣϯΛ
 γϯϓϧʹ •

    Event SourcingʹΑͬͯɺΞϓϦέʔγϣϯʹՁ஋Λ௥ Ճ͢Δ͜ͱ͕Ͱ͖Δ(෼ੳͳͲ) • Command QueryͷಉظΛαϙʔτ͠ɺ
 ΞϓϦέʔγϣϯʹ߹Θͤͯෛՙ෼ࢄ͕༰қ
  78. ఆظߋ৽ͤ͞Δόονॲཧͷߟ͑ํ΋มΘΔ

  79. ࣮ྫ

  80. ྫ ༷ʑͳΞϓϦέʔγϣϯ͔Β
 σʔλૹ৴

  81. ྫ "QBDIF,BGLB͕
 ϝοηʔδड৴

  82. ྫ ,BGLB 4QBSL4USFBNJOH ΞϓϦέʔγϣϯ͔Βૹ৴͞Εͨσʔλͱɺ 3%#.4ʹ֨ೲ͞ΕͨσʔλΛ݁߹͠ɺ ूܭɾू໿Λߦ͏

  83. ྫ ूܭɾू໿͞ΕͨσʔλΛɺ
 ಡΈࠐΈͰར༻͢ΔΞϓϦέʔγϣϯʹ
 ߹Θͤͯอ؅ $BTTBOESBͱ4QBSL4USFBNJOHͷΈͰ
 ೖग़ྗΛߦ͏έʔε΋

  84. ྫ ूܭɾू໿͞Εͨσʔλͷ͏ͪ ༷ʑͳՕॴͰར༻͞ΕΔ΋ͷ͸ɺ)%'4΁ ࠶ܭࢉ΍ɺো֐ൃੜ࣌ʹ෮چͤ͞ΔͳͲ

  85. ྫ ΞϓϦέʔγϣϯଆ͔Β͸ $BTTBOESBͷΈʹ໰͍߹ΘͤΛߦ͏

  86. ྫ ͦΕͧΕͷΞϓϦέʔγϣϯͷ $POTVNFSͰͦΕͧΕͷॲཧ

  87. ྫ 3BCCJU.2΁ϝοηʔδૹ৴
 ଞαʔϏε͕ར༻

  88. ྫ ֤αʔϏεͷίϯγϡʔϚʔ͕ ଞͷαϒγεςϜͰར༻͢ΔσʔλΛੜ੒

  89. ·ͱΊ • େن໛ΞϓϦέʔγϣϯʹͳΔ͜ͱ͕༧ଌͰ͖ͨΒ • ϚΠΫϩαʔϏεΞʔΩςΫνϟΛࢧ͑Δ • ϛυϧ΢ΣΞͷ஌ࣝ͸ඞཁʹ • ෼ࢄॲཧ΍ෳࡶͳσʔλॲཧͷ
 ϦϑΝΫλϦϯάʹ΋׆͖Δύλʔϯ