Event Sourcing+CQRS を活用するスケーラブルアプリケーション開発 / event-sourcing-cqrs-v2

Transcript

1. Event Sourcing+CQRSΛ׆༻͢Δ εέʔϥϒϧΞϓϦέʔγϣϯ։ൃ yuuki takezawa AWS Dev Day Tokyo 2018

2. Profile • ஛ᖒ ༗و / ytake • גࣜձࣾΞΠελΠϧ CTO •

   PHP, Hack, Go, Scala • Apache Hadoop, Apache Spark, Apache Kafka
 • twitter https://twitter.com/ex_takezawa • facebook https://www.facebook.com/yuuki.takezawa • github https://github.com/ytake

3. ΞϓϦέʔγϣϯͷ੒௕

4. ΞϓϦέʔγϣϯྫ • ϩάΠϯϢʔβʔ͸ίϝϯτ͕౤ߘͰ͖Δ • ະϩάΠϯϢʔβʔ͸ӾཡͷΈ͕Ͱ͖Δ • ͓ؾʹೖΓͨ͠ϢʔβʔͷίϝϯτΛӾཡͰ͖Δ • ࣌ܥྻͰίϝϯτΛӾཡͰ͖Δ •

   ίϝϯτͷ࡟আ͕Ͱ͖Δ • ίϝϯτͷฤू͸Ͱ͖ͳ͍ 
 ͳͲ

5. ΞϓϦέʔγϣϯ࣮૷ • ೝূػೳͷ࣮૷
 • ίϝϯτ౤ߘɺฤूɺӾཡػೳ࣮૷
 • ݕࡧػೳͷ࣮૷

6. γϯϓϧͳΞϓϦέʔγϣϯߏ੒

7. ෳࡶԽ͢ΔΞϓϦέʔγϣϯ • ػೳ௥Ճ
 • ίϝϯτ৘ใΛଞͷػೳͰར༻͍ͨ͠
 • ৚݅ʹΑΔҰཡ৘ใͷιʔτมߋ
 Ϣʔβʔ৘ใଐੑʹΑΔιʔτॱมߋ

8. ෳࡶԽ͢ΔΞϓϦέʔγϣϯ

9. ංେԽ͢ΔΞϓϦέʔγϣϯ • ऩӹԽͷͨΊͷػೳ
 • རศੑͷͨΊͷػೳ • طଘػೳͷ֦ு

10. ංେԽ͢ΔΞϓϦέʔγϣϯ

11. ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά • ݁߹ͨ͠ػೳΛίϯϙʔωϯτఏڙ
 • APIʹΑΔσʔλૢ࡞ͷந৅Խ
 • σʔλϕʔεύϑΥʔϚϯεվળ


12. ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά

13. ڊେͳΞϓϦέʔγϣϯ΁

14. ਂࠁͳ໰୊ • ࢓༷มߋʹ൐͏ίϯϙʔωϯτमਖ਼ͱϦϦʔεෳࡶԽ
 ϥΠϒϥϦͷόʔδϣϯ͋͛ͳ͖Όɾɾʂ
 ͋ͷػೳΛऔΓࠐΉʹ͸͋Εͱ͜Εͱɾɾɾ • ΞϓϦέʔγϣϯؒ࿈ܞͷෳࡶԽ
 APIमਖ਼ʹ൐͏Өڹൣғௐࠪ • σʔλಉظͷෳࡶԽ


    ϦΞϧλΠϜͰσʔλ͕ཉ͍͠
 ΠϯσοΫεߋ৽͕ఆظ࣮ߦ

15. ੒௕͍ͯ͘͠ࣄۀʹ߹Θͤͯɺ ΞϓϦέʔγϣϯΛ Ͳ͏εέʔϧ͍͔ͤͯ͘͞

16. มΘΓΏ͘؀ڥ • σʔλΛऔಘ͢Δʹ͸ʁ
 APIίʔϧʹΑΔ౎౓֬ೝʁ
 Ωϟογϡ͸Կ෼ʁ
 • ͦΕͧΕͷαʔϏεͰͦΕͧΕͷݴޠ΁
 ͋ΔαʔϏε͸PHPɺ͋ΔαʔϏεͰ͸Go ScalaͳͲ


17. σʔλॲཧ Ϣʔβʔ৘ใ͸౎౓໰͍߹Θͤ ৘ใΛߋ৽ͨ͠৔߹Ͱ΋
 ґଘαʔϏε͕ͳ͍ͨΊ໰୊ͳ͠

18. σʔλॲཧͷෳࡶԽ Ϣʔβʔ৘ใมߋ Ϣʔβʔ৘ใ࡟আ Ϣʔβʔ৘ใ͕ݟ͔ͭΒͳ͍ͨΊ
 σʔλॲཧ͝ͱʹ"1*ΞΫηε ͋ͦ͜ͱͦ͜Ͱ৘ใ͕ҧ͏ʂ
 Ͳͷσʔλ͕ຊ෺ʁ શͯͷΞϓϦέʔγϣϯ͔Β
 Ωϟογϡͳ͠ͰΞΫηε
 ৗʹߴෛՙঢ়ଶʹ

19. Event / Application Side

20. Event? • Observerύλʔϯ • ʙͨ࣌͠ʹɺʙ͢Δ Λදݱ͢Δ • ༷ʑͳΞϓϦέʔγϣϯͰར༻͞ΕΔ

21. Event / Application • γεςϜཁ݅ɾػೳͱͯ͠औΓೖΕΔέʔε΋ଟ͍ • Request͕དྷͨΒɺRouteΛىಈͤ͞Δ • ʙͷॲཧ͕׬ྃͨ͠Βɺϩάʹॻ͖ग़͢

22. public function __invoke( FulltextRequest $request ): RedirectResponse { // ొ࿥ॲཧޙʹevent

    try { $entity = new Entity(1, ‘testing’); $this->dispatcher->dispatch( new SinkConnect($this->usecase->run($entity)) ); } catch(\RuntimeException $e) { // Կ͔errorॲཧ } return $this->redirector->route('fulltext.index'); }

23. ҰͭͷΞϓϦέʔγϣϯͷ৔߹ • ΞϓϦέʔγϣϯ಺ͷ༷ʑͳग़དྷࣄΛ
 ObserverΛ࢖࣮ͬͯ૷͢Δ • ΞϓϦέʔγϣϯͷΫϥε͕ංେԽͨ࣌͠
 খ͞ͳΫϥεʹ෼ׂ͢Δ৔߹΍ɺ
 ϦϑΝΫλϦϯάʹར༻͢Δ͜ͱ΋
 • ୯ҰͷΞϓϦέʔγϣϯ಺Ͱ࣮૷͢Δ৔߹ʹ༗༻

24. ߟྀ͢Δ͜ͱ • ΞϓϦέʔγϣϯؒͰ࿈ܞ͢Δ৔߹͸Ͳ͏͢΂͖͔ • ʙͨ࣌͠ʹɺʙ͢Δ
 ଞͷΞϓϦέʔγϣϯɾݴޠͰ࣮ߦ͞ΕΔ͔
 ੾Γग़͢ඞཁ͕͋Δ͔Ͳ͏͔

25. Event Sourcing

26. Event Sourcing • ΠϕϯτΛه࿥ͯ͠࠶ݱ͢Δύλʔϯ • ΠϕϯτΛΠϕϯτετΞʹอଘ • ه࿥͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩ • ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτ͸ෆม

    • ൃੜͨ͠ࣄ࣮Λ఻͑Δ

27. Event SourcingΛಋೖ͢΂͖Ͱ͸ͳ͍৔߹ • ΠϕϯτΛه࿥ͯ͠࠶ݱ͢Δඞཁ͕ͳ͍৔߹ • ه࿥͞ΕͨΠϕϯτ͔Β
 ΦϒδΣΫτΛ෮ݩ͢Δ͜ͱ͕ͳ͍৔߹ • খن໛ͳΞϓϦέʔγϣϯɺ
 γϯϓϧͳCRUD͔͠ͳ͍ΞϓϦέʔγϣϯ

28. Event Sourcing • Ͳ͏΍࣮ͬͯ૷͢Δ͔ʁ • RDBMSͱ૊Έ߹ΘͤͯશͯͷΠϕϯτ৘ใΛอ؅ʁ • େن໛ͳσʔλʹ͸ରԠͰ͖Δ͔ʁ • ר͖໭͠ͷॲཧ͸Ͳ͏͢Δ͔ʁ

29. Asynchronous / Synchronous Event

30. ಉظΠϕϯτ • Πϕϯτ͸ಉظॲཧͰ͋Δ΂͖͔Ͳ͏͔
 Πϕϯτൃੜ࣌ʹଈ࠲ʹଞΞϓϦέʔγϣϯʹ൓ө͢Δ ඞཁ͕͋Δ͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋ɺ
 αϒεΫϥΠόʹΑΔσʔλӬଓԽΛߦ͏

31. ඇಉظΠϕϯτ • Πϕϯτ͸ඇಉظʹͰ͖Δ͔Ͳ͏͔
 Πϕϯτ͕ൃੜͨ͜͠ͱΛ୲อ͠ɺ
 ൓ө͸஗Ԇͤͯ͞΋Α͍͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋·Ͱ
 σʔλӬଓԽ͸஗ԆͰ࣮ߦ͞ΕΔ

32. CQRS

33. ΞϓϦέʔγϣϯͷ੒௕Ͱൃੜ͢Δ໰୊ • σʔλͱύϑΥʔϚϯε • αʔϏε੒௕ͱڞʹ૿͑ΔෳࡶͳΫΤϦͱ༰ྔ • σʔλͷਖ਼͠͞(ਖ਼نԽ)͔ޮ཰Խ͔(ඇਖ਼نԽ) • RDBMSͷෆಘҙ෼໺ +

    NoSQLʹΑΔΧόʔ
 ಉظํ๏ͷෳࡶԽ

34. σʔλϕʔεઃܭͷ೰Έ • ਖ਼نԽͱநग़͍ͨ͠σʔλͷΪϟοϓ • நग़͍ͨ͠σʔλʹ߹ΘͤͨઃܭΛ͢Δ͜ͱʹΑΔ
 ੍໿͕͔͚ΒΕͳ͍δϨϯϚ • ࣮૷ίʔυͰ୲อ͢ΔͨΊʹෳࡶԽ

35. ཁٻ࢓༷ʹରԠ͢ΔͨΊʹ
 σʔλϕʔεͰશͯରԠ͢Δͷ͸ਖ਼͍͔͠ ਖ਼نԽɺඇਖ਼نԽ͸
 ಉ࣠͡Ͱߟ͑Δ΂͖͔

36. CQRS "A few myths about CQRS". Ouarzy's Blog. 
 http://www.ouarzy.com/2016/10/02/a-few-myths-about-cqrs/

    ࢀর

37. CQRSͱ͸Կ͔ • ॻ͖ࠐΈ(෭࡞༻͋Γ)ͱ
 ಡΈࠐΈ(෭࡞༻ͳ͠)Λ෼཭͢Δ • ෭࡞༻͋Γ͸σʔλͳͲʹԿΒ͔ͷมߋΛ༩͑Δ΋ͷ
 • ॻ͖ࠐΈ => Command


    • ಡΈࠐΈ => Query

38. CQRS • ͋ΒΏΔϝιου͸ɺΞΫγϣϯΛ࣮ߦ͢ΔίϚϯυ ͔ɺݺͼग़͠ݩʹσʔλΛฦ͢ΫΤϦ͔ͷ͍ͣΕ͔Ͱ ͋ͬͯɺ྆ํΛߦͬͯ͸ͳΒͳ͍ɻ͜Ε͸ɺ࣭໰Λ͢Δ ͜ͱͰճ౴ΛมԽͤͯ͞͸ͳΒͳ͍ͱ͍͏͜ͱͩɻ
 
 "Greg YoungྲྀCQRS -

    Mark Nijhof". Digital Romanticism. 
 http://d.hatena.ne.jp/digitalsoul/20100712/1278886009 ࢀর 


39. Command • ੔߹ੑ
 ੍໿Λ͔͚ͯਖ਼͘͠ɺݎ͘ • ॻ͖ࠐΈɺߋ৽ɺ࡟আॲཧ͸
 ಡΈࠐΈʹൺ΂ͯޮ཰ੑ΍଎౓ɺ
 εέʔϥϏϦςΟΛҙࣝ͢Δ͜ͱ͸͋·Γͳ͍

40. Query • ੔߹ੑΑΓ΋ݕࡧޮ཰ • ΄ͱΜͲͷΞϓϦέʔγϣϯʹ͓͍ͯɺ
 ॻ͖ࠐΈΑΓ΋ݕࡧͷൺॏ͕େ͖͍͜ͱ͕΄ͱΜͲ • εέʔϧ͕༰қͰ͋Δ͜ͱɺ
 ߴ଎ͳύϑΥʔϚϯεͰ͋Δ͜ͱ

41. Command, Query෼཭࣌ͷσʔλϕʔε

42. Command

43. Command

44. Query

45. CQRS Application Application

46. ཧ۶͸Θ͔Δ͚Ͳɺ Ͳ͏࣮૷͢Ε͹͍͍ͷ͔ʁ ಉظ͸Ͳ͏͢Δͷʁ

47. Event Sourcing + CQRS

48. Event Sourcing / Command ྫ

49. Event Sourcing / Command ྫ ΞϓϦέʔγϣϯ͔Β Πϕϯτૹ৴

50. Event Sourcing / Command ྫ ϝοηʔδΛड͚औΓɺ ஋ͷՃ޻ͳͲ

51. Event Sourcing / Command ྫ RDBMS΍ɺNoSQLͳͲ ༻్ʹ߹Θͤͯॻ͖ࠐΈ

52. Event Sourcing / Command ྫ ଞͷΞϓϦέʔγϣϯ͔Β σʔλϕʔεͷΈΛࢀর Ճ޻ʹ͍ͭͯͷ஌ࣝ͸஌Βͳ͍

53. Cache System σʔλϕʔεߋ৽
 $PNNBOE σʔλϕʔεߋ৽׬ྃΠϕϯτ
 &WFOU
    4PVSDJOH σʔλϕʔεߋ৽׬ྃݕ஌
 &WFOU
    -JTUFOFS Ωϟογϡ࡟আͳͲ

54. Event Sourcing • CommandɺQueryɺEvent Sourcing͸
 ผΞϓϦέʔγϣϯɾݴޠͰ࣮૷Ͱ͖Δ͘Β͍ͷ
 γϯϓϧ͞ • EventอଘͱΞϓϦέʔγϣϯʹରԠͨ͠
 QueryΛͲ͏΍ͬͯ


    ޮ཰తʹ࣮૷͢Δ͔

55. ΞϓϦέʔγϣϯ෼ׂྫ

56. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ

57. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH

    etc

58. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH

    etc ޿ࠂ഑৴ܥAPI ޿ࠂ഑৴։ൃνʔϜ

59. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH

    etc ޿ࠂ഑৴ܥAPI ޿ࠂ഑৴։ൃνʔϜ ϢʔβʔͷӾཡ৘ใʹج͍ͮͨ ৘ใ͕ཉ͍͠Ͱ͢ ϦΞϧλΠϜͰ

60. Event Sourcing / Message Broker

61. Event SourcingͱMiddlewareબఆ • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ • ϝοηʔδอ؅͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷ Λ࠾༻͢Δͷ͕๬·͍͠ • εέʔϥϒϧͰ͋Δ͜ͱ

62. Event Sourcing / RDBMS • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ
 ػೳ͸ͳ͍ͷͰόονॲཧͳͲͰ
 ఆظతʹσʔλ௥Ճ͕͋Δ͔Ͳ͏͔Λݕࡧ͢Δ
 • ϝοηʔδอ؅͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷΛ࠾༻͢Δͷ

    ͕๬·͍͠
 ಘҙ෼໺ • εέʔϥϒϧͰ͋Δ͜ͱ
 Slaveɺmulti masterͳͲͰରԠ͸Ͱ͖Δ

63. Event Sourcing / Redis • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ
 PUBLISH SUBSCRIBEͳͲ͕͋ΔͷͰ࣮ݱ͸Ͱ͖Δ͕ɺ
 شൃੑʹ͍ͭͯ͸ઃఆͰؤுΔ
 •

    ϝοηʔδอ؅͕ՄೳͰɺ
 ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷΛ࠾༻͢Δͷ͕๬·͍͠
 τϥϯβΫγϣϯ͸͋Δ • εέʔϥϒϧͰ͋Δ͜ͱ
 ༰қʹͰ͖Δ
64. None
65. None

66. Message Broker

67. None

68. Apache Kafka • ZookeeperΛར༻ͨ͠ΫϥελϦϯάʹΑΔߴՄ༻ੑ • ϝοηʔδͷӬଓԽɺϨϓϦέʔγϣϯɺ࠶औಘՄ • ϏοάσʔλରԠ • ετϦʔϜରԠͷϝοηʔδϯάϛυϧ΢ΣΞ

    • Kafka ConnectʹΑΔपลγεςϜͱͷߴ͍਌࿨ੑ

69. Event Sourcing • ΠϕϯτΛΠϕϯτετΞʹอଘ
 Kafkaࣗମʹอଘ͢Δ͜ͱ͕Ͱ͖ɺ
 σʔλϕʔεʹసૹՄೳ • ه࿥͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩͰ͖Δ
 ॲཧ͞ΕͨΠϕϯτ͸࠶ൃߦՄೳ •

    ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτ͸ෆม
 ൃߦޙͷΠϕϯτΛॻ͖׵͑Δ͜ͱ͸Ͱ͖ͳ͍ • ൃੜͨ͠ࣄ࣮Λ఻͑Δ
 τϥϯβΫγϣϯ͋Γ

70. Apache Kafka֓ཁ • Partition
 ෛՙ෼ࢄ༻్ʹར༻
 ෳ਺ͷConsumer͕ͦΕͧΕͷPartitionΛࢀর͠ɺ
 ͦΕͧΕ͕ॲཧΛߦ͏
 ॲཧϑϩʔͷσβΠϯʹΑͬͯଟ༷ͳར༻ํ๏ • librdkafka͕ඇৗʹߴ଎

71. Example Partition

72. Consumer

73. Kafka Connect • Kafka Connectͱ͸ɺ
 पลγεςϜ͔ΒͷσʔλΛऔΓࠐΈ(Source)ɺ
 σʔλૹ৴(Sink)ͷೋछྨΛαϙʔτ͢Δػೳ • Amazon SQS΍MongoDBͷσʔλΛKafkaͰऔࠐΉɺ


    ϝοηʔδΛͦͷ··Elasticsearch΍RDBMSʹ֨ೲɺ
 ͕ߦ͑Δ • Connect͸ࣗ༝ʹ֦ுͯ͠ಠࣗConnectΛ࣮૷Մೳ

74. Kafka Connect • γϯϓϧͳέʔεͰ͋Ε͹ɺ
 ಛʹ࣮૷͢Δ͜ͱͳ͘QueryΛαϙʔτ • ෳࡶͳσʔλΛ࡞੒͢ΔͷͰ͋Ε͹
 Kafka StreamsɺSpark StreamingͰσʔλՃ޻

75. None

76. Amazon Kinesis • εέʔϥϒϧ • ϝοηʔδ 7೔ؒอ؅(࠷େ) • ετϦʔϜରԠͷKinesis Stream

    • Kinesis FirehoseΛ࢖ͬͨσʔλసૹ
 Amazon S3ɺAmazon Redshiftɺ
 Amazon Elasticsearch Service etc

77. Event Sourcing+ CQRS / Kafka • Command QueryΛ୲౰͢ΔWebΞϓϦέʔγϣϯΛ
 γϯϓϧʹ •

    Event SourcingʹΑͬͯɺΞϓϦέʔγϣϯʹՁ஋Λ௥ Ճ͢Δ͜ͱ͕Ͱ͖Δ(෼ੳͳͲ) • Command QueryͷಉظΛαϙʔτ͠ɺ
 ΞϓϦέʔγϣϯʹ߹Θͤͯෛՙ෼ࢄ͕༰қ

78. ఆظߋ৽ͤ͞Δόονॲཧͷߟ͑ํ΋มΘΔ

79. ࣮ྫ

80. ྫ

81. ྫ ༷ʑͳΞϓϦέʔγϣϯ͔Β
 σʔλૹ৴

82. ྫ "QBDIF
    ,BGLB͕
 ϝοηʔδड৴

83. ྫ ,BGLB 4QBSL
    4USFBNJOH
    ΞϓϦέʔγϣϯ͔Βૹ৴͞Εͨσʔλͱɺ 3%#.4ʹ֨ೲ͞ΕͨσʔλΛ݁߹͠ɺ
    ूܭɾू໿Λߦ͏

84. ྫ ूܭɾू໿͞ΕͨσʔλΛɺ
 ಡΈࠐΈͰར༻͢ΔΞϓϦέʔγϣϯʹ
 ߹Θͤͯอ؅
    $BTTBOESBͱ4QBSL
    4USFBNJOHͷΈͰ
 ೖग़ྗΛߦ͏έʔε΋

85. ྫ ूܭɾू໿͞Εͨσʔλͷ͏ͪ
    ༷ʑͳՕॴͰར༻͞ΕΔ΋ͷ͸ɺ)%'4΁
    ࠶ܭࢉ΍ɺো֐ൃੜ࣌ʹ෮چͤ͞ΔͳͲ

86. ྫ ΞϓϦέʔγϣϯଆ͔Β͸
    $BTTBOESBͷΈʹ໰͍߹ΘͤΛߦ͏

87. ྫ ͦΕͧΕͷΞϓϦέʔγϣϯͷ
    $POTVNFSͰͦΕͧΕͷॲཧ

88. ྫ 3BCCJU.2΁ϝοηʔδૹ৴
 ଞαʔϏε͕ར༻

89. ྫ ֤αʔϏεͷίϯγϡʔϚʔ͕
    ଞͷαϒγεςϜͰར༻͢ΔσʔλΛੜ੒

90. ·ͱΊ • େن໛ΞϓϦέʔγϣϯʹͳΔ͜ͱ͕༧ଌͰ͖ͨΒ • ϚΠΫϩαʔϏεΞʔΩςΫνϟΛࢧ͑Δ • ϛυϧ΢ΣΞͷ஌ࣝ͸ඞཁʹ • ෼ࢄॲཧ΍ෳࡶͳσʔλॲཧͷ
 ϦϑΝΫλϦϯάʹ΋׆͖Δύλʔϯ