Event Sourcing architecture using Elixir (old version)

Event Sourcing architecture using Elixir (old version)

NEW SLIDE VERSION: https://speakerdeck.com/macabeus/event-sourcing-architecture-using-elixir

Talk at "Elug Meetups #13".

Video (in other meetup, using old slides version): https://youtu.be/tixWGkkZbXE
Projects used on this talk: https://github.com/macabeus/event-sourcing-example
My blog: http://macalogs.com.br/

F380535da59d6cdd5754e2e31bda8a0e?s=128

Bruno Macabeus

April 07, 2018
Tweet

Transcript

  1. EVENT SOURCING ARCHITECTURE USING ELIXIR Bruno Macabeus

  2. HELLO! ͜Μʹͪ͸ʂ

  3. Bruno Macabeus github.com/macabeus macalogs.com.br I’m a developer at Pagar.me ❤

  4. WHAT IS EVENT SOURCING?

  5. “The core idea of event sourcing is that whenever we

    make a change to the state of a system, we record that state change as an event, and we can confidently rebuild the system state by reprocessing the events at any time in the future.” - Martin Fowler youtu.be/STKCRSUsyP0 bit.ly/martin-event-driven
  6. This architecture has many interesting benefits!

  7. This architecture has many interesting benefits! Event log provides a

    strong audit capability
  8. This architecture has many interesting benefits! Event log provides a

    strong audit capability We can recreate historic states by replaying the event log up to a point
  9. This architecture has many interesting benefits! We can explore alternative

    histories by injecting hypothetical events when replaying Event log provides a strong audit capability We can recreate historic states by replaying the event log up to a point
  10. This architecture has many interesting benefits! Event sourcing makes it

    plausible to have non-durable working copies, such as a Memory Image. We can explore alternative histories by injecting hypothetical events when replaying Event log provides a strong audit capability We can recreate historic states by replaying the event log up to a point
  11. This architecture has many interesting benefits! Event sourcing makes it

    plausible to have non-durable working copies, such as a Memory Image. We can explore alternative histories by injecting hypothetical events when replaying Event log provides a strong audit capability We can recreate historic states by replaying the event log up to a point
  12. This architecture has many interesting benefits! Event sourcing makes it

    plausible to have non-durable working copies, such as a Memory Image. We can explore alternative histories by injecting hypothetical events when replaying Event log provides a strong audit capability We can recreate historic states by replaying the event log up to a point
  13. Did you know? Database journal file and Git are examples

    of using event sourcing!
  14. Did you know? Database journal file and Git are examples

    of using event sourcing!
  15. THE (VERY) SIMPLE FLUX

  16. A SIMPLE COUNTER

  17. User Server Current state: Events log: 0

  18. User Server Current state: Events log: 0 in-memory persistent

  19. User Server Current state: Events log: 0 User “creates" an

    event. An event represents something that happened, a fact
  20. User Server Current state: Events log: 0 In this example,

    the user “creates an event”. But, on the others implementations of this architecture, another part could create the events
  21. User Server Current state: Events log: 0

  22. User Server Current state: Events log: 0 Event “arrives" on

    the server. On the others implementations, the server could “create" the event
  23. User Server Current state: Events log: +1 1

  24. User Server Current state: Events log: +1 1

  25. User Server Current state: Events log: +1 1

  26. User Server Current state: Events log: +1 2 +1

  27. User Server Current state: Events log: +1 2 +1 Note

    that the events log aren’t the represent the final state
  28. User Server Current state: Events log: +1 2 +1

  29. User Server Current state: Events log: +1 2 +1

  30. User Server Current state: Events log: -1 1 +1 +1

  31. User Server Current state: Events log: -1 1 +1 +1

    It’s a non-deterministic event
  32. User Server Current state: Events log: -1 1 +1 +1

  33. User Server Current state: Events log: -1 1 +1 +1

  34. User Server Current state: Events log: -1 5 +1 +1

    +4
  35. User Server Current state: Events log: -1 5 +1 +1

    +4 Now the events log has too many events
  36. User Server Current state: Events log: -1 5 +1 +1

    +4 Snapshot: state 5
  37. User Server Current state: Events log: -1 5 +1 +1

    +4 Snapshot: state 5
  38. User Server Current state: Events log: -1 5 +1 +1

    +4 Snapshot: state 5
  39. User Server Current state: Events log: -1 6 +1 +1

    +4 +1 Snapshot: state 5
  40. User Server Current state: Events log: -1 ?? +1 +1

    +4 +1 Snapshot: state 5 What if we lose the state?
  41. User Server Current state: Events log: -1 ?? +1 +1

    +4 +1 Snapshot: state 5 We could run again the events to restore the previous state! Using the events, we could convert the stream of events into a structural representation. This process is called “projection".
  42. GIVE ME A CODE EXAMPLE!

  43. A SIMPLE BANKING SYSTEM

  44. Let’s see an Elixir project that uses event sourcing architecture!

    github.com/macabeus/event-sourcing-example
  45. GIVE ME AN EXAMPLE! HOW EVENT SOURCING WAS APPLIED IN

    THIS PROJECT?
  46. None
  47. None
  48. None
  49. None
  50. GIVE ME AN EXAMPLE! API

  51. None
  52. None
  53. None
  54. GIVE ME AN EXAMPLE! EVENTS

  55. None
  56. None
  57. Hey! It’s important the events be agnostic of a language,

    then you’ll can work using another languages on same project. Flux Standard Action is an example of solution about it. I didn't do it in my example. My example works only on Elixir
  58. GIVE ME AN EXAMPLE! PROCESS AND BUS PIPELINE

  59. None
  60. None
  61. None
  62. None
  63. None
  64. None
  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. GIVE ME AN EXAMPLE! DATABASE

  72. In this project, I used two databases:

  73. - An in-memory database, Mnesia, using the wrapper Amnesia In

    this project, I used two databases: Mnesia
  74. - An in-memory database, Mnesia, using the wrapper Amnesia In

    this project, I used two databases: - A storage file database, DETS Mnesia DETS
  75. - An in-memory database, Mnesia, using the wrapper Amnesia In

    this project, I used two databases: - A storage file database, DETS Both, Mnesia and DETS, are in OTP! That is, built-in on Erlang/Elixir! Mnesia DETS OTP
  76. BUT THERE ARE MANY MANY OTHERS WAYS TO BUILD AN

    EVENT SOURCING APPLICATION!
  77. MY SOLUTION SUCKS!

  78. ? ? ? ? !

  79. ? ! PROJECTIONS ? ?

  80. A projection is the transformation of a set of data

    from one form into another.
  81. A projection is the transformation of a set of data

    from one form into another. In the case of event sourcing architecture, we convert a stream of events into any structural representation.
  82. A projection is the transformation of a set of data

    from one form into another. In the case of event sourcing architecture, we convert a stream of events into any structural representation. -1 +1 +1 +4
  83. Projection isn’t exclusive for event sourcing. But, it’s useful for

    event sourcing. !
  84. CQRS REPORTING DATABASE PROJECTIONS ? !

  85. CQRS: split the responsibility of "read" and “write" on the

    database.
  86. CQRS: split the responsibility of "read" and “write" on the

    database. Reporting database: a database optimized for read. If a database is optimized for read, then could have redundancy and multiples databases to query.
  87. CQRS: split the responsibility of "read" and “write" on the

    database. Reporting database: a database optimized for read. If a database is optimized for read, then could have redundancy and multiples databases to query. And, applying it with event sourcing: we could have a database that centralize all events received by the system (“event storage”), and many small databases optimized to read. This both reads the same bus and be updates when a new event is forwarded on the bus.
  88. CQRS and Reporting Database aren’t exclusive for event sourcing. But,

    it’s useful when we are using event sourcing. !
  89. COMMANDS ! CQRS REPORTING DATABASE PROJECTIONS

  90. Commands are de requests issued by the user, and maybe

    fail. E.g: AddFunds
  91. Commands are de requests issued by the user, and maybe

    fail. E.g: AddFunds When a command are performed, maybe it generates events - that is, facts about the application state changes E.g: FundsAdded
  92. Life cycle:

  93. VALIDATION Validations: checks if the command makes sense and objects

    are properly suited for further actions (is this event is to me?) Life cycle:
  94. VALIDATION CONSEQUENCE Validations: checks if the command makes sense and

    objects are properly suited for further actions (is this event is to me?) Consequences: initiating some action that will change the state of the world (do I need to forward a new event?) Life cycle:
  95. VALIDATION CONSEQUENCE DERIVATION Validations: checks if the command makes sense

    and objects are properly suited for further actions (is this event is to me?) Consequences: initiating some action that will change the state of the world (do I need to forward a new event?) Derivations: figuring out some information based on information we already have (I created a token and I’ll send it to user) Life cycle:
  96. CQRS REPORTING DATABASE PROJECTIONS COMMANDS THE BIG FLUX!

  97. User Event storage User can send a command on bus…

  98. User Event storage Create an account List accounts Send a

    validate e-mail Validate an account User can send a command on bus, and many process manager (PM) reads this bus…
  99. User Event storage Create an account List accounts Send a

    validate e-mail Validate an account User can send a command on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus.
  100. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account User can send a command on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data.
  101. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send a command on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  102. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send a command on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  103. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  104. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  105. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  106. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  107. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  108. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  109. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  110. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  111. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  112. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  113. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  114. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  115. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  116. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  117. User Event storage Create an account Send a validate e-mail

    List accounts Validate an account Events processing are asynchronous! User can send an event on bus, and many process manager (PM) reads this bus. Maybe a PM could send a new event on the bus. And, the PM needs read events in Event Storage, to re-build when it loses the data. And everything could be async!
  118. YOU DON’T NEED TO DO IT YOURSELF

  119. COMMANDED FRAMEWORK

  120. Commanded framework! http://github.com/commanded

  121. Commanded has more concepts that I didn’t say, (e.g: aggregate)

    and a great wiki.
  122. Talk at Elixir Brasil by Bernardo Amorim! % http://bit.ly/elixir-brasil-bernado

  123. Talk at Code Beam by Bernardo Amorim! % http://bit.ly/code-beam-bernardo-amorim

  124. FURTHERMORE…

  125. An idle game using event sourcing! https://github.com/telemmo

  126. Visit my blog! ❤ http://macalogs.com.br/