Events. Events. Events! - krk.rb

Basic conceptions of event sourcing and how to use it in ruby.

Talk from https://krk-rb.pl

Transcript

4. Hello krk.rb
5. Hard day
8. Second time in Poland
11. What I know now
14. Pierogi
15. Pelmeni -> Pierogi z mięsem Varenyky -> Pierogi ruskie
16. Awesome people and community
17. Anton Davydov github.com/davydovanton
 twitter.com/anton_davydov davydovanton.com
20. • Architect/software developer • OpenSource evangelist • Hanami core developer • Writer/programming streamer • Technical consulting
21. Stickers
22. • Coffee • Beer • Psychology • SW-2039-0208-4228 / 3DS also welcome • How to draw images for presentations • Bad stories
23. • Coffee • Beer • Psychology • SW-2039-0208-4228 / 3DS also welcome • How to draw images for presentations • Bad stories
24. Tried to be a whiter and make a story
25. expectations
26. reality
27. But I got help from my friend
28. Meet George
29. He is quiet and love to ask questions
32. Events
36. Git
37. Each commit - something that happened
39. Each commit contains description and changes (payload)
41. What happened
42. Payload
43. List of cart items which user deleted and added
44. Activity audit
45. Don’t delete deleted data
46. deleted_at
47. But we usually work with the current state of application
48. events -> actions
49. events -> actions current state -> data in DB
50. Knok knok ▼ George
51. … ▼ George
52. What will happen if we change this situation? ▼ George
53. And start to store events ▼ George
54. And calculate current state when we need it ▼ George
55. Event Sourcing
56. Store that happened instead of current state
57. And get the specific state when you need it
58. Where to store?
59. Event storage
60. It can be anything
62. Rules
63. Event happened
64. Immutable events
65. Can’t delete or update event
69. Only a valid data in the events
75. Data evolution
81. Stop ▼ George
82. We can break the app! ▼ George
83. >_< ▼ George
84. Pro tips: optional fields, binary data structures
85. more in this book
86. How to get the state from the event store?
87. We have to calculate it using current event list
88. Projections
89. project(projection, events, init_state) -> state
90. project(projection, events, init_state) -> state same as a block in ruby’s enumerator
91. project(projection, events, init_state) -> state pure function
92. Similar to #each_with_object
93. events.project(init_state, &projection) -> state
94. events.project(init_state, &projection) -> state [].each_with_object({}, &projection)
107. We recalculate current state for all events each time
108. … ▼ George
109. We can’t store all data for last 10 years ▼ George
110. But we will have a performance issues in getting current state ▼ George
111. How to fix it?
112. Streams!
114. Event bus#1 Event bus#1 Event bus#2 Event bus#1 Event bus#2 Event bus#2
115. Event stream: bus#1 Event stream: bus#1 Event stream: bus#2 Event stream: bus#1 Event stream: bus#2 Event stream: bus#2
116. Stream: bus#1
 Event stream: bus#1 Event stream: bus#1 Event stream: bus#1
117. Stream: bus#2
 Event stream: bus#2 Event stream: bus#2 Event stream: bus#2
118. Pros: less events for processing
119. Cons: one more abstraction
120. Add snapshot for the current data state!
121. project(function, events, state) -> state
122. project(function, events, state) -> state {}
123. project(function, events, state) -> state { users: […] }
130. You can use any DB as a cache
132. Idea: use more that one DB for cache
133. Relation DB for search and index page
134. Document oriented DB for show page
135. Real world
136. Where it can be useful
137. E-comerse systems
138. Order and checkout flow
139. Something where you need a version control (wiki, docs, group editing)
140. Domain depends on events (tracking systems)
141. redux? ▼ George
142. Event sourcing and event driven architecture are different things
143. And they use events in different ways
144. ES -> event happened and I store it ED -> event happened and I notify everyone about it
145. Blockchain! ▼ George
146. pros
147. Easier to communicate with domain experts
148. • Logging out from box • Time traveling • Restore system’s state
149. • You don’t work with tables you work with events and changes • Experemental data structures • Easy to change database implementations
150. cons
151. • Hard to understand idea and complicated abstraction • Not popular in ruby • Developers need deprogramming
152. • Harder to get state than CRUD • Hard to understand the whole chain of events • Another architecture type
 with different DB structure
153. • Versions and versions compatibility • Updating or deleting events • Eventual Consistency
154. Advanced topics
155. How to display created data ASAP?
156. Solution 1
157. cheat user display real data
163. solution 2
164. WebSockets and Long Polling
165. How to make right events order?
170. distributed systems
171. more here
172. And here https://youtu.be/ArTS_AJ-smQ
173. How to delete a user related data
174. Retroactive Event
176. Lose encrypted data key
177. How to use ES or something like this for searching in the event store?
178. How to use ES in ruby
179. Simple way
180. gems
182. github.com/zilverline/sequent
184. eventide-project.org
186. Brave way
187. Build all stuff by yourself
190. Or pure kafka
192. Or pure SQL
193. github.com/davydovanton/ivento * Repository with pure sql implementation
194. github.com/davydovanton/ivento * And without tests
195. hanami-events
196. Event driven transport layer with event versions and types
197. Connection without global state
198. Hanami::Events.new(:memory_sync) Hanami::Events.new(:memory_async) Hanami::Events.new(:redis)
199. Hanami::Events::Adapter.register(:sql) do Adapter::SQL end Hanami::Events.new(:sql) # => event instance with your sql adapter
200. events = Hanami::Events.new(:memory_sync) events.broadcast('user.created', user: user) Broadcaster
201. events.subscribe('user.created') do |payload| puts payload end events.subscribe('user.*') do |payload| puts payload end events.subscribe(/\Auser\..*/) do |payload| puts payload end Subscriber
202. Event sourcing looks cool. ▼ George
203. But I have legacy app. How can I migrate? ▼ George
204. User adds something to cart
205. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? if cart_repo.item_exist?(params[:item]) item = CartItemRepository.new.create_copy(params[:item]) else item = CartItemRepository.new.create(params[:item]) end analytics.call('Item Added') redirect_to routes.order_path(params[:order_id]) end end end
206. Step 1: Event Storming
207. Why? Detect important business events in the system
208. Orders::Events::ItemAdded Orders::Events::CoppyItemAdded
209. Step 2: store (apply) event
210. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? event_store.apply(Order::Events::ItemAdded.new(payload)) if cart_repo.item_exist?(params[:item]) item = CartItemRepository.new.create_copy(params[:item]) else item = CartItemRepository.new.create(params[:item]) end analytics.call('Item Added') redirect_to routes.order_path(params[:order_id]) end end end
211. How to validate item?
212. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? event_store.apply(Orders::Events::ItemAdded.new(payload)) if cart_repo.item_exist?(params[:item]) item = CartItemRepository.new.create_copy(params[:item]) else item = CartItemRepository.new.create(params[:item]) end analytics.call('Item Added') redirect_to routes.order_path(params[:order_id]) end end end
213. Step 3: calculate state
214. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? events = event_store.get_stream(params[:order_id]) items = @project.call(Order::Projections::ItemList, events) if items.include?(params[:item]) event_store.apply(Orders::Events::ItemAdded.new(payload)) else event_store.apply(Orders::Events::CoppyItemAdded.new(payload)) end analytics.call('Item Added') redirect_to routes.order_path(params[:order_id]) end end end
215. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? events = event_store.get_stream(params[:order_id]) items = @project.call(Order::Projections::ItemList, events, {}) if items.include?(params[:item]) event_store.apply(Orders::Events::ItemAdded.new(payload)) else event_store.apply(Orders::Events::CoppyItemAdded.new(payload)) end analytics.call('Item Added') redirect_to routes.order_path(params[:order_id]) end end end
216. module Projections class AllTask def call(event, state) case event when Orders::Events::ItemAdded state[:orders] ||= [] state[:orders] << event.payload when Orders::Events::ItemRemoved # ... end state end end end
217. module Projections class AllTask def call(event, state) case event when Orders::Events::ItemAdded state[:orders] ||= [] state[:orders] << event.payload when Orders::Events::ItemRemoved # ... end state end end end
218. module Projections class AllTask def call(event, state) case event when Orders::Events::ItemAdded state[:orders] ||= [] state[:orders] << event.payload when Orders::Events::ItemRemoved # ... end state end end end
219. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? events = event_store.get_stream(params[:order_id]) items = @project.call(Order::Projections::ItemList, events, {}) if items.include?(params[:item]) event_store.apply(Orders::Events::ItemAdded.new(payload)) else event_store.apply(Orders::Events::CoppyItemAdded.new(payload)) end analytics.call('Item Added') redirect_to routes.order_path(params[:order_id]) end end end
220. Step 4: subscribers
221. # analytics subscribers event_store.subscribe(Orders::Events::ItemAdded) do |event| analytics.call('Item removed', event.payload) end event_store.subscribe(Orders::Events::CoppyItemAdded) do |event| analytics.call(‘Copy of item added’, event.payload) end event_store.subscribe(Orders::Events::ItemRemoved) do |event| analytics.call(‘Item removed', event.payload) end
222. module Web::Controllers::CartItem class Create include Web::Action def call(params) halt(400) unless params.valid? events = event_store.get_stream(params[:order_id]) items = @project.call(Order::Projections::ItemList, events, {}) if items.include?(params[:item]) event_store.apply(Orders::Events::ItemAdded.new(payload)) else event_store.apply(Orders::Events::CoppyItemAdded.new(payload)) end redirect_to routes.order_path(params[:order_id]) end end end
223. Step 4: move logic to domain
224. PROFIT
225. pro tip: you can use ES only as a part of your system
226. What next?
229. SAGA PATTERN
230. Long business transactions
233. CQRS
 
 (Command Query Responsibility Segregation)
234. – Martin Fowler “You can use a different model to update information than the model you use to read information”
235. Looking outside ruby world
236. github.com/heynickc/ awesome-ddd
237. github.com/davydovanton/event_sourcing_ruby
238. DDD
239. Again? ▼ George
240. Detect and store business events
241. Split system by domains
245. Distributed systems
246. more here
247. Conclusions
248. Sometimes event base architecture wins
249. Sometimes not
250. EventSourcing is simple as conception
251. But it introduces a lot of practical problems
 (data evolution, event order, etc)
252. ES expensive
253. It’s not about distributed systems and communication across services
254. ES will be a trend in the future
255. Don’t you trust me?
256. – Andrzej Krzywda “ES will be a trend in the future”
257. twitter.com/anton_davydov github.com/davydovanton mail@davydovanton.com davydovanton.com Thank you ❤