Have your cake and eat it: an introduction to event-driven architecture

Every day we lose data. Not by accident or disaster, but by implementation.

However, we can keep our data as we consume and act upon it. This talk is an introduction to event-driven architecture. We’ll learn how to implement event sourcing; revealing intent, decoupling our microservices and even allowing us to time-travel and change history when fixing bugs.

We’ll be simultaneously making our lives easier as developers whilst better supporting the growth of our businesses – it doesn’t get much better than that, does it?

This talk was given at Code Europe on December 13th, 2017. The recording is pending upload.

Transcript

1. Have your cake and eat it
 An introduction to event-driven architecture @RyanTownsend December 13th, 2017 Code Europe, Wrocław
2. Ryan Townsend Chief Technology Officer,
4. ✋ I know what event-driven architecture means ✋ I know what event sourcing means ✋ I know what CQRS means show of hands
5. 1. What is event-driven architecture? 2. Why is it important? 3. What’s possible? 4. Are there any downsides? 5. How can we use it today? agenda
6. “Data is the new oil”
7. “Data is the new oil.” Clive Humby, Architect of Tesco’s Clubcard, 2006 “Personal data is the new oil of the internet” Meglena Kuneva, European Consumer Commissioner, 2009 “Data is the new oil” Ann Winblad, Senior Partner at Hummer-Winblad, 2012 “Data is the new oil. The companies that will win are using math.”
 Kevin Plank, founder and CEO of Under Armour, 2016 “Data is the new oil.”
 Richard Titus, CEO AND, 2010 “We’re able to view just everything that they do. [...] Data is the new oil.”
 Bill Diggins, CEO Diggit, 2012 “I want you to think about data as the next natural resource.”
 Virginia Rometty, IBM CEO, 2013 “Information is the oil of the 21st century”
 Peter Sondergaard, SVP Gartner, 2011 “Data is going to be the natural resource for this industrial revolution.” Abhishek Mehta, CEO Tresata, 2013 Source: https://www.quora.com/Who-should-get-credit-for-the-quote-data-is-the-new-oil
8. Traditional relational data is a snapshot of ‘now’
9. Life and data are fluid
10. They are the product of events
11. • Financial transactions • Sports tournaments • Order shipments • Election polls • Flight arrivals, departures • Hotel check-in & check-out • Restaurant table availability • Read receipts for messages all kinds of events
12. And there are knock-on actions
13. Does the flap of a butterfly's wings in Brazil set off a tornado in Texas? butterfly effect
14. Can we model our applications closer to real-life?
15. • Notifications • State transfer • Event-sourcing • CQRS event-driven arch. Source: https://martinfowler.com/articles/201701-event-driven.html
16. Why are we throwing away our events in our applications?
17. Accountants and banks have known this for centuries
18. update accounts set balance = balance + 100 where id = ‘ryans-account’ update accounts set balance = balance - 100 where id = ‘someone-elses-account’
19. }, { event: ‘transaction-initiated’, from: ‘someone-elses-account’, to: ‘ryans-account’, amount: 100, timestamp: ‘2017-12-13T09:30:00’ }, {
20. Databases also know this: transaction logs
21. Apache Kafka: “distributed commit log”
22. So what makes this a good idea?
23. “You can’t have your cake and eat it”
24. Build a better product Provide a natural audit log Fix state, not just bugs Decouple services De-risk launches benefits
25. Item Quantity Pixel 2 1
26. Event Parameters Added Item To Cart Item: iPhone X, Quantity: 1 Changed Item Quantity Item: iPhone X, Quantity: 2 Removed Item From Cart Item: iPhone X Added Item To Cart Item: Pixel 2, Quantity: 2 Changed Item Quantity Item: Pixel 2, Quantity: 1
27. Item Quantity Pixel 2 1 iPhone X 2
28. Type Parameters Added Item To Cart Item: iPhone X, Quantity: 1 Changed Item Quantity Item: iPhone X, Quantity: 2 Removed Item From Cart Item: iPhone X Added Item To Cart Item: Pixel 2, Quantity: 2 Changed Item Quantity Item: Pixel 2, Quantity: 1 Event Parameters Added Item To Cart Item: iPhone X, Quantity: 1 Changed Item Quantity Item: iPhone X, Quantity: 2 Removed Item From Cart Item: iPhone X Added Item To Cart Item: Pixel 2, Quantity: 2 Changed Item Quantity Item: Pixel 2, Quantity: 1
29. Item Quantity Pixel 2 1 iPhone X 2
30. Item Quantity Pixel 2 1 ✅
31. ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫
32. ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫
33. ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫
34. “an immutable, time-ordered sequence of events as our primary store of state” Chris Castle, @crc
35. Event-driven architecture isn’t a silver bullet…
36. downsides ⏲ Asynchronicity & eventual consistency ✏ Corrections need events Replaying must factor in external systems Doesn’t play well with CRUD Demand on storage *
37. Event Streaming
38. Votes Tweets Tweets Votes Votes Tweets Consumers Tweet Analyser Tweet Broadcaster Vote Broadcaster Vote Persister Event Stream
39. Votes Tweets Tweets, Votes Tweets Consumers Tweet Analyser Message Archiver Event Stream
40. Event-Sourcing & CQRS
41. Command Query Responsibility Segregation • Commonly paired with event-sourcing • The two aren’t mutually exclusive • Sounds complicated, but isn’t… • Reality: your input and output don’t share models or APIs • Great way to collect user intent cqrs
42. ❗ ❓ alter read Command Query State cqrs
43. ❗ ❓ generates reads Command Query State Events alter cqrs
44. REST vs CQRS REST • PATCH /line_items/123 • Updating quantity? • Adding a custom logo? CQRS • /update_line_item_quantity • /update_custom_graphic • Specific methods
45. GraphQL Mutations : CQRS
46. Why Apache Kafka? Open-sourced by LinkedIn, now 1.0 Fault-tolerant and robust Deliverability guarantees ⏰ Chronological ordering of messages ⬇ Pull-based asynchronous processing event stream
47. Why Kafka on Heroku? ⚡ Instant provisioning Plans from $100/mo High-availability as standard ↔ Easy to share between applications Thoughtful developer experience event stream
48. I know what event-driven architecture means I know what event sourcing means I know what CQRS means summary
49. Thank you Code Europe
50. Kafka vs RabbitMQ • Kafka is more horizontally-scalable • Kafka is better at acting as a buffer • RabbitMQ has per-message acknowledgements • Kafka has message offset acknowledgement • Both have ordering guarantees • Kafka can handle higher message input