Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Have your cake and eat it: an introduction to event-driven architecture

Have your cake and eat it: an introduction to event-driven architecture

Every day we lose data. Not by accident or disaster, but by implementation.

However, we can keep our data as we consume and act upon it. This talk is an introduction to event-driven architecture. We’ll learn how to implement event sourcing; revealing intent, decoupling our microservices and even allowing us to time-travel and change history when fixing bugs.

We’ll be simultaneously making our lives easier as developers whilst better supporting the growth of our businesses – it doesn’t get much better than that, does it?

This talk was given at Code Europe on December 13th, 2017. The recording is pending upload.

F085bf2092cb300bac787cc5bc65d301?s=128

Ryan Townsend

December 13, 2017
Tweet

Transcript

  1. Have your cake and eat it
 An introduction to event-driven

    architecture @RyanTownsend December 13th, 2017 Code Europe, Wrocław
  2. Ryan Townsend Chief Technology Officer,

  3. None
  4. ✋ I know what event-driven architecture means ✋ I know

    what event sourcing means ✋ I know what CQRS means show of hands
  5. 1. What is event-driven architecture? 2. Why is it important?

    3. What’s possible? 4. Are there any downsides? 5. How can we use it today? agenda
  6. “Data is the new oil”

  7. “Data is the new oil.” Clive Humby, Architect of Tesco’s

    Clubcard, 2006 “Personal data is the new oil of the internet” Meglena Kuneva, European Consumer Commissioner, 2009 “Data is the new oil” Ann Winblad, Senior Partner at Hummer-Winblad, 2012 “Data is the new oil. The companies that will win are using math.”
 Kevin Plank, founder and CEO of Under Armour, 2016 “Data is the new oil.”
 Richard Titus, CEO AND, 2010 “We’re able to view just everything that they do. [...] Data is the new oil.”
 Bill Diggins, CEO Diggit, 2012 “I want you to think about data as the next natural resource.”
 Virginia Rometty, IBM CEO, 2013 “Information is the oil of the 21st century”
 Peter Sondergaard, SVP Gartner, 2011 “Data is going to be the natural resource for this industrial revolution.” Abhishek Mehta, CEO Tresata, 2013 Source: https://www.quora.com/Who-should-get-credit-for-the-quote-data-is-the-new-oil
  8. Traditional relational data is a snapshot of ‘now’

  9. Life and data are fluid

  10. They are the product of events

  11. • Financial transactions • Sports tournaments • Order shipments •

    Election polls • Flight arrivals, departures • Hotel check-in & check-out • Restaurant table availability • Read receipts for messages all kinds of events
  12. And there are knock-on actions

  13. Does the flap of a butterfly's wings in Brazil set

    off a tornado in Texas? butterfly effect
  14. Can we model our applications closer to real-life?

  15. • Notifications • State transfer • Event-sourcing • CQRS event-driven

    arch. Source: https://martinfowler.com/articles/201701-event-driven.html
  16. Why are we throwing away our events in our applications?

  17. Accountants and banks have known this for centuries

  18. update accounts set balance = balance + 100 where id

    = ‘ryans-account’ update accounts set balance = balance - 100 where id = ‘someone-elses-account’
  19. }, { event: ‘transaction-initiated’, from: ‘someone-elses-account’, to: ‘ryans-account’, amount: 100,

    timestamp: ‘2017-12-13T09:30:00’ }, {
  20. Databases also know this: transaction logs

  21. Apache Kafka: “distributed commit log”

  22. So what makes this a good idea?

  23. “You can’t have your cake and eat it”

  24. Build a better product Provide a natural audit log Fix

    state, not just bugs Decouple services De-risk launches benefits
  25. Item Quantity Pixel 2 1

  26. Event Parameters Added Item To Cart Item: iPhone X, Quantity:

    1 Changed Item Quantity Item: iPhone X, Quantity: 2 Removed Item From Cart Item: iPhone X Added Item To Cart Item: Pixel 2, Quantity: 2 Changed Item Quantity Item: Pixel 2, Quantity: 1
  27. Item Quantity Pixel 2 1 iPhone X 2

  28. Type Parameters Added Item To Cart Item: iPhone X, Quantity:

    1 Changed Item Quantity Item: iPhone X, Quantity: 2 Removed Item From Cart Item: iPhone X Added Item To Cart Item: Pixel 2, Quantity: 2 Changed Item Quantity Item: Pixel 2, Quantity: 1 Event Parameters Added Item To Cart Item: iPhone X, Quantity: 1 Changed Item Quantity Item: iPhone X, Quantity: 2 Removed Item From Cart Item: iPhone X Added Item To Cart Item: Pixel 2, Quantity: 2 Changed Item Quantity Item: Pixel 2, Quantity: 1
  29. Item Quantity Pixel 2 1 iPhone X 2

  30. Item Quantity Pixel 2 1 ✅

  31. ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫

  32. ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫

  33. ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ ⚫

  34. “an immutable, time-ordered sequence of events as our primary store

    of state” Chris Castle, @crc
  35. Event-driven architecture isn’t a silver bullet…

  36. downsides ⏲ Asynchronicity & eventual consistency ✏ Corrections need events

    Replaying must factor in external systems Doesn’t play well with CRUD Demand on storage *
  37. Event Streaming

  38. Votes Tweets Tweets Votes Votes Tweets Consumers Tweet Analyser Tweet

    Broadcaster Vote Broadcaster Vote Persister Event Stream
  39. Votes Tweets Tweets, Votes Tweets Consumers Tweet Analyser Message Archiver

    Event Stream
  40. Event-Sourcing & CQRS

  41. Command Query Responsibility Segregation • Commonly paired with event-sourcing •

    The two aren’t mutually exclusive • Sounds complicated, but isn’t… • Reality: your input and output don’t share models or APIs • Great way to collect user intent cqrs
  42. ❗ ❓ alter read Command Query State cqrs

  43. ❗ ❓ generates reads Command Query State Events alter cqrs

  44. REST vs CQRS REST • PATCH /line_items/123 • Updating quantity?

    • Adding a custom logo? CQRS • /update_line_item_quantity • /update_custom_graphic • Specific methods
  45. GraphQL Mutations : CQRS

  46. Why Apache Kafka? Open-sourced by LinkedIn, now 1.0 Fault-tolerant and

    robust Deliverability guarantees ⏰ Chronological ordering of messages ⬇ Pull-based asynchronous processing event stream
  47. Why Kafka on Heroku? ⚡ Instant provisioning Plans from $100/mo

    High-availability as standard ↔ Easy to share between applications Thoughtful developer experience event stream
  48. I know what event-driven architecture means I know what event

    sourcing means I know what CQRS means summary
  49. Thank you Code Europe

  50. Kafka vs RabbitMQ • Kafka is more horizontally-scalable • Kafka

    is better at acting as a buffer • RabbitMQ has per-message acknowledgements • Kafka has message offset acknowledgement • Both have ordering guarantees • Kafka can handle higher message input