Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Events, Workflows, Sagas? Keep Your Event-drive...

Lutz Hühnken
June 27, 2023
25

Events, Workflows, Sagas? Keep Your Event-driven Architecture Sane at Developer Week Nuremberg

When dealing with (micro-)services, we often face business processes that span multiple bounded contexts. Different ways of implementing this lead to massively divergent results, varying above all in complexity and manageability. Especially when applying Event-driven Architecture, its asynchronous nature clashes with a desire for global control and synchronous responses, sometimes leading to chimeras that combine the “worst of both worlds”.

This talk will establish some guiding principles, and use them to assess common approaches. It will explore the importance of responsibility boundaries and of bringing the data to the behavior.

A common mistake in the event-driven world is to replicate monolith workflows, resulting in an excessive need for orchestration or workflow tooling, or - in the worst case - in a “distributed monolith”. How can this be avoided? The talk will introduce the concept of micro-workflows and show, how challenging and re-designing your processes will lead to better systems. And lastly, we will ask ourselves: Should the saga pattern be considered harmful?

Lutz Hühnken

June 27, 2023
Tweet

Transcript

  1. @[email protected] In 2008, a large German publishing company announced that

    they’ll switch their IT to Apple Computers. Somewhere down in the press statement, it read “the desktop computers will run Windows XP or Vista”. 4
  2. @[email protected] Why me? - Build my first message-driven, asynchronous system

    for the Bundesbank starting 2001 - Worked on various event-driven systems, e.g. for Zalando, ING, ista, Maersk… - Currently building a bank with event-driven architecture! 5
  3. @[email protected] So what have we looked at so far? Implementing

    a workflow by: - Making one service the coordinator, or - Using a workflow engine for coordination, and - Implementing a saga (can be done with either of the above) What do all approaches have in common? 10
  4. @[email protected] - Introduce a complex piece of software - makes

    it harder to run, change, test - Introduce a split responsibility - Are not event-driven 11
  5. @[email protected] Technical complexity: - Must be able to resume after

    crash - Must be able to handle timeouts - Must be distributed to avoid single point of failure - Needs distributed coordination (stateful) - (Some of) your business logic is now in a YAML or XML file 12
  6. @[email protected] Organizational complexity: - Business logic is split - Error

    handling is split, leading to reduced clarity of responsibility if something goes wrong 13
  7. @[email protected] Reminder: Events vs. Commands 14 Is.. Command An intention

    to perform an operation or change a state Event A fact, something that undisputedly happened in the past
  8. @[email protected] Reminder: Events vs. Commands 15 Is.. Expected Response Communication

    Pattern Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message Request-Response Event A fact, something that undisputedly happened in the past None Fire-And-Forget
  9. @[email protected] Q: So how to implement complex workflows over multiple

    microservices? A: Don’t. Instead, break up the workflow. To match actual event-driven architecture. Monolith -> Microservices Workflow -> Microworkflows 19
  10. @[email protected] Q: But what about timeouts? A: - flaky connections,

    retries - within service - processing times - SLA - service needs to scale, e.g. based on consumer lag - overall process - observer 24
  11. @[email protected] Q: But what about error handling? A: Errors must

    be handled within a service. The service gives a promise. This is a fundamental decoupling in event-driven architecture. 25
  12. @[email protected] Q: Surely, this only works for unrealistic, simplistic examples?

    A: No, it’s just much harder to fit it on slides in a non-confusing way 󰤇. 26
  13. @[email protected] Think Unix Philosophy 1. Make each program do one

    thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". 2. Expect the output of every program to become the input to another, as yet unknown, program. ... 32 $ Cat file3.txt | grep “dwx” | tee file4.txt | wc –l
  14. @[email protected] Enjoy the (load) testing 36 While you’re freed from

    having e.g. the workflow engine in your end-to-end test, there are other challenges. Also: Testing a microworkflow is still an integration test. But it’s nice to be able to test e.g. throughput in a very isolated way.
  15. 37

  16. @[email protected] Orchestration vs. Choreography While these terms might be fitting

    to describe the concepts, most comparisons are not useful, but based on some folkloristic concerns. Hence I tried to avoid them. 38
  17. @[email protected] Example Heuristic “Complex workflows..” Problem: Taking a workflow as

    given, not as something that is designed and can be changed. 39
  18. @[email protected] Workflows are useful tools. Workflow engines too, probably. But

    not for implementing complex workflows on top of Microservices. That use case shouldn’t exist. Consider going event-driven instead. Break the workflow down into microworkflows. 42
  19. @[email protected] People are quick to jump to sagas and workflow

    engines! - Is it someone who’s familiar with it, but not with EDA, and is scared to leave their comfort zone? - Is it someone who wants to sell you something? 43
  20. @[email protected] Summary: Workflows and Sagas are at odds with event

    driven architecture. If you really want to get the benefits of event-driven architecture, go all-in, don’t impose an orchestrator on top of it. Keep your event-driven architecture sane. 44
  21. Ask your question now! Follow me on Mastodon @[email protected] Follow

    me on LinkedIn https://linkedin.com/in/lutzh Read my blog https://www.reactivesystems.eu/ 45 Also: I’ll be around until tomorrow 15:00hrs. Ask me anything, don’t be shy.