$30 off During Our Annual Pro Sale. View Details »

20221101 Events, Workflows, Sagas KanDDDinsky

Lutz Hühnken
November 01, 2022

20221101 Events, Workflows, Sagas KanDDDinsky

When dealing with (micro-) services, we often face business processes that span multiple bounded contexts. Different ways of implementing this lead to massively divergent results, varying above all in complexity and manageability. Especially when applying Event-driven Architecture, its asynchronous nature clashes with a desire for global control and synchronous responses, sometimes leading to chimeras that combine the “worst of both worlds”.
This talk will establish some guiding principles, and use them to assess common approaches. It will explore the importance of responsibility boundaries, and of bringing the data to the behavior.
A common mistake in the event-driven world is to replicate monolith workflows, resulting in an excessive need for orchestration or workflow tooling, or (in the worst case) a “distributed monolith”. How can this be avoided? The talk will introduce the concept of microworkflows, and show how challenging and re-designing your processes will lead to better systems. And lastly, we will ask ourselves: Should the saga pattern be considered harmful?

Lutz Hühnken

November 01, 2022
Tweet

More Decks by Lutz Hühnken

Other Decks in Technology

Transcript

  1. Events, Workflows, Sagas? Keep Your Event-driven Architecture Sane. 1

  2. In 2008, a large German publishing company announced that they’ll

    switch their IT to Apple Computers. 2
  3. In 2008, a large German publishing company announced that they’ll

    switch their IT to Apple Computers. Somewhere down in the press statement, it read “the desktop computers will run Windows XP or Vista”. 3
  4. 4

  5. 5

  6. 6

  7. 7

  8. So what have we looked at so far? Implementing a

    workflow by: - Making one service the coordinator, or - Using a workflow engine for coordination, and - Implementing a saga (can be done with either of the above) What do all approaches have in common? 8
  9. - Introduce a complex piece of software - makes it

    harder to run, change, test - Introduce a split responsibility - Are not event-driven 9
  10. Technical complexity: - Must be able to resume after crash

    - Must be able to handle timeouts - Must be distributed to avoid single point of failure - Needs distributed coordination (stateful) - (Some of) your business logic is now in a YAML or XML file 10
  11. Organizational complexity: - Business logic is split - Error handling

    is split, leading to reduced clarity of responsibility if something goes wrong 11
  12. 12

  13. 13

  14. 14

  15. Q: So how to implement complex workflows over multiple microservices?

    A: Don’t. Instead, break up the workflow. To match actual event driven architecture. Monolith -> Microservices Workflow -> Microworkflows 15
  16. 16

  17. 17

  18. Q: But what about visibility of the end-to-end process? A:

    Don’t mix control and observability! 18
  19. 19

  20. But what about timeouts? - flaky connections, retries - within

    service - processing times - SLA - service needs to scale, e.g. based on consumer lag - overall process - observer 20
  21. But what about error handling? Errors must be handled within

    a service. The service gives a promise. This is a fundamental decoupling in event-driven architecture. 21
  22. Surely, this only works for unrealistic, simplistic examples. No, it’s

    just much harder to fit it on slides in a non-confusing way. 22
  23. Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 23

  24. Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 24

  25. Technicalities are not the problem, it’s a mindset issue. Think

    Promises. Enjoy the organizational clarity that it produces. 25
  26. Think Unix Philosophy 1. Make each program do one thing

    well. To do a new job, build afresh rather than complicate old programs by adding new "features". 2. Expect the output of every program to become the input to another, as yet unknown, program. ... 26
  27. Think reducing complexity, not managing it. https://twitter.com/bitfield/status/1411731604335214592?ref_src=twsrc%5Etfw 27

  28. Think Agile. And then putting SAFE on top of it.

    28
  29. 29

  30. Orchestration vs. Choreography While these terms might be fitting to

    describe the concepts, most comparisons are not useful, but based on some folkloristic concerns (e.g. taking a workflow as given, not as something that is designed and can be changed). Hence I tried to avoid them. 30
  31. Workflows are useful tools. Workflow engines too, probably. But not

    for implementing complex workflows on top of Microservices. That use case shouldn’t exist. Consider going event-driven instead. Break the workflow down into microworkflows. 31
  32. People are quick to jump to sagas and workflow engines!

    - Is it someone who’s familiar with it, but not with EDA, and is scared to leave their comfort zone? - Is it someone who wants to sell you something? - Is it someone who really understands EDA, and what you want to achieve, and, after considering all trade-offs, gives genuine, well-founded advice? 32
  33. Summary: Workflows and Sagas are at odds with event driven

    architecture. If you really want to get the benefits of event-driven architecture, go all-in, don’t impose an orchestrator on top of it. Keep your event-driven architecture sane. 33
  34. Thank you! Mastodon https://social.tchncs.de/@lutzhuehnken Twitter https://twitter.com/lutzhuehnken Blog https://www.reactivesystems.eu/ 34