$30 off During Our Annual Pro Sale. View Details »

Events, Workflows, Sagas: Complex Flows in Dist...

Events, Workflows, Sagas: Complex Flows in Distributed Applications (2024-11-12 Software Architecture Gathering, Berlin)

In the development of microservices, sooner or later everyone comes across business processes that involve several services, and usually, in DDD terminology, several bounded contexts. Different approaches to implementing such processes lead to fundamentally changed outcomes. With consequences for the complexity, and thus for the operation and expandability of the system. Especially when event-driven architecture is applied, its asynchronous nature often collides with the desire for global control and synchronous responses. This can lead to “chimeras” that combine the “worst of both worlds”.
In this talk, he will look at some principles of event-driven architecture. He’ll explore how they relate to possible approaches to implementing complex processes. Special consideration will be given to boundaries of responsibility, and the relationship of processes and data.
A common pitfall in the event-driven world is simply copying workflows from a monolithic application. This can lead to hastily falling back on orchestration or workflow tools, and possibly to a “distributed monolith”. How can we avoided this? The talk will introduce the idea of “micro-workflows” and explore how challenging and redesigning complex processes can lead to better systems. And finally, we’ll ask the question: Should the Saga pattern be considered harmful?

Lutz Hühnken

November 12, 2024
Tweet

More Decks by Lutz Hühnken

Other Decks in Programming

Transcript

  1. Events, Workflows, Sagas? Complex Workflows in Distributed Applications 1 Mastodon

    @[email protected] LinkedIn https://linkedin.com/in/lutzh Blog https://www.reactivesystems.eu/
  2. @[email protected] Why me? - Built my first message-driven, asynchronous system

    for the Bundesbank in 2002 - Worked on various Event-Driven systems, e.g. for Zalando, ING, ista, Maersk - Currently building a bank !) with Event-Driven Architecture, Microservices, DDD 3
  3. @[email protected] Reminder: Queries, Commands, Events 11 Is.. Query A request

    for information about the current state of one or many objects Command An intention to perform an operation or change a state Event A fact, something that undisputedly happened in the past
  4. @[email protected] Reminder: Queries, Commands, Events 15 Is.. Expected Response Query

    A request for information about the current state of one or many objects The requested information Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”)
  5. @[email protected] Reminder: Queries, Commands, Events 16 Is.. Expected Response Communication

    Pattern Query A request for information about the current state of one or many objects The requested information Request-Response Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Request-Response Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget
  6. @[email protected] Reminder: Queries, Commands, Events 17 Is.. Expected Response Communication

    Pattern Query A request for information about the current state of one or many objects The requested information Request-Response Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Request-Response Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget But how can I know it’ll be picked up and processed?
  7. @[email protected] Reminder: Queries, Commands, Events 20 Is.. Expected Response Communication

    Pattern Query A request for information about the current state of one or many objects The requested information Request-Response Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Request-Response Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget (rely on promises) But surely there’ll still be errors we need to handle?
  8. @[email protected] 26 item, err := reserveItem(itemNumber) if err != nil

    { return nil, errors.New("out of stock") } confirmation, err := getPayment(itemNumber) if err != nil { return nil, errors.New("insufficient funds") } result, err := shipItem(itemNumber)
  9. @[email protected] The unreasonable effectiveness of events: Beyond asynchronous communication, being

    event-driven has a massive effect on the overall workflow and system design. Event-Driven Architecture EDA) is at least as much about the flow as it is about the actual events. 28
  10. @[email protected] Brain Teaser: We could treat a command like an

    event 29 Is.. Expected Response Communication Pattern Command An intention to perform an operation or change a state None (receiver is not allowed to reject) Fire-and-Forget Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget
  11. @[email protected] Maybe, instead of Event-Driven Architecture, we should talk about

    microworkflows? 30 Coming soon: https://microworkflows.org
  12. @[email protected] Events help us to avoid two types of coupling:

    - Temporal Coupling (leading to better scalability and resilience) - Control Coupling (leading to enhanced organizational clarity / domain boundaries) 31
  13. 38

  14. @[email protected] Orchestration vs. Choreography While these are fitting terms to

    describe the concepts, most comparisons are not useful, but based on some folkloristic concerns. Hence I often try to avoid them. 39
  15. @[email protected] Example Heuristic “Complex workflows..ˮ Problem: Taking a workflow as

    given, not as something that is designed and can be changed. Remember 2-phase-commit? 40
  16. @[email protected] Workflows and Orchestration are useful tools. Workflow engines too.

    But not for implementing complex workflows on top of Microservices. That use case shouldnʼt exist. Consider going event-driven instead. Break the workflow down into microworkflows. 42
  17. @[email protected] Orchestration and Choreography can complement each other. Orchestration within

    a bounded context. Within a microworkflow. As an integration tool. Choreography between different bounded contexts / between microworkflows. 43
  18. @[email protected] Event-Driven Architecture • enables scalability, resilience (removes temporal coupling)

    • promotes clear responsibility boundaries (removes control coupling) 44
  19. @[email protected] Event-Driven Architecture Thinking in events is not enough -

    youʼll also need to think in terms of promises and railways. Make sure not to approach it with an imperative mindset. 45
  20. What do you think? Letʼs discuss! Please rate your experience

    ⭐⭐⭐⭐⭐ 46 Mastodon @[email protected] LinkedIn https://linkedin.com/in/lutzh Blog https://www.reactivesystems.eu/
  21. @[email protected] Think Unix Philosophy 1. Make each program do one

    thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". 2. Expect the output of every program to become the input to another, as yet unknown, program. ... 52 $ Cat file3.txt | grep “dwx” | tee file4.txt | wc –l