Events, Workflows, Sagas? Complex Workflows in Distributed Applications 1 Mastodon @[email protected] LinkedIn https://linkedin.com/in/lutzh Blog https://www.reactivesystems.eu/ 

No content

@[email protected] Why me? - Built my first message-driven, asynchronous system for the Bundesbank in 2002 - Worked on various Event-Driven systems, e.g. for Zalando, ING, ista, Maersk - Currently building a bank !) with Event-Driven Architecture, Microservices, DDD 3 

@[email protected] 4 4 

@[email protected] 5 5 

@[email protected] 6 6 

@[email protected] 7 7 

@[email protected] 8 8 

@[email protected] 9 

@[email protected] 10 10 Event-Driven Architecture Google Trend 

@[email protected] Reminder: Queries, Commands, Events 11 Is.. Query A request for information about the current state of one or many objects Command An intention to perform an operation or change a state Event A fact, something that undisputedly happened in the past 

@[email protected] 12 12 

@[email protected] 13 

@[email protected] 14 

@[email protected] Reminder: Queries, Commands, Events 15 Is.. Expected Response Query A request for information about the current state of one or many objects The requested information Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) 

@[email protected] Reminder: Queries, Commands, Events 16 Is.. Expected Response Communication Pattern Query A request for information about the current state of one or many objects The requested information Request-Response Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Request-Response Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget 

@[email protected] Reminder: Queries, Commands, Events 17 Is.. Expected Response Communication Pattern Query A request for information about the current state of one or many objects The requested information Request-Response Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Request-Response Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget But how can I know it’ll be picked up and processed? 

@[email protected] 18 Synchronous Command Asynchronous Command Asynchronous Event 

@[email protected] Events are “fire-and-forgetˮ... … but based on previous agreements (promises) 19 

@[email protected] Reminder: Queries, Commands, Events 20 Is.. Expected Response Communication Pattern Query A request for information about the current state of one or many objects The requested information Request-Response Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message if the command failed Request-Response Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget (rely on promises) But surely there’ll still be errors we need to handle? 

@[email protected] 21 

@[email protected] 22 

@[email protected] Brain Teaser Functional Programming 23 

@[email protected] 24 

@[email protected] 25 

@[email protected] 26 item, err := reserveItem(itemNumber) if err != nil { return nil, errors.New("out of stock") } confirmation, err := getPayment(itemNumber) if err != nil { return nil, errors.New("insufficient funds") } result, err := shipItem(itemNumber) 

@[email protected] 27 reserveItem(itemNumber).flatMap(getPayment).flatMap(shipItem) 

@[email protected] The unreasonable effectiveness of events: Beyond asynchronous communication, being event-driven has a massive effect on the overall workflow and system design. Event-Driven Architecture EDA) is at least as much about the flow as it is about the actual events. 28 

@[email protected] Brain Teaser: We could treat a command like an event 29 Is.. Expected Response Communication Pattern Command An intention to perform an operation or change a state None (receiver is not allowed to reject) Fire-and-Forget Event A fact, something that undisputedly happened in the past None (events are facts, they can’t “fail”) Fire-and-Forget 

@[email protected] Maybe, instead of Event-Driven Architecture, we should talk about microworkflows? 30 Coming soon: https://microworkflows.org 

@[email protected] Events help us to avoid two types of coupling: - Temporal Coupling (leading to better scalability and resilience) - Control Coupling (leading to enhanced organizational clarity / domain boundaries) 31 

@[email protected] 32 

@[email protected] Workflow Engines, Durable Execution, Sagas 33 

@[email protected] 34 

@[email protected] The Saga Pattern 35 

@[email protected] 36 How it started How itʼs going 

@[email protected] 37 

38 

@[email protected] Orchestration vs. Choreography While these are fitting terms to describe the concepts, most comparisons are not useful, but based on some folkloristic concerns. Hence I often try to avoid them. 39 

@[email protected] Example Heuristic “Complex workflows..ˮ Problem: Taking a workflow as given, not as something that is designed and can be changed. Remember 2-phase-commit? 40 

@[email protected] Think reducing complexity, not managing it. https://twitter.com/bitfield/status/1411731604335214592 41 

@[email protected] Workflows and Orchestration are useful tools. Workflow engines too. But not for implementing complex workflows on top of Microservices. That use case shouldnʼt exist. Consider going event-driven instead. Break the workflow down into microworkflows. 42 

@[email protected] Orchestration and Choreography can complement each other. Orchestration within a bounded context. Within a microworkflow. As an integration tool. Choreography between different bounded contexts / between microworkflows. 43 

@[email protected] Event-Driven Architecture ● enables scalability, resilience (removes temporal coupling) ● promotes clear responsibility boundaries (removes control coupling) 44 

@[email protected] Event-Driven Architecture Thinking in events is not enough - youʼll also need to think in terms of promises and railways. Make sure not to approach it with an imperative mindset. 45 

What do you think? Letʼs discuss! Please rate your experience ⭐⭐⭐⭐⭐ 46 Mastodon @[email protected] LinkedIn https://linkedin.com/in/lutzh Blog https://www.reactivesystems.eu/ 

@[email protected] Important Patterns 47 

@[email protected] 48 Separate Observing from Control 

@[email protected] 49 Bring the data to the process 

@[email protected] 50 

@[email protected] More Brain Teasers 51 

@[email protected] Think Unix Philosophy 1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". 2. Expect the output of every program to become the input to another, as yet unknown, program. ... 52 $ Cat file3.txt | grep “dwx” | tee file4.txt | wc –l 

@[email protected] Think Stream Processing 53 

@[email protected] Think Actors 54 Supervisor Actor Client