Tweet
Tweet

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Events, Workflows, Sagas? Keep Your Event-driven Architecture Sane. 2

Slide 3

Slide 3 text

@[email protected] In 2008, a large German publishing company announced that they’ll switch their IT to Apple Computers. 3

Slide 4

Slide 4 text

@[email protected] In 2008, a large German publishing company announced that they’ll switch their IT to Apple Computers. Somewhere down in the press statement, it read “the desktop computers will run Windows XP or Vista”. 4

Slide 5

Slide 5 text

@[email protected] Why me? - Build my first message-driven, asynchronous system for the Bundesbank starting 2001 - Worked on various event-driven systems, e.g. for Zalando, ING, ista, Maersk… - Currently building a bank with event-driven architecture! 5

Slide 6

Slide 6 text

@[email protected] 6

Slide 7

Slide 7 text

@[email protected] 7

Slide 8

Slide 8 text

@[email protected] 8

Slide 9

Slide 9 text

@[email protected] 9

Slide 10

Slide 10 text

@[email protected] So what have we looked at so far? Implementing a workflow by: - Making one service the coordinator, or - Using a workflow engine for coordination, and - Implementing a saga (can be done with either of the above) What do all approaches have in common? 10

Slide 11

Slide 11 text

@[email protected] - Introduce a complex piece of software - makes it harder to run, change, test - Introduce a split responsibility - Are not event-driven 11

Slide 12

Slide 12 text

@[email protected] Technical complexity: - Must be able to resume after crash - Must be able to handle timeouts - Must be distributed to avoid single point of failure - Needs distributed coordination (stateful) - (Some of) your business logic is now in a YAML or XML file 12

Slide 13

Slide 13 text

@[email protected] Organizational complexity: - Business logic is split - Error handling is split, leading to reduced clarity of responsibility if something goes wrong 13

Slide 14

Slide 14 text

@[email protected] Reminder: Events vs. Commands 14 Is.. Command An intention to perform an operation or change a state Event A fact, something that undisputedly happened in the past

Slide 15

Slide 15 text

@[email protected] Reminder: Events vs. Commands 15 Is.. Expected Response Communication Pattern Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message Request-Response Event A fact, something that undisputedly happened in the past None Fire-And-Forget

Slide 16

Slide 16 text

@[email protected] 16

Slide 17

Slide 17 text

@[email protected] 17

Slide 18

Slide 18 text

@[email protected] 18

Slide 19

Slide 19 text

@[email protected] Q: So how to implement complex workflows over multiple microservices? A: Don’t. Instead, break up the workflow. To match actual event-driven architecture. Monolith -> Microservices Workflow -> Microworkflows 19

Slide 20

Slide 20 text

@[email protected] 20

Slide 21

Slide 21 text

@[email protected] 21

Slide 22

Slide 22 text

@[email protected] Q: But what about visibility of the end-to-end process? A: Don’t mix control and observability! 22

Slide 23

Slide 23 text

@[email protected] 23

Slide 24

Slide 24 text

@[email protected] Q: But what about timeouts? A: - flaky connections, retries - within service - processing times - SLA - service needs to scale, e.g. based on consumer lag - overall process - observer 24

Slide 25

Slide 25 text

@[email protected] Q: But what about error handling? A: Errors must be handled within a service. The service gives a promise. This is a fundamental decoupling in event-driven architecture. 25

Slide 26

Slide 26 text

@[email protected] Q: Surely, this only works for unrealistic, simplistic examples? A: No, it’s just much harder to fit it on slides in a non-confusing way 󰤇. 26

Slide 27

Slide 27 text

@[email protected] Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 27

Slide 28

Slide 28 text

@[email protected] Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 28

Slide 29

Slide 29 text

@[email protected] Challenge accepted! Send your workflow to: [email protected] 29

Slide 30

Slide 30 text

@[email protected] Technology is not the problem, it’s a mindset issue. Think Promises. 30

Slide 31

Slide 31 text

@[email protected] Some ideas seem outlandish, but they’re nothing new. Prior art: - Unix pipes - Actors - Stream Processing 31

Slide 32

Slide 32 text

@[email protected] Think Unix Philosophy 1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". 2. Expect the output of every program to become the input to another, as yet unknown, program. ... 32 $ Cat file3.txt | grep “dwx” | tee file4.txt | wc –l

Slide 33

Slide 33 text

@[email protected] Think Actors 33 Supervisor Actor Client

Slide 34

Slide 34 text

@[email protected] Think Stream Processing 34

Slide 35

Slide 35 text

@[email protected] Enjoy the organizational clarity 35 ❌ ❌

Slide 36

Slide 36 text

@[email protected] Enjoy the (load) testing 36 While you’re freed from having e.g. the workflow engine in your end-to-end test, there are other challenges. Also: Testing a microworkflow is still an integration test. But it’s nice to be able to test e.g. throughput in a very isolated way.

Slide 37

Slide 37 text

37

Slide 38

Slide 38 text

@[email protected] Orchestration vs. Choreography While these terms might be fitting to describe the concepts, most comparisons are not useful, but based on some folkloristic concerns. Hence I tried to avoid them. 38

Slide 39

Slide 39 text

@[email protected] Example Heuristic “Complex workflows..” Problem: Taking a workflow as given, not as something that is designed and can be changed. 39

Slide 40

Slide 40 text

@[email protected] Think reducing complexity, not managing it. https://twitter.com/bitfield/status/1411731604335214592?ref_src=twsrc%5Etfw 40

Slide 41

Slide 41 text

@[email protected] Think Agile. And then putting SAFE on top of it. 41

Slide 42

Slide 42 text

@[email protected] Workflows are useful tools. Workflow engines too, probably. But not for implementing complex workflows on top of Microservices. That use case shouldn’t exist. Consider going event-driven instead. Break the workflow down into microworkflows. 42

Slide 43

Slide 43 text

@[email protected] People are quick to jump to sagas and workflow engines! - Is it someone who’s familiar with it, but not with EDA, and is scared to leave their comfort zone? - Is it someone who wants to sell you something? 43

Slide 44

Slide 44 text

@[email protected] Summary: Workflows and Sagas are at odds with event driven architecture. If you really want to get the benefits of event-driven architecture, go all-in, don’t impose an orchestrator on top of it. Keep your event-driven architecture sane. 44

Slide 45

Slide 45 text

Ask your question now! Follow me on Mastodon @[email protected] Follow me on LinkedIn https://linkedin.com/in/lutzh Read my blog https://www.reactivesystems.eu/ 45 Also: I’ll be around until tomorrow 15:00hrs. Ask me anything, don’t be shy.