Tweet
Tweet

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Events, Workflows, Sagas? Keep Your Event-driven Architecture Sane. 2

Slide 3

Slide 3 text

@lutzhuehnken@mastodon.social In 2008, a large German publishing company announced that they’ll switch their IT to Apple Computers. 3

Slide 4

Slide 4 text

@lutzhuehnken@mastodon.social In 2008, a large German publishing company announced that they’ll switch their IT to Apple Computers. Somewhere down in the press statement, it read “the desktop computers will run Windows XP or Vista”. 4

Slide 5

Slide 5 text

@lutzhuehnken@mastodon.social Why me? - Build my first message-driven, asynchronous system for the Bundesbank starting 2001 - Worked on various event-driven systems, e.g. for Zalando, ING, ista, Maersk… - Currently building a bank with event-driven architecture! 5

Slide 6

Slide 6 text

@lutzhuehnken@mastodon.social 6

Slide 7

Slide 7 text

@lutzhuehnken@mastodon.social 7

Slide 8

Slide 8 text

@lutzhuehnken@mastodon.social 8

Slide 9

Slide 9 text

@lutzhuehnken@mastodon.social 9

Slide 10

Slide 10 text

@lutzhuehnken@mastodon.social So what have we looked at so far? Implementing a workflow by: - Making one service the coordinator, or - Using a workflow engine for coordination, and - Implementing a saga (can be done with either of the above) What do all approaches have in common? 10

Slide 11

Slide 11 text

@lutzhuehnken@mastodon.social - Introduce a complex piece of software - makes it harder to run, change, test - Introduce a split responsibility - Are not event-driven 11

Slide 12

Slide 12 text

@lutzhuehnken@mastodon.social Technical complexity: - Must be able to resume after crash - Must be able to handle timeouts - Must be distributed to avoid single point of failure - Needs distributed coordination (stateful) - (Some of) your business logic is now in a YAML or XML file 12

Slide 13

Slide 13 text

@lutzhuehnken@mastodon.social Organizational complexity: - Business logic is split - Error handling is split, leading to reduced clarity of responsibility if something goes wrong 13

Slide 14

Slide 14 text

@lutzhuehnken@mastodon.social Reminder: Events vs. Commands 14 Is.. Command An intention to perform an operation or change a state Event A fact, something that undisputedly happened in the past

Slide 15

Slide 15 text

@lutzhuehnken@mastodon.social Reminder: Events vs. Commands 15 Is.. Expected Response Communication Pattern Command An intention to perform an operation or change a state A confirmation that the command has been executed, or an error message Request-Response Event A fact, something that undisputedly happened in the past None Fire-And-Forget

Slide 16

Slide 16 text

@lutzhuehnken@mastodon.social 16

Slide 17

Slide 17 text

@lutzhuehnken@mastodon.social 17

Slide 18

Slide 18 text

@lutzhuehnken@mastodon.social 18

Slide 19

Slide 19 text

@lutzhuehnken@mastodon.social Q: So how to implement complex workflows over multiple microservices? A: Don’t. Instead, break up the workflow. To match actual event-driven architecture. Monolith -> Microservices Workflow -> Microworkflows 19

Slide 20

Slide 20 text

@lutzhuehnken@mastodon.social 20

Slide 21

Slide 21 text

@lutzhuehnken@mastodon.social 21

Slide 22

Slide 22 text

@lutzhuehnken@mastodon.social Q: But what about visibility of the end-to-end process? A: Don’t mix control and observability! 22

Slide 23

Slide 23 text

@lutzhuehnken@mastodon.social 23

Slide 24

Slide 24 text

@lutzhuehnken@mastodon.social Q: But what about timeouts? A: - flaky connections, retries - within service - processing times - SLA - service needs to scale, e.g. based on consumer lag - overall process - observer 24

Slide 25

Slide 25 text

@lutzhuehnken@mastodon.social Q: But what about error handling? A: Errors must be handled within a service. The service gives a promise. This is a fundamental decoupling in event-driven architecture. 25

Slide 26

Slide 26 text

@lutzhuehnken@mastodon.social Q: Surely, this only works for unrealistic, simplistic examples? A: No, it’s just much harder to fit it on slides in a non-confusing way 󰤇. 26

Slide 27

Slide 27 text

@lutzhuehnken@mastodon.social Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 27

Slide 28

Slide 28 text

@lutzhuehnken@mastodon.social Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 28

Slide 29

Slide 29 text

@lutzhuehnken@mastodon.social Challenge accepted! Send your workflow to: challenge@microworkflows.org 29

Slide 30

Slide 30 text

@lutzhuehnken@mastodon.social Technology is not the problem, it’s a mindset issue. Think Promises. 30

Slide 31

Slide 31 text

@lutzhuehnken@mastodon.social Some ideas seem outlandish, but they’re nothing new. Prior art: - Unix pipes - Actors - Stream Processing 31

Slide 32

Slide 32 text

@lutzhuehnken@mastodon.social Think Unix Philosophy 1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". 2. Expect the output of every program to become the input to another, as yet unknown, program. ... 32 $ Cat file3.txt | grep “dwx” | tee file4.txt | wc –l

Slide 33

Slide 33 text

@lutzhuehnken@mastodon.social Think Actors 33 Supervisor Actor Client

Slide 34

Slide 34 text

@lutzhuehnken@mastodon.social Think Stream Processing 34

Slide 35

Slide 35 text

@lutzhuehnken@mastodon.social Enjoy the organizational clarity 35 ❌ ❌

Slide 36

Slide 36 text

@lutzhuehnken@mastodon.social Enjoy the (load) testing 36 While you’re freed from having e.g. the workflow engine in your end-to-end test, there are other challenges. Also: Testing a microworkflow is still an integration test. But it’s nice to be able to test e.g. throughput in a very isolated way.

Slide 37

Slide 37 text

37

Slide 38

Slide 38 text

@lutzhuehnken@mastodon.social Orchestration vs. Choreography While these terms might be fitting to describe the concepts, most comparisons are not useful, but based on some folkloristic concerns. Hence I tried to avoid them. 38

Slide 39

Slide 39 text

@lutzhuehnken@mastodon.social Example Heuristic “Complex workflows..” Problem: Taking a workflow as given, not as something that is designed and can be changed. 39

Slide 40

Slide 40 text

@lutzhuehnken@mastodon.social Think reducing complexity, not managing it. https://twitter.com/bitfield/status/1411731604335214592?ref_src=twsrc%5Etfw 40

Slide 41

Slide 41 text

@lutzhuehnken@mastodon.social Think Agile. And then putting SAFE on top of it. 41

Slide 42

Slide 42 text

@lutzhuehnken@mastodon.social Workflows are useful tools. Workflow engines too, probably. But not for implementing complex workflows on top of Microservices. That use case shouldn’t exist. Consider going event-driven instead. Break the workflow down into microworkflows. 42

Slide 43

Slide 43 text

@lutzhuehnken@mastodon.social People are quick to jump to sagas and workflow engines! - Is it someone who’s familiar with it, but not with EDA, and is scared to leave their comfort zone? - Is it someone who wants to sell you something? 43

Slide 44

Slide 44 text

@lutzhuehnken@mastodon.social Summary: Workflows and Sagas are at odds with event driven architecture. If you really want to get the benefits of event-driven architecture, go all-in, don’t impose an orchestrator on top of it. Keep your event-driven architecture sane. 44

Slide 45

Slide 45 text

Ask your question now! Follow me on Mastodon @lutzhuehnken@mastodon.social Follow me on LinkedIn https://linkedin.com/in/lutzh Read my blog https://www.reactivesystems.eu/ 45 Also: I’ll be around until tomorrow 15:00hrs. Ask me anything, don’t be shy.