20221101 Events, Workflows, Sagas KanDDDinsky

When dealing with (micro-) services, we often face business processes that span multiple bounded contexts. Different ways of implementing this lead to massively divergent results, varying above all in complexity and manageability. Especially when applying Event-driven Architecture, its asynchronous nature clashes with a desire for global control and synchronous responses, sometimes leading to chimeras that combine the “worst of both worlds”.
This talk will establish some guiding principles, and use them to assess common approaches. It will explore the importance of responsibility boundaries, and of bringing the data to the behavior.
A common mistake in the event-driven world is to replicate monolith workflows, resulting in an excessive need for orchestration or workflow tooling, or (in the worst case) a “distributed monolith”. How can this be avoided? The talk will introduce the concept of microworkflows, and show how challenging and re-designing your processes will lead to better systems. And lastly, we will ask ourselves: Should the saga pattern be considered harmful?

Transcript

1. Events, Workflows, Sagas?
   Keep Your Event-driven
   Architecture Sane.
   1
   

   View Slide

2. In 2008, a large German publishing company
   announced that they’ll switch their IT to Apple
   Computers.
   2
   

   View Slide

3. In 2008, a large German publishing company
   announced that they’ll switch their IT to Apple
   Computers.
   Somewhere down in the press statement, it read
   “the desktop computers will run Windows XP or
   Vista”.
   3
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. So what have we looked at so far?
   Implementing a workflow by:
   - Making one service the coordinator, or
   - Using a workflow engine for coordination, and
   - Implementing a saga (can be done with either of
   the above)
   What do all approaches have in common?
   8
   

   View Slide

9. - Introduce a complex piece of software
   - makes it harder to run, change, test
   - Introduce a split responsibility
   - Are not event-driven
   9
   

   View Slide

10. Technical complexity:
    - Must be able to resume after crash
    - Must be able to handle timeouts
    - Must be distributed to avoid single point of failure
    - Needs distributed coordination (stateful)
    - (Some of) your business logic is now in a YAML or
    XML file
    10
    

    View Slide

11. Organizational complexity:
    - Business logic is split
    - Error handling is split, leading to reduced clarity of
    responsibility if something goes wrong
    11
    

    View Slide

12. View Slide

13. View Slide

14. View Slide

15. Q: So how to implement complex workflows over
    multiple microservices?
    A: Don’t. Instead, break up the workflow. To match
    actual event driven architecture.
    Monolith -> Microservices
    Workflow -> Microworkflows
    15
    

    View Slide

16. View Slide

17. View Slide

18. Q: But what about visibility of the end-to-end
    process?
    A: Don’t mix control and observability!
    18
    

    View Slide

19. View Slide

20. But what about timeouts?
    - flaky connections, retries - within service
    - processing times - SLA - service needs to scale,
    e.g. based on consumer lag
    - overall process - observer
    20
    

    View Slide

21. But what about error handling?
    Errors must be handled within a service. The
    service gives a promise.
    This is a fundamental decoupling in event-driven
    architecture.
    21
    

    View Slide

22. Surely, this only works for unrealistic, simplistic
    examples.
    No, it’s just much harder to fit it on slides in a
    non-confusing way.
    22
    

    View Slide

23. Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 23
    

    View Slide

24. Image: https://blog.codecentric.de/wer-microservices-richtig-macht-braucht-keine-workflow-engine-und-kein-bpmn 24
    

    View Slide

25. Technicalities are not the problem, it’s a mindset
    issue.
    Think Promises.
    Enjoy the organizational clarity that it produces.
    25
    

    View Slide

26. Think Unix Philosophy
    1. Make each program do one thing well. To do a new job, build afresh
    rather than complicate old programs by adding new "features".
    2. Expect the output of every program to become the input to another, as
    yet unknown, program. ...
    26
    

    View Slide

27. Think reducing complexity, not managing it.
    https://twitter.com/bitfield/status/1411731604335214592?ref_src=twsrc%5Etfw
    27
    

    View Slide

28. Think Agile. And then putting SAFE on top of it.
    28
    

    View Slide

29. View Slide

30. Orchestration vs. Choreography
    While these terms might be fitting to describe the
    concepts, most comparisons are not useful, but
    based on some folkloristic concerns (e.g. taking a
    workflow as given, not as something that is
    designed and can be changed).
    Hence I tried to avoid them.
    30
    

    View Slide

31. Workflows are useful tools. Workflow engines too,
    probably.
    But not for implementing complex workflows on top
    of Microservices. That use case shouldn’t exist.
    Consider going event-driven instead.
    Break the workflow down into microworkflows.
    31
    

    View Slide

32. People are quick to jump to sagas and workflow
    engines!
    - Is it someone who’s familiar with it, but not with
    EDA, and is scared to leave their comfort zone?
    - Is it someone who wants to sell you something?
    - Is it someone who really understands EDA, and
    what you want to achieve, and, after considering
    all trade-offs, gives genuine, well-founded
    advice?
    32
    

    View Slide

33. Summary:
    Workflows and Sagas are at odds with event
    driven architecture.
    If you really want to get the benefits of
    event-driven architecture, go all-in, don’t impose
    an orchestrator on top of it.
    Keep your event-driven architecture sane.
    33
    

    View Slide

34. Thank you!
    Mastodon https://social.tchncs.de/@lutzhuehnken
    Twitter https://twitter.com/lutzhuehnken
    Blog https://www.reactivesystems.eu/
    34
    

    View Slide