Break your event chains! Complex event flows in distributed systems. @berndruecker @martinschimak

Berlin, Germany bernd.ruecker@camunda.com @berndruecker Bernd Ruecker Co-founder and Developer Advocate of Camunda

Vienna, Austria martin.schimak@plexiti.com @martinschimak Martin Schimak Into and onto de|coding domain language with Plexiti

Distributed systems

No content

Let‘s have a look what we can learn from DDD

Example process / Event Storming! Goods shipped Order placed Goods fetched Payment received Events are a concept easy to grasp for everybody in the room.

Events are … Payment received Payment Strong consistency It already happened. Events let us look into the past.

Events help to decouple services. Order summary Payment received Payment UI raises projected onto Payment Service Order Service

Events help with cross-cutting concerns. Payment received Inform customer listened to by Payment Service Notification Service

What about purely event-driven architectures? Order placed Payment Service Payment received Inventory Service Goods fetched Shipment Service Goods shipped When relying only on events, „intent“ is always formed inside the event consumer.

Autonomous services can make their own decisions. To change the order of two services, we touch 4 sticky notes… but wait, even three services! Shipment Service Goods shipped Order placed Inventory Service Goods fetched Payment Service Payment received

Our policies decide to act on facts – with commands Domain Event Policy Command listened to by triggers

Who owns the policy in a pure event choreography? Payment received Start Fetching Policy Fetch goods listened to by triggers Payment Service Inventory Service

Neither of both truly „owns“ that policy? Payment received Fetch goods listened to by triggers Payment Service Inventory Service Start Fetching Policy

Extracting the policy into a mediator service Payment received Fetch goods listened to by triggers Payment Service Inventory Service Order Service Start Fetching Policy

The order as the company‘s core business Place order Order placed Request payment Order Service Payment received Fetch goods Goods fetched Ship goods Goods shipped “When the order is placed…” “When the payment is received…” “When the goods are fetched…” “Then we are done!” Mark order as fulfilled Order fulfilled Payment Service Inventory Service Shipment Service

Double check: what happens when we charge later? Place order Order placed Request payment Order Service Payment received Fetch goods Goods fetched Ship goods Goods shipped “When the order is placed…” “When the payment is received…” “When the goods are fetched…” “Then we are done!” Mark order as fulfilled Order fulfilled Payment Service Inventory Service Shipment Service “When the order is placed…” “When the payment is received…” “When the goods are fetched…”

Multiple services need to collaborate Payment Shipping Order Inventory

Multiple services need to collaborate Payment Shipping Order Inventory https://www.infoworld.com/article/3254777/application-development/ 3-common-pitfalls-of-microservices-integrationand-how-to-avoid-them.html

Photo by Tookapic, available under Creative Commons CC0 1.0 license.

„There was an error while sending your boarding pass“

Check-in Web-UI Me Current situation

Check-in Barcode Generator Web-UI Me Output Mgmt Current situation

Check-in Barcode Generator Web-UI Me Output Mgmt Current situation – the bad part

Check-in Barcode Generator Web-UI Me Output Mgmt Current situation – the bad part

Check-in Barcode Generator Web-UI Me Output Mgmt Current situation – the bad part Stateful Retry

No content

We are having some technical difficulties and cannot present you your boarding pass right away. But we do actively retry ourselves, so lean back, relax and we will send it on time. …I just made this up…

Check-in Barcode Generator Web-UI Me Output Mgmt Possible situation – much better!

Check-in Barcode Generator Web-UI Me Output Mgmt Possible situation – much better! Stateful Retry

Check-in Barcode Generator Web-UI Me Output Mgmt Stateful Retry Possible situation – much better! The failure never leaves this scope!

Persist thing (Entity, Document, Actor, …) State machine or workflow engine Typical concerns DIY = effort, accidental complexity Complex, proprietary, heavyweight, slow, don‘t scale, developer adverse Scheduling, Versioning, operating, visibility, scalability, … Handling State

Workflow engines, state machines It is relevant in modern architectures

CADENCE Silicon valley has recognized Workflow engines, state machines

CADENCE There are lightweight open source options Workflow engines, state machines

CADENCE also at scale Workflow engines, state machines

CADENCE for todays demo Workflow engines, state machines

Check-in Barcode Generator Web-UI Me Output Mgmt A state machine!

Check-in Barcode Generator Web-UI Me Output Mgmt A state machine!

Example Credit Card Payment REST also available in C#

Live hacking https://github.com/flowing/flowing-retail/

Payment Now you have a state machine! Credit Card REST

Distributed systems introduce complexity you have to tackle! Credit Card Payment REST

Distributed systems

It is impossible to differentiate certain failure scenarios. Independant of communication style! Service Provider Client

Distributed systems introduce complexity you have to tackle! Credit Card Payment REST

Distributed systems introduce complexity you have to tackle! Credit Card Payment REST Do it reliably

Distributed systems 2007

Distributed transactions using compensation * Compensation *aka Saga pattern

Live hacking https://github.com/flowing/flowing-retail/

Long running services Credit Card Payment Order

God services? Credit Card Payment A few smart god services tell anemic CRUD services what to do Sam Newmann Order

God services? Credit Card Payment Order A few smart god services tell anemic CRUD services what to do Sam Newmann

„By delegating our problems to our clients we force them into mitigation. They become god services.“ Credit card failed Retry with new credit card details Charge credit card listened to by triggers Payment Service Order Service

Just the first requirements for a payment service Request payment Payment received Payment requested Charge credit card Depending on account balance Withdraw amount from account Amount Withdrawn Credit card charged Credit card failed Update credit card details Account details Credit card details updated Whenever card details are updated After two weeks Depending on the amount Amount credited Mark payment received Mark payment canceled Payment canceled Credit amount to account Whenever payment is canceled Mark payment received Payment received

Payment as a „long-running“ service Request payment Payment received Payment canceled 1) creates value – serving a result its client is really interested in 2) assumes responsibility – instead of delegating troubles with internal steps to its client 3) may go asynchronous – if needed for composite execution Payment Service

@Saga class Payment { private var paymentAmount = 0F @StartSaga @SagaEventHandler(associationProperty = "paymentId") fun on(event: PaymentRequested) { } } e.g. Axon Messaging & Saga Management is cool, but Java works as well! :-) 1) Correlate domain event 2) Save process state 3) Evaluate business policy 4) Create and issue command

Credit amount back to account Withdraw amount from cust. account Charge amount by credit card e.g. with Camunda as message-driven workflow Charge amount by credit card Credit amount back to account Update credit card details Check customer account balance Payment requested Depending on account balance Charge credit card Withdraw amount from account Credit card charged Whenever credit card is charged Mark payment received Payment received Amount Withdrawn Depending on the amount Mark payment received Payment received Mark paym. partly covered Payment partly covered Whenever payment p. covered Charge credit card Credit card failed Whenever credit card failed Update credit card details Credit card details updated Whenever card details are updated Charge credit card After two weeks Mark payment canceled Payment canceled Whenever payment is canceled Credit amount to account Amount credited Every two days Remind customer Customer reminded Withdraw amount from cust. account

DDD? Long-running services respect bounded contexts! Order service Payment service Request payment Payment received Request payment Payment canceled

Integration of Axon Messaging with Camunda. Order service Payment service Request payment Payment received Payment canceled Request payment plexiti de|coding domain language @martinschimak https://github.com/plexiti/axon-camunda-poc

Flowing retail example: https://github.com/flowing/flowing-retail/ Using Java + Spring Boot or C# REST or Kafka Camunda or Zeebe

Zeebe.io

bernd.ruecker@camunda.com @berndruecker http://bernd-ruecker.com/ martin.schimak@plexiti.com @martinschimak http://plexiti.com/ https://github.com/plexiti/axon-camunda-poc/ http://github.com/flowing/flowing-retail/ https://www.infoq.com/articles/events- workflow-automation Contact: Contact: Examples: https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html With thoughts from http://flowing.io @berndruecker | @martinschimak