The Rise of Reactive Microservices

The Rise of Reactive Microservices CraftConf 2022 @duﬄeit

David Leitner @duffleit Coding Architect [email protected] @duﬄeit

The Rise of Reactive Microservices CraftConf 2022 @duﬄeit

@duffleit “ Programming with, or designing upon, asynchronous data streams.

@duffleit

“ streams are just another data structure. @duffleit Time Space
Value Getter Setter Collections Iterators Generators Promise Deferred Resolver Stream Subscriber Emitter

“ streams are just another data structure. @duffleit

@duffleit “ Programming with, or designing upon, asynchronous data streams.

@duffleit var counter = 2; var doubled = counter *
2; 1 2

2; assert(doubled, 4); 1 2 3 4 ✅

2; // updating the counter counter = 3; 1 2 3 4 5

2; // updating the counter counter = 3; assert(doubled, 6) 1 2 3 4 5 6 7 💔

2; // updating the counter counter = 3; doubled = counter * 2; 1 2 3 4 5 6

2; // updating the counter counter = 3; doubled = counter * 2; assert(doubled, 6) 1 2 3 4 5 6 7 8

2; // updating the counter counter = 3; doubled = counter * 2; assert(doubled, 6) 1 2 3 4 5 6 7 8 ✅ doubled 🔗 counter paulstovell.com/reactive-programming/

@duffleit var counter = 2; var doubled <= counter *
2; 1 2 doubled 🔗 counter 👆

2; // updating the counter counter = 3; 1 2 3 4 5 doubled 🔗 counter

2; // updating the counter counter = 3; assert(doubled, 6) 1 2 3 4 5 6 7 doubled 🔗 counter ✅

@duffleit var counter = stream(2); 1 🔗 = streams 👈
counter: [2]

@duffleit var counter = stream(2); var doubled = counter.map(x ->
x * 2) 1 2 👈 counter: [2]

x * 2) 1 2 🔗 = streams 👆 counter: [2]

x * 2) 1 2 👆 counter: [2] doubled: [4]

x * 2) 1 2 “ we project the data from our counter stream into our doubled stream. 👆 counter: [2] doubled: [4]

x * 2) // updating the counter 1 2 3 4 👈 counter: [2] doubled: [4]

x * 2) // updating the counter counter.emit(3); 1 2 3 4 5 👈 counter: [2] doubled: [4]

x * 2) // updating the counter counter.emit(3); 1 2 3 4 5 counter: [2, 3] doubled: [4] 👈

x * 2) // updating the counter counter.emit(3); 1 2 3 4 5 counter: [2, 3] doubled: [4, 6] 👈

x * 2) // updating the counter counter.emit(3); assert(doubled.value, 6) 1 2 3 4 5 6 7 👈 counter: [2, 3] doubled: [4, 6] ✅

@duffleit

@duffleit var counter = new BehaviorSubject<number>(2); 1 👈

@duffleit var counter = new BehaviorSubject<number>(2); var doubled = counter.pipe(map(x
-> x * 2)); 1 2 👈

-> x * 2)); // updating the counter counter.emit(3); 1 2 3 4 5 👈

-> x * 2)); // updating the counter counter.emit(3); assert(doubled.value, 6); 1 2 3 4 5 6 7 👈 ✅

@duffleit

@duffleit var counter = 2; 1 👈

@duffleit var counter = 2; $: doubled = counter *
2; 1 2 👈

2; // updating the counter counter = 3; 1 2 3 4 5 👈

2; // updating the counter counter = 3; assert(doubled, 6); 1 2 3 4 5 6 7 👈 ✅

@duffleit We can apply those Concepts of Reactive Programming on
Architecture as well. * and hope they still make sense.

@duffleit User <μService> Account <μService> Legder <μService> WebBanking New Payment
💸 Perform Payment € 20, 00 👧 Lisa to David from transferMoney triggerPayment createTransactions

@duffleit User <μService> Account <μService> Legder <μService> WebBanking Your payment
was successfully executed. transferMoney triggerPayment createTransactions ✅

@duffleit User <μService> Account <μService> Legder <μService> WebBanking New Payment
💸 Perform Payment € 20, 00 👧 Lisa to David from transferMoney triggerPayment createTransactions <μService> Legder Legder <μService>

@duffleit User <μService> Account <μService> Legder <μService> WebBanking Your payment
was successfully executed. transferMoney triggerPayment createTransactions ✅ Retries Timeouts Circuit Breakers Synchronise

First Generation of MicroServices Synchronously Integrated @duffleit

@duffleit User <μService> Account <μService> WebBanking transferMoney triggerPayment New Payment
💸 € 20, 00 👧 Lisa to David from Perform Payment <μService> Legder

@duffleit User <μService> Account <μService> WebBanking transferMoney triggerPayment createTransactions New
Payment 💸 € 20, 00 👧 Lisa to David from Perform Payment Queue <μService> Legder

@duffleit User <μService> Account <μService> WebBanking transferMoney triggerPayment createTransactions Queue
<μService> Legder Your payment was accepted. ✅

<μService> Legder Your payment was accepted. ✅ Legder <μService>

<μService> Legder Your payment was accepted. ✅ Legder <μService> Queue

<μService> Legder Your payment was accepted. ✅ Legder <μService> Queue Queue

Second Generation of MicroServices Message-Driven and Asynchronously Integrated @duffleit

Payment 💸 € 20, 00 👧 Lisa to David from Perform Payment Legder <μService> Queue Queue Queue

Payment 💸 € 20, 00 👧 Lisa to David from Perform Payment Legder <μService> Queue Queue Queue As a Customer, I want my transaction to be booked by the ledger.

Payment 💸 € 20, 00 👧 Lisa to David from Perform Payment Legder <μService> Queue Queue Queue As a Customer, I want the account service to update my balance.

Payment 💸 € 20, 00 👧 Lisa to David from Perform Payment Legder <μService> Queue Queue Queue As a Customer, I want to transfer money to someone else.

The Entity Service Antipattern. @duffleit

Let's create Services along Customer Journeys. @duffleit

@duffleit “ On the Criteria To Be Used in Decomposing
Systems into Modules — Parnas, 1972

Let's create Services along Customer Journeys. @duffleit

@duffleit User <μService> Account <μService> WebBanking New Payment 💸 €
20, 00 👧 Lisa to David from Perform Payment Legder <μService> Payments <μService> Domain Services Journey Services As a Customer, I want to transfer money to someone else.

20, 00 👧 Lisa to David from Perform Payment Legder <μService> Web-BFF <μService> Domain Services Backend For Frontends As a Customer, I want to transfer money to someone else. Mobile-BFF <μService> Mobile Banking € 20, 00 Pay Payment Flow Payment Flow

@duffleit No Domain Logic in Backend For Frontends. It's about
Ui-Speciﬁc Aggregation, to get rid of over-fetching and over-requesting.

20, 00 👧 Lisa to David from Perform Payment Legder <μService> Payments <μService> Domain Services Journey Services

20, 00 👧 Lisa to David from Perform Payment Legder <μService> Payments <μService> Domain Services Journey Services Balance <μService>

@duffleit ⚡ Customer Journey and Entity Data Mismatch Problem

@duffleit Users Accounts Transactions Entities Customer Journeys Payments Onboarding Fraud
Detection ⚡

@duffleit User Account WebBanking New Payment 💸 € 20, 00
👧 Lisa to David from Perform Payment Transactions Payments Domain Services Journey Services Onboarding Fraud Detection User Account Account Transactions

@duffleit User Account WebBanking New Payment 💸 € 20, 00
👧 Lisa to David from Perform Payment Transactions Payments Domain Services Journey Services Onboarding Fraud Detection User Account Account Transactions ⚡ ⚡ ⚡

@duffleit WebBanking New Payment 💸 € 20, 00 👧 Lisa
to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Service Autnomy

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Data Duplication ⚡

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Data Syncronisation 🤯 ⚡

@duffleit clarify who owns which Data. Always

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Synchronous Calls 😔

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Cache Account Transactions Synchronous Calls 😔

@duffleit “ we project data from our counter variable into
our doubled variable. There must be a better solution 🤔

@duffleit “ we project the users from our onboarding service
into our payments service. There must be a better solution 🤔 Payments User Onboarding User project

@duffleit Payments Onboarding User Aggregate User Readmodel UserChangedEvent UserChange Command
That's CQRS in a nutshell 🥜 update

@duffleit Payments Onboarding User Aggregate User Readmodel UserChangedEvent UserChange Command
Stream

@duffleit Payments Onboarding User Aggregate User Readmodel UserChange Command Stream
UserChangedEvent transform apply

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Stream

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Stream Projections

to David from Perform Payment Payments Journey Services Onboarding Fraud Detection User Account Account Transactions User Account Transactions Stream

@duffleit But didn't we just introduce Eventual Consistency?

@duffleit The diﬀerent faces of Eventual Consistency

@duffleit Eventual Consistency Eventual Divergence Dashboard User (age: 22) UserChanged
(age: 21) UserChanged (age: 22) Proﬁle User (age: 21) ⚡ timestamp timestamp

@duffleit Eventual Consistency Eventual Divergence Eventual Variance Ledger Stream Transaction
(+10€) 🔥

@duffleit Eventual Consistency Eventual Divergence Eventual Variance Ledger Dashboard Stream
🔥 Transaction (+10€) Transaction (+10€) Balance: 20€

@duffleit Eventual Consistency Eventual Divergence Eventual Variance Eventual Latency Service
B Stream UserChanged (age: 21) UserChanged (age: 22) Proﬁle User (age: 21) Dashboard User (age: 21) User (age: 22)

Single User Context @duffleit Eventual Consistency Eventual Divergence Eventual Variance
Eventual Latency Service B Stream UserChanged (age: 21) UserChanged (age: 22) Proﬁle User (age: 21) Dashboard User (age: 21) User (age: 22) 🕰

@duffleit Eventual Consistency Eventual Divergence Eventual Variance Eventual Latency same
events, different results. same events, different quantity. same events, different realisation. Ordering Guarantees Optimistic UIs At-Least-Once Delivery Deduplication Mechanisms

@duffleit tl/dr: Eventual Consistency is often used as a knock-out
argument @duffleit

@duffleit tl/dr: Eventual Consistency is often used as a knock-out
argument but isn't, in most cases. @duffleit

@duffleit Mobile Banking Usage 93% — Show balance & last
transactions 3% — Initiate new payment 4% — Any other functionality

@duffleit Payments Journey Services Onboarding Overview User Account Account Transactions
User Account Transactions Stream

@duffleit Payments Journey Services Onboarding Overview User Account Account Transactions
User Account Transactions Stream 93% of what our users need can still be performed. WebBanking Current Balance: 200€ 👧 You sent Lisa 100€ Christoph sent you 300€

@duffleit WebBanking Current Balance: 200€ 👧 You sent Lisa 100€
Christoph sent you 300€ Payments Journey Services Onboarding Overview User Account Transactions User Account Transactions Stream Overview Transactions We can easily scale horizontally.

@duffleit WebBanking of David Current Balance: 300€ Christoph sent you
300€ Payments Journey Services Onboarding Overview User Account Transactions User Account Transactions Stream WebBanking of Lisa 👧 New Payment 💸 € 20, 00 David to 👧 Lisa from Perform Payment

300€ Payments Journey Services Onboarding Overview User Account Transactions User Account Transactions Stream WebBanking of Lisa 👧 New Payment 💸 € 20, 00 David to 👧 Lisa from Perform Payment 👧 Lisa sent you 20€

300€ Payments Journey Services Onboarding Overview User Account Transactions User Account Transactions Stream WebBanking of Lisa 👧 New Payment 💸 € 20, 00 David to 👧 Lisa from Perform Payment 👧 Lisa sent you 20€ GraphQL subscriptions mutations Out-of-the-box real-time capabilities.

300€ Payments Journey Services Legacy System Accounts Stream 👧 Lisa sent you 20€ User 😮 💨😮 💨 😮 💨 Change Data Capture (CDC) User User

Third Generation of MicroServices Built on Reactive Data-Streams @duffleit High
Resilience 🔥 Horizontal Scaling ↔ Real Time Integration ⏱ Good Integration with Legacy Systems 🏚

Dimensions of Scaling @duffleit Vertical Horizontal Service

Dimensions of Scaling Vertical Horizontal Service Service Service Stream

Dimensions of Scaling Vertical Horizontal Service Service Service Sharding

Dimensions of Scaling Vertical Horizontal Service Service Service Sharding Users
A-H Users I-P Users Q-Z

Dimensions of Scaling Vertical Horizontal Service Service Service User from
User from Users from Service Service Service Sharding

@duffleit UserTriggeredPayment

@duffleit Stream UserTriggeredPayment Partition 0 Partition 1 Partition 2 UserTriggeredPayment

Payments Payments Payments

Payments Payments Payments ValidPaymentReceived

Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2

Fraud Detection Fraud Detection @duffleit Stream UserTriggeredPayment Partition 0 Partition
1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection PaymentCheckedAgainstFraud

1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection Stream PaymentChecked Partition 0 Partition 1 Partition 2

1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection Stream PaymentChecked Partition 0 Partition 1 Partition 2 Ledger Ledger Ledger TransactionBooked

1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection Stream PaymentChecked Partition 0 Partition 1 Partition 2 Ledger Ledger Ledger Partition 0 Partition 1 Partition 2 Stream TransactionBooked

1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection Partition 0 Partition 1 Partition 2 Ledger Ledger Ledger Partition 0 Partition 1 Partition 2 Projecting Transactions Streams Flink Stream PaymentChecked Stream TransactionBooked Independent Shards

1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection Stream PaymentChecked Partition 0 Partition 1 Partition 2 Ledger Ledger Ledger Partition 0 Partition 1 Partition 2 Stream TransactionBooked Shard 0 Shard 1 Shard 2 DataSink Streams Flink Projecting Transactions WebBanking WebBanking WebBanking

1 Partition 2 UserTriggeredPayment Payments Payments Payments Stream ValidPaymentReceived Partition 0 Partition 1 Partition 2 Fraud Detection Stream PaymentChecked Partition 0 Partition 1 Partition 2 Ledger Ledger Ledger Partition 0 Partition 1 Partition 2 Stream TransactionBooked Shard 0 Shard 1 Shard 2 DataSink 🚀 We can scale nearly unlimited. Projecting Transactions WebBanking WebBanking WebBanking

Forth Generation of MicroServices Shared and fully Stream Based @duffleit
Massive Throughput 🚀 Extremely Scalable ↔

So many choices, Lets sum up. @duffleit

Monolith @duffleit

Syncronously 😩 Pain @duffleit

Modulith @duffleit

Asyncronously 📦 Shared Data @duffleit

Reactive ⚡ From Pull to Push @duffleit

Streaming Massiv Throughput 🚀 @duffleit

Monolith Reactive Push-Based Async Pull-Based Streamed Availability 🟢 , Resilience
🦺 , Scalability ↔ , Throughput 🚀 Complexity 💰 @duffleit

Monolith Reactive Push-Based Async Pull-Based Streamed @duffleit The Rise of
Reactive Microservices. And the future of fully Stream Based Architectures.

David Leitner @duffleit Coding Architect [email protected] @duﬄeit

squer.link / slicing-microservices

The Rise of Reactive Microservices

The Rise of Reactive Microservices

More Decks by SQUER Solutions

Featured

Transcript