Microservices: Where Are My
Transactions And My
Consistency???
Eberhard Wolff
Head of Architecture
https://swaglab.rocks/
https://ewolff.com/
Slide 2
Slide 2 text
Microservices are
Distributed Systems
Slide 3
Slide 3 text
Distributed Systems Are Hard:
Consistency & Reliability
Slide 4
Slide 4 text
Happened-before
ordering in
distributed
systems earned
me a reward.
“Time, Clocks and the Ordering of
Events in a Distributed System”
Leslie Lamport
Slide 5
Slide 5 text
A distributed system is
one in which the failure
Leslie Lamport
of a computer you didn't
even know existed
can render your own
computer unusable.
Slide 6
Slide 6 text
Databases have a
hard time
Kyle Kingsbury
implementing
distributed algorithms
correctly
https://aphyr.com/tag
s/jepsen
Slide 7
Slide 7 text
I am not nearly as smart as
Leslie Lamport
or Kyle Kingsbury.
Slide 8
Slide 8 text
Avoid Consistency Issues!
•Store each piece of data in one microservice
•Microservice uses a database
•Database solves the problem
…and Kyle Kingsbury helps me to choose the right one.
Slide 9
Slide 9 text
Entity Service
Customer
Service
Product
Service
Order
Process
Delivery
Process
Invoicing
Database for scaling
…and consistency
Slide 10
Slide 10 text
Entity Service
• Every call goes through three services.
• Performance
• Latency
Slide 11
Slide 11 text
Entity Service
• Failure can easily propagate.
• How can you make the system resilient?
• Eberhard = default customer, PS5 = default product ?
Customer
Service
Product
Service
Order
Process
Delivery
Process
Invoicing
Slide 12
Slide 12 text
Transactions
• Order process changes customer and product
• Probably need a transaction
Customer
Service
Product
Service
Order
Process
Delivery
Process
Invoicing
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
ACID
• Properties of classic transactions (tx)
• Atomicity: Either do full tx full or nothing
• Change customer and product
• Consistency: No violated constraints
• No negative number of products in stock
• Otherwise: rollback
Slide 15
Slide 15 text
ACID
• Isolation: Parallel tx isolated
• Other orders either processed before or after
• i.e. no concurrent decrease of stock
• Durability: Result of tx stored
• e.g. # in stock stored
2PC
Customer
Service
Product
Service
Order
Process
Delivery
Process
Invoicing
Transfer Tx ID
via network
Pass Tx ID to database
Start commit / rollback
Execute commit / rollback
Slide 21
Slide 21 text
2PC
•Outdated solutions existed
•Borland / Inprise ITS
•CORBA OTS, JTA, C++ interfaces
•Oracle, IMS, CICS, Tuxedo, MQ Series interfaces
•🤔 Can I make a fortune implementing that for HTTP,
gRPC…?
Slide 22
Slide 22 text
2PC
•Problem: slows down the happy path
•2nd phase
+ additional overhead
Slide 23
Slide 23 text
2PC
•Problem: Blocks resource
•Resource must be blocked until 2PC is fully done.
•Potentially long
•What if client crashes?
…or if there are network problems?
Slide 24
Slide 24 text
2PC
•Problem: no fully reliable
Commit phase 2:
What if resource is now unable to commit?
Crashes of client / coordinator?
•Tx should prevent problems
•I.e. reliability is key
Slide 25
Slide 25 text
2PC
•Not possible with current technologies
•For good reason!
Slide 26
Slide 26 text
2PC = less performance
+ no 100% reliability
Slide 27
Slide 27 text
2PC: No option nowadays
for good reasons
Slide 28
Slide 28 text
Entity Service: DON’T
• Bad performance
• Bad reliability
• Needs distributed tx – impossible
Customer
Service
Product
Service
Order
Process
Delivery
Process
Invoicing
Slide 29
Slide 29 text
Entity Service: DON’T
Slide 30
Slide 30 text
ACID Transactions:
DON’T and CAN’T
Slide 31
Slide 31 text
Loose Coupling & Modules
• Put everything for a set of features in one microservice
• Including all data
• Result: bounded context with loose coupling
Order
Process
Including all
customer & product
data for orders
Delivery
Process
All data for delivery
Invoicing
Process
All data for invoicing
Slide 32
Slide 32 text
Saga
•Saga: Sequence of local transactions
•Local transaction triggers next local transaction
•Process done stepwise
Slide 33
Slide 33 text
Saga
•Failure: compensation
•Compensation undoes the original transaction
•e.g. cancel an order
•Canceled order are stored
•i.e. the state change is not undone
Slide 34
Slide 34 text
Saga
Order Process Delivery Process
Invoicing
Process
Slide 35
Slide 35 text
Saga
Order Process Delivery Process
Invoicing
Process
Compensate!
Reimburse
Slide 36
Slide 36 text
Saga
•Closer to what happens in the real world
•No atomicity
•No isolation
•Also: not all steps have received the order
•i.e. transaction in flight:
some services have not received it yet
•WS-Transaction standard (2002)
Slide 37
Slide 37 text
Saga
•Requires reliable communication
•E.g. messaging
+ retransmission
+ idempotency
Slide 38
Slide 38 text
2PC instead of Saga
Order Process Delivery Process
Invoicing
Process
Rollback!
Atomicity: All information about the order is gone!
Rollback!
Slide 39
Slide 39 text
What is the status of my
order #42?
WAT?????
WAT?????????????
There is no order #42.
That happens if the order
was not paid.
It’s safer and more
consistent that way.
But we have no data - can’t
tell you whether that was
the problem.
Transactions
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
Saga vs 2PC
•Saga might catch business logic better
www.enterpriseintegrationpatterns.com/ramblings/18
_starbucks.html
•Sagas cause less problems if distributed
•Don’t do 2PC!
•If ACID is needed: Put it in one microservice!
Slide 43
Slide 43 text
Consistency in DDD
•Entity: Object distinguished by identity
•Value object: distinguished by values
•Aggregate: Cluster of entities and value object
•Service: Process or transformation outside entity or
value object
•Transactions / ACID consistency: where?
Slide 44
Slide 44 text
Consistency in DDD
•Use the same aggregate boundaries to govern
transactions and distribution.
•Within an aggregate boundary,
apply consistency rules synchronously.
Across boundaries, handle updates asynchronously.
•I.e. aggregates order, delivery, invoice inconsistent
Slide 45
Slide 45 text
Consistency in DDD
•Entities / value objects inside order aggregate are
consistent
•Aggregate in order process bounded context might be
inconsistent
•Bounded contexts might be inconsistent
Slide 46
Slide 46 text
CAP
Slide 47
Slide 47 text
CAP Theorem: Chose 2 of C, A, P
Consistency
Partition
Tolerance
Availability
Quorum
Replication
DNS
NOT POSSIBLE!*
* https://codahale.com/you-cant-sacrifice-partition-tolerance
Slide 48
Slide 48 text
BASE
Basically
Available
Soft state
Eventually consistent
AP = Available not consistent
Slide 49
Slide 49 text
ACID Consistency is not
CAP Consistency!
Slide 50
Slide 50 text
ACID vs CAP Consistency
• ACID Consistency: No violated constraints
• i.e. invariants, referential integrity etc
• CAP consistency: All nodes have the same data
Slide 51
Slide 51 text
What Did I Order?
Order
Process
Invoicing Process
Delivery
Process
Slide 52
Slide 52 text
Has the Invoice been Payed?
Order
Process
Invoicing Process
Delivery
Process
Slide 53
Slide 53 text
Where is my Delivery?
Order
Process
Invoicing Process
Delivery
Process
Slide 54
Slide 54 text
Where is the distribution?
Slide 55
Slide 55 text
Order Accepted
•Start delivery & invoicing
•Possible inconsistency:
invoice, no delivery yet
Order
Process
Delivery
Process
Invoicing Process
Slide 56
Slide 56 text
Delivery Not Possible
•Cancel order
•Reimburse!
•Possible inconsistence: no reimbursement yet
Order
Process
Delivery
Process
Invoicing Process
Slide 57
Slide 57 text
Conclusion
Slide 58
Slide 58 text
Conclusion
•No ACID transactions between microservices
•Saga: String of transactions + compensation
•Saga might be the better option to model domain
logic
Slide 59
Slide 59 text
Conclusion
•Aggregates are supposed to be ACID consistent,
not bounded contexts.
•CAP shouldn’t be a huge issue
•All depends on good split of business logic!