Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
The Path Towards Simplifying Consistency in Distributed Systems
Caitie McCaffrey
April 24, 2017
Technology
0
89
The Path Towards Simplifying Consistency in Distributed Systems
Caitie McCaffrey
April 24, 2017
Tweet
Share
More Decks by Caitie McCaffrey
See All by Caitie McCaffrey
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
315
19k
Argus Papers We Love
caitiem20
13
980
The Verification of a Distributed System
caitiem20
22
2k
We Hear You Like Papers: Eventual Consistency
caitiem20
14
580
The Verification of a Distributed System
caitiem20
12
570
The Verification of a Distributed System
caitiem20
6
580
A Brief History of Distributed Programming: RPC
caitiem20
31
5.6k
Building Scalable Stateful Services
caitiem20
12
1k
Papers we Love PDX
caitiem20
2
970
Other Decks in Technology
See All in Technology
Design for Humans: How to make better modernization decisions
indualagarsamy
2
120
Citizen 개발기
outsider
0
290
開発組織の生産性を可視化する State of DevOpsとFour Keysとは / deep dive into State of DevOps
yfcgpsebp
0
290
Autonomous Database Cloud 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
10
18k
要約 "Add Live Text interaction to your app"
ushisantoasobu
0
150
Building smarter apps with machine learning, from magic to reality
picardparis
4
3.2k
JAWS-UG re:Habilitaion 報告 / JAWS-UG OITA rehabilitation
hiranofumio
0
130
eBPF for Security Observability
lizrice
0
200
セキュリティ 開運研修2022 / security 2022
cybozuinsideout
PRO
3
3.9k
Modern Android dependency injection
hugovisser
1
130
RDRA + JavaによるレジャーSaaSプロダクトの要件定義と実装のシームレスな接続
jjebejj
PRO
3
750
Target SDK Versionを上げない Notification runtime permission対応
napplecomputer
0
150
Featured
See All Featured
Three Pipe Problems
jasonvnalue
89
8.7k
GitHub's CSS Performance
jonrohan
1020
420k
Making Projects Easy
brettharned
98
4.3k
Clear Off the Table
cherdarchuk
79
280k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
269
11k
Pencils Down: Stop Designing & Start Developing
hursman
112
9.8k
Why Our Code Smells
bkeepers
PRO
324
55k
The Invisible Side of Design
smashingmag
290
48k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
655
120k
The Invisible Customer
myddelton
110
11k
Docker and Python
trallard
27
1.6k
Atom: Resistance is Futile
akmur
255
20k
Transcript
The Path Towards Simplifying Consistency in Distributed Systems
Caitie McCaffrey Distributed Systems Engineer caitiem.com @caitie
Distributed Systems Engineer Feral Concurrency Control Programmer Caitie McCaffrey
2015
2015 “Application-level Mechanisms for maintaining database integrity”
Halo 4 Statistics Service Halo 4 Statistics Service Record &
Aggregate Statistics From Every Multiplayer Halo Game a User Plays
Halo 4 Statistics Service
Observability at Twitter Make Dashboards & Alerts Always Available
Cmd Line Tool Viz / Dashboad Alerting Svc Cuckoo-Read Cuckoo-Write
Indexing Svc Relay Svc Twitter Front End Twitter Svc Twitter Statsite Twitter Svc Twitter Svc Scribe Collection Agent HDFS Manhattan Database Public Cloud Observability at Twitter
Notify Users When We Receive Abuse Reports
Notify Users If An Action is Taken
Report Events Stream Modification Events Stream Enrollment Events Stream Listener
Service Daemons Service Executor Service Notification Events Stream Notification Service Manhattan Macaw
Feral Concurrency Control
Service Service Service Monoliths
The Rise of NoSQL
None
Microservices
State of Affairs
Linearizable Sequential Causal Pipelined Random Access Memory Read Your Write
Monotonic Read Monotonic Write Write From Read Consistency Models
None
None
None
None
None
None
Can We Do Better?
Finance & Accounting
VisiCalc 1979
1983 Lotus 123
None
1987 Excel
None
& Standardization Computer Aided Computation
Standardization for Consistency in Distributed Systems
1988
1988 “We believe the form of communication that is needed
is remote procedure call with at- most-once semantics”
1988 “An adequate language must provide a modular, reasonably automatic
method for achieving consistency ”
Branch B Account 789: $250 Branch A Account 123 :
$100 Account 456: $50 SubAction: Deposit (123, 50) Transfer Action SubAction: Withdraw(789, 50) Transfer Deposit Withdraw enter topaction coenter action branchA.Deposit(123, 50) action branchB.Withdraw(789, 50) end end
What about Spanner?
2015
2015 “The biggest barrier…is that consistency mechanisms must integrate across
many stateful services”
Microservices
CRDTs * Stolen from Chris Meiklejohn in practice
CRDTs * Stolen from Chris Meiklejohn in practice
There Is No One Size Fits All Consistency Model
Mixed Consistency Data Stores Manhattan
None
Computer Aided Computation for Consistency in Distributed Systems
2016
Bank Application Bank Account must be > 0 Deposit Money
Withdrawal Money
‘Cause I’m Strong Enough: Reasoning About Consistency Choices in Distributed
Systems
Define Invariants & Actions Deposit Money Withdrawal Money Determine Consistency
Required Bank Account > 0 Eventual Consistency Strong Consistency Code Generation?
& Standardization Computer Aided Computation
Thank you @caitie