Eventually Consistent Computations

React 2014; Lightning Talk

Transcript

1. Eventually Consistent Computations Christopher S. Meiklejohn Peter Van Roy
2. large-scale computation without synchronization
3. large-scale computation without synchronization
4. large-scale computation without synchronization
5. large-scale computation without synchronization
6. the Plan
7. conflict-free replicated data types (CRDTs)
8. distributed deterministic dataflow
9. compose CRDTs using deterministic data flow
10. the Data Types
11. state-based conflict-free replicated data types
12. state-based conflict-free replicated data types
13. semantic resolution
14. semantic resolution “In all such systems, we find developers spend a significant fraction of their time building extremely complex and error-prone mechanisms to cope with eventual consistency and handle data that may be out of date.” “F1: A Distributed SQL Database That Scales” (VLDB 2013)
15. strong eventual consistency
16. strong eventual consistency “Systems in which replica conflicts are impossible by design exhibit Strong Eventual Consistency (SEC).” - Wikipedia, ”Eventual Consistency”
17. state-based conflict-free replicated data types
18. bounded Join-Semilattices* * drastically simplified example
19. bounded Join-Semilattices* [1] [6] [2] [1,6] [6,2] [1,6,2] [1,2] [] * drastically simplified example
20. the Programming Model
21. the Research
22. the Research
23. the Constraint Store* σ P1 P2 Pn-1 Pn … * monotonic; single-assignment
24. the Constraint Store* σ P1 P2 Pn-1 Pn … * monotonic; single-assignment read(X) bind(X, 1)
25. the Constraint Store* σ P1 P2 Pn-1 Pn … * monotonic; lattice variables
26. the Constraint Store* σ P1 P2 Pn-1 Pn … * monotonic; lattice variables read(X) bind(X, 1) bind(X, 2) read(X, 2)
27. the Distribution Model
28. the Distribution Model
29. the Distribution Model 4 6 5 the Application 1 3 2 read(X) bind(Y, 2) read(Y) bind(X, 1)
30. the Distribution Model
31. 1 3 2 the Application the Distribution Model bind(X, 1) 1 2 3 ok
32. 1 3 2 the Application the Distribution Model
33. 1 3 2 the Application the Distribution Model register(application) 1 2 3 ok A’ A’
34. 1 3 2 the Application the Distribution Model
35. 1 3 2 the Application the Distribution Model execute(application) 1 2 3 {ok, Result} A’ A’ ⊔
36. the Distribution Model 1 3 2 4 6 5 the Application
37. the Distribution Model 1 3 2 4 6 5 the Application 1 4 ⊕ 2 5
38. the Distribution Model 1 3 2 4 6 5 the Application 2 5
39. the Distribution Model 1 3 2 4 6 5 the Application 1 4 ⊕ 2 5 5 2 ⊔ ⊔
40. the Examples
41. C1 C2 C3 S1 S2 M1 M2 M3 the Ad Counter the Diagram
42. C1 C2 C3 S1 S2 M1 M2 M3 view inc rem view view the Ad Counter the Diagram
43. the Secondary Index the Program
44. the Secondary Index the Program filter(fun even/1)* (* computes -CRDT) [1, 2] [2, 6] [1, 2, 5] [2] [2, 6] [2, 6]
45. the Secondary Index the Diagram
46. ⊕ the Secondary Index the Diagram 3 1 2 ⊔
47. the Conclusion
48. sequential composability
49. divergence control
50. moving computation to the client
51. thanks peter Van Roy russell Brown marc Shapiro annette Bieniusa manuel Bravo jordan West sean Cribbs carlos Baquero nuno Preguica joao Leitao ryan Zezeski tom Santero peter Alvaro image credits: Wikipedia, “The Village (The Prisoner)” slides: https://speakerdeck.com/cmeiklejohn/eventually-consistent-computations prototype: https://github.com/cmeiklejohn/derflow
52. be seeing you Christopher S. Meiklejohn @cmeik