Eventually Consistent Computations

Eventually Consistent Computations

React 2014; Lightning Talk

3e09fee7b359be847ed5fa48f524a3d3?s=128

Christopher Meiklejohn

November 18, 2014
Tweet

Transcript

  1. Eventually Consistent Computations Christopher S. Meiklejohn Peter Van Roy

  2. large-scale computation without synchronization

  3. large-scale computation without synchronization

  4. large-scale computation without synchronization

  5. large-scale computation without synchronization

  6. the Plan

  7. conflict-free replicated data types (CRDTs)

  8. distributed deterministic dataflow

  9. compose CRDTs using deterministic data flow

  10. the Data Types

  11. state-based conflict-free replicated data types

  12. state-based conflict-free replicated data types

  13. semantic resolution

  14. semantic resolution “In all such systems, we find developers spend

    a significant fraction of their time building extremely complex and error-prone mechanisms to cope with eventual consistency and handle data that may be out of date.” “F1: A Distributed SQL Database That Scales” (VLDB 2013)
  15. strong eventual consistency

  16. strong eventual consistency “Systems in which replica conflicts are impossible

    by design exhibit Strong Eventual Consistency (SEC).” - Wikipedia, ”Eventual Consistency”
  17. state-based conflict-free replicated data types

  18. bounded Join-Semilattices* * drastically simplified example

  19. bounded Join-Semilattices* [1] [6] [2] [1,6] [6,2] [1,6,2] [1,2] []

    * drastically simplified example
  20. the Programming Model

  21. the Research

  22. the Research

  23. the Constraint Store* σ P1 P2 Pn-1 Pn … *

    monotonic; single-assignment
  24. the Constraint Store* σ P1 P2 Pn-1 Pn … *

    monotonic; single-assignment read(X) bind(X, 1)
  25. the Constraint Store* σ P1 P2 Pn-1 Pn … *

    monotonic; lattice variables
  26. the Constraint Store* σ P1 P2 Pn-1 Pn … *

    monotonic; lattice variables read(X) bind(X, 1) bind(X, 2) read(X, 2)
  27. the Distribution Model

  28. the Distribution Model

  29. the Distribution Model 4 6 5 the Application 1 3

    2 read(X) bind(Y, 2) read(Y) bind(X, 1)
  30. the Distribution Model

  31. 1 3 2 the Application the Distribution Model bind(X, 1)

    1 2 3 ok
  32. 1 3 2 the Application the Distribution Model

  33. 1 3 2 the Application the Distribution Model register(application) 1

    2 3 ok A’ A’
  34. 1 3 2 the Application the Distribution Model

  35. 1 3 2 the Application the Distribution Model execute(application) 1

    2 3 {ok, Result} A’ A’ ⊔
  36. the Distribution Model 1 3 2 4 6 5 the

    Application
  37. the Distribution Model 1 3 2 4 6 5 the

    Application 1 4 ⊕ 2 5
  38. the Distribution Model 1 3 2 4 6 5 the

    Application 2 5
  39. the Distribution Model 1 3 2 4 6 5 the

    Application 1 4 ⊕ 2 5 5 2 ⊔ ⊔
  40. the Examples

  41. C1 C2 C3 S1 S2 M1 M2 M3 the Ad

    Counter the Diagram
  42. C1 C2 C3 S1 S2 M1 M2 M3 view inc

    rem view view the Ad Counter the Diagram
  43. the Secondary Index the Program

  44. the Secondary Index the Program filter(fun even/1)* (* computes -CRDT)

    [1, 2] [2, 6] [1, 2, 5] [2] [2, 6] [2, 6]
  45. the Secondary Index the Diagram

  46. ⊕ the Secondary Index the Diagram 3 1 2 ⊔

  47. the Conclusion

  48. sequential composability

  49. divergence control

  50. moving computation to the client

  51. thanks peter Van Roy russell Brown marc Shapiro annette Bieniusa

    manuel Bravo jordan West sean Cribbs carlos Baquero nuno Preguica joao Leitao ryan Zezeski tom Santero peter Alvaro image credits: Wikipedia, “The Village (The Prisoner)” slides: https://speakerdeck.com/cmeiklejohn/eventually-consistent-computations prototype: https://github.com/cmeiklejohn/derflow
  52. be seeing you Christopher S. Meiklejohn @cmeik