Eventually Consistent Computations

React 2014; Lightning Talk

Transcript

1. Eventually
   Consistent Computations
   Christopher S. Meiklejohn
   Peter Van Roy
   

   View Slide

2. large-scale computation
   without synchronization
   

   View Slide

3. large-scale computation
   without synchronization
   

   View Slide

4. large-scale computation
   without synchronization
   

   View Slide

5. large-scale computation
   without synchronization
   

   View Slide

6. the Plan
   

   View Slide

7. conflict-free
   replicated data types
   (CRDTs)
   

   View Slide

8. distributed
   deterministic dataflow
   

   View Slide

9. compose CRDTs using
   deterministic data flow
   

   View Slide

10. the Data Types
    

    View Slide

11. state-based
    conflict-free replicated
    data types
    

    View Slide

12. state-based
    conflict-free replicated
    data types
    

    View Slide

13. semantic resolution
    

    View Slide

14. semantic resolution
    “In all such systems, we find developers spend a significant fraction of their
    time building extremely complex and error-prone mechanisms to cope with
    eventual consistency and handle data that may be out of date.”
    “F1: A Distributed SQL Database That Scales” (VLDB 2013)
    

    View Slide

15. strong eventual consistency
    

    View Slide

16. strong eventual consistency
    “Systems in which replica conflicts are impossible by design exhibit
    Strong Eventual Consistency (SEC).”
    - Wikipedia, ”Eventual Consistency”
    

    View Slide

17. state-based
    conflict-free replicated
    data types
    

    View Slide

18. bounded Join-Semilattices*
    * drastically simplified example
    

    View Slide

19. bounded Join-Semilattices*
    [1] [6] [2]
    [1,6] [6,2]
    [1,6,2]
    [1,2]
    []
    * drastically simplified example
    

    View Slide

20. the Programming Model
    

    View Slide

21. the Research
    

    View Slide

22. the Research
    

    View Slide

23. the Constraint Store*
    σ
    P1
    P2
    Pn-1
    Pn
    …
    * monotonic; single-assignment
    

    View Slide

24. the Constraint Store*
    σ
    P1
    P2
    Pn-1
    Pn
    …
    * monotonic; single-assignment
    read(X)
    bind(X, 1)
    

    View Slide

25. the Constraint Store*
    σ
    P1
    P2
    Pn-1
    Pn
    …
    * monotonic; lattice variables
    

    View Slide

26. the Constraint Store*
    σ
    P1
    P2
    Pn-1
    Pn
    …
    * monotonic; lattice variables
    read(X)
    bind(X, 1)
    bind(X, 2) read(X, 2)
    

    View Slide

27. the Distribution Model
    

    View Slide

28. the Distribution Model
    

    View Slide

29. the Distribution Model
    4
    6
    5
    the
    Application
    1
    3
    2
    read(X)
    bind(Y, 2)
    read(Y)
    bind(X, 1)
    

    View Slide

30. the Distribution Model
    

    View Slide

31. 1
    3
    2
    the
    Application
    the Distribution Model
    bind(X, 1)
    1
    2
    3
    ok
    

    View Slide

32. 1
    3
    2
    the
    Application
    the Distribution Model
    

    View Slide

33. 1
    3
    2
    the
    Application
    the Distribution Model
    register(application)
    1
    2
    3 ok
    A’
    A’
    

    View Slide

34. 1
    3
    2
    the
    Application
    the Distribution Model
    

    View Slide

35. 1
    3
    2
    the
    Application
    the Distribution Model
    execute(application)
    1
    2
    3 {ok, Result}
    A’
    A’
    ⊔
    

    View Slide

36. the Distribution Model
    1
    3
    2
    4
    6
    5
    the
    Application
    

    View Slide

37. the Distribution Model
    1
    3
    2
    4
    6
    5
    the
    Application
    1 4
    ⊕
    2 5
    

    View Slide

38. the Distribution Model
    1
    3
    2
    4
    6
    5
    the
    Application
    2 5
    

    View Slide

39. the Distribution Model
    1
    3
    2
    4
    6
    5
    the
    Application
    1 4
    ⊕
    2 5
    5
    2
    ⊔
    ⊔
    

    View Slide

40. the Examples
    

    View Slide

41. C1 C2 C3
    S1 S2
    M1 M2 M3
    the Ad Counter
    the Diagram
    

    View Slide

42. C1 C2 C3
    S1 S2
    M1 M2 M3
    view
    inc
    rem
    view view
    the Ad Counter
    the Diagram
    

    View Slide

43. the Secondary Index
    the Program
    

    View Slide

44. the Secondary Index
    the Program
    filter(fun even/1)*
    (* computes -CRDT)
    [1, 2]
    [2, 6]
    [1, 2, 5]
    [2]
    [2, 6]
    [2, 6]
    

    View Slide

45. the Secondary Index
    the Diagram
    

    View Slide

46. ⊕
    the Secondary Index
    the Diagram
    3
    1
    2
    ⊔
    

    View Slide

47. the Conclusion
    

    View Slide

48. sequential composability
    

    View Slide

49. divergence control
    

    View Slide

50. moving computation
    to the client
    

    View Slide

51. thanks
    peter Van Roy russell Brown marc Shapiro
    annette Bieniusa manuel Bravo jordan West
    sean Cribbs carlos Baquero nuno Preguica joao
    Leitao ryan Zezeski tom Santero peter Alvaro
    image credits: Wikipedia, “The Village (The Prisoner)”
    slides: https://speakerdeck.com/cmeiklejohn/eventually-consistent-computations
    prototype: https://github.com/cmeiklejohn/derflow
    

    View Slide

52. be seeing you
    Christopher S. Meiklejohn
    @cmeik
    

    View Slide