Convergent/Divergent

Presented at EmberConf 2014

Transcript

1. CONVERGENT
   DIVERGENT
   ⊤ FAITS PRÉCIS
   1
   

   View Slide

2. @ C M E I K
   CMEIKLEJOHN
   B A S H O
   GEORGIA TECH
   TDISTRIBUTED
   2
   

   View Slide

3. SYNCFREE
   3
   Funded by the European Union
   

   View Slide

4. THE MOTIVATION
   SOME THEORY
   A SOLUTION
   4
   

   View Slide

5. MOTIVATION
   EMBER DEV
   RICON 2013
   5
   

   View Slide

6. INTRODUCTION
   6
   

   View Slide

7. YOU’RE BUILDING
   A DISTRIBUTED SYSTEM
   7
   

   View Slide

8. PUSH STATE
   TO CLIENT
   8
   

   View Slide

9. REGARDLESS OF
   DATABASE ISOLATION
   9
   

   View Slide

10. REGARDLESS OF
    PUSHING STATE
    TO CLIENTS
    10
    

    View Slide

11. DROPPED MESSAGES
    REORDERED MESSAGES
    RACE CONDITIONS
    PARTIAL FAILURES
    CUSTOM MERGES
    11
    

    View Slide

12. DROPPED MESSAGES
    REORDERED MESSAGES
    TCP INCAST
    TCP SLOW-START
    NAGLE’S ALGORITHM
    12
    

    View Slide

13. RACE CONDITIONS
    PARTIAL FAILURES
    13
    

    View Slide

14. CUSTOM MERGES
    14
    

    View Slide

15. DISTRIBUTED
    S Y S T E M S
    T H E O R Y
    15
    

    View Slide

16. CONSISTENCY
    16
    

    View Slide

17. The data consistency model specifies a contract
    between programmer and system, wherein the
    system guarantees that if the programmer
    follows the rules, memory will be consistent
    and the results of memory operations will be
    predictable.
    !
    Wikipedia, Consistency Model
    17
    

    View Slide

18. STRICT
    LINEAR

    IZABLE
    18
    

    View Slide

19. EVEN

    TUAL
    19
    

    View Slide

20. CAUSAL
    20
    

    View Slide

21. SAFETY VS
    L I V E N E S S
    21
    

    View Slide

22. CONSENSUS
    22
    

    View Slide

23. A fundamental problem in distributed
    computing is to achieve overall system
    reliability in the presence of a number of faulty
    processes. This often requires processes to
    agree on some data value that is needed
    during computation. Examples of applications
    of consensus include whether to commit a
    transaction to a database, agreeing on the
    identity of a leader, state machine replication,
    and atomic broadcasts.
    !
    Wikipedia, Consensus
    23
    

    View Slide

24. TERMINATION

    AGREE MENT
    V A L I D I T Y
    24
    

    View Slide

25. T W O
    GENERALS
    25
    

    View Slide

26. BYZANTINE
    GENERALS
    26
    

    View Slide

27. 2 PC
    3 PC
    PAXOS
    R A F T
    27
    

    View Slide

28. CAUSALITY
    28
    

    View Slide

29. TIME CLOCKS AND THE
    ORDERING OF EVENTS
    IN A DISTRIBUTED SYSTEM
    29
    

    View Slide

30. VECTOR
    CLOCKS
    30
    

    View Slide

31. VERSION
    VECTORS
    31
    

    View Slide

32. CHARRON -
    B O S T
    32
    

    View Slide

33. D O T T E D
    VERSION VECTORS
    33
    

    View Slide

34. CONFLICT FREE
    R E P L I C A T E D
    DATA TYPES
    34
    

    View Slide

35. C O N V E R G E N T
    C O M M U T A T I V E
    R E P L I C A T E D
    DATA TYPES
    35
    

    View Slide

36. TWO FLAVORS
    STATE BASED
    OPERATIONS BASED
    36
    

    View Slide

37. MONOTONICITY
    37
    

    View Slide

38. ASSOCIATIVITY
    (X · Y) · Z = X · (Y · Z)
    38
    

    View Slide

39. COMMUTATIVITY
    X · Y = Y · X
    39
    

    View Slide

40. IDEMPOTENCE
    X · X = X
    40
    

    View Slide

41. BOUNDED JOIN
    S E M I L A T T I C E S
    41
    

    View Slide

42. 42
    b a c
    a, b a, c
    a, b, c
    SET; MERGE: UNION
    b, c
    

    View Slide

43. 43
    3 5 7
    5 7
    7
    INC NAT; MERGE: MAX
    

    View Slide

44. 44
    F F T
    F T
    T
    BOOL; MERGE: OR
    

    View Slide

45. STATE BASED
    CVRDT EXAMPLE
    THE OR-SET
    45
    

    View Slide

46. [ [{1, a}], [] ] [ [{1, a}], [] ]
    

    View Slide

47. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}], [{1, a}] ]
    

    View Slide

48. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}, {2, a}], [{1, a}] ]
    

    View Slide

49. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}, {2, a}], [{1, a}] ]
    [ [{1, a}, {2, a}], [{1, a}] ]
    

    View Slide

50. [ [{1, a}], [] ] [ [{1, a}], [] ]
    

    View Slide

51. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}, {2, b}], [] ]
    

    View Slide

52. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}, {2, b}], [] ]
    [ [{1, a}], [{1, a}] ]
    

    View Slide

53. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}, {2, b}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}, {2, b}], [{1, a}] ]
    

    View Slide

54. WHY
    54
    

    View Slide

55. CONSENSUS
    IS HARD
    55
    

    View Slide

56. AVOID COORDINATION
    56
    

    View Slide

57. WEAK CONSISTENCY
    HIGHER AVAILABILITY
    57
    

    View Slide

58. CONCLUSION
    58
    

    View Slide

59. YOU’RE BUILDING
    A DISTRIBUTED SYSTEM
    59
    

    View Slide

60. OR-SET IS
    ONE EXAMPLE
    60
    

    View Slide

61. COUNTERS
    MAPS SETS
    REGISTERS
    BOOLEANS
    61
    

    View Slide

62. LOOK AT
    EXISTING WORK
    62
    

    View Slide

63. 63
    PAPEC 2014
    

    View Slide

64. SYNCFREE.LIP6.FR
    64
    

    View Slide

65. THINKDISTRIBUTED.IO
    65
    

    View Slide

66. FIN
    66
    

    View Slide