Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Convergent/Divergent

 Convergent/Divergent

Presented at EmberConf 2014

Christopher Meiklejohn

March 26, 2014
Tweet

More Decks by Christopher Meiklejohn

Other Decks in Programming

Transcript

  1. CONVERGENT
    DIVERGENT
    ⊤ FAITS PRÉCIS
    1

    View Slide

  2. @ C M E I K
    CMEIKLEJOHN
    B A S H O
    GEORGIA TECH
    TDISTRIBUTED
    2

    View Slide

  3. SYNCFREE
    3
    Funded by the European Union

    View Slide

  4. THE MOTIVATION
    SOME THEORY
    A SOLUTION
    4

    View Slide

  5. MOTIVATION
    EMBER DEV
    RICON 2013
    5

    View Slide

  6. INTRODUCTION
    6

    View Slide

  7. YOU’RE BUILDING
    A DISTRIBUTED SYSTEM
    7

    View Slide

  8. PUSH STATE
    TO CLIENT
    8

    View Slide

  9. REGARDLESS OF
    DATABASE ISOLATION
    9

    View Slide

  10. REGARDLESS OF
    PUSHING STATE
    TO CLIENTS
    10

    View Slide

  11. DROPPED MESSAGES
    REORDERED MESSAGES
    RACE CONDITIONS
    PARTIAL FAILURES
    CUSTOM MERGES
    11

    View Slide

  12. DROPPED MESSAGES
    REORDERED MESSAGES
    TCP INCAST
    TCP SLOW-START
    NAGLE’S ALGORITHM
    12

    View Slide

  13. RACE CONDITIONS
    PARTIAL FAILURES
    13

    View Slide

  14. CUSTOM MERGES
    14

    View Slide

  15. DISTRIBUTED
    S Y S T E M S
    T H E O R Y
    15

    View Slide

  16. CONSISTENCY
    16

    View Slide

  17. The data consistency model specifies a contract
    between programmer and system, wherein the
    system guarantees that if the programmer
    follows the rules, memory will be consistent
    and the results of memory operations will be
    predictable.
    !
    Wikipedia, Consistency Model
    17

    View Slide

  18. STRICT
    LINEAR

    IZABLE
    18

    View Slide

  19. EVEN

    TUAL
    19

    View Slide

  20. CAUSAL
    20

    View Slide

  21. SAFETY VS
    L I V E N E S S
    21

    View Slide

  22. CONSENSUS
    22

    View Slide

  23. A fundamental problem in distributed
    computing is to achieve overall system
    reliability in the presence of a number of faulty
    processes. This often requires processes to
    agree on some data value that is needed
    during computation. Examples of applications
    of consensus include whether to commit a
    transaction to a database, agreeing on the
    identity of a leader, state machine replication,
    and atomic broadcasts.
    !
    Wikipedia, Consensus
    23

    View Slide

  24. TERMINATION

    AGREE MENT
    V A L I D I T Y
    24

    View Slide

  25. T W O
    GENERALS
    25

    View Slide

  26. BYZANTINE
    GENERALS
    26

    View Slide

  27. 2 PC
    3 PC
    PAXOS
    R A F T
    27

    View Slide

  28. CAUSALITY
    28

    View Slide

  29. TIME CLOCKS AND THE
    ORDERING OF EVENTS
    IN A DISTRIBUTED SYSTEM
    29

    View Slide

  30. VECTOR
    CLOCKS
    30

    View Slide

  31. VERSION
    VECTORS
    31

    View Slide

  32. CHARRON -
    B O S T
    32

    View Slide

  33. D O T T E D
    VERSION VECTORS
    33

    View Slide

  34. CONFLICT FREE
    R E P L I C A T E D
    DATA TYPES
    34

    View Slide

  35. C O N V E R G E N T
    C O M M U T A T I V E
    R E P L I C A T E D
    DATA TYPES
    35

    View Slide

  36. TWO FLAVORS
    STATE BASED
    OPERATIONS BASED
    36

    View Slide

  37. MONOTONICITY
    37

    View Slide

  38. ASSOCIATIVITY
    (X · Y) · Z = X · (Y · Z)
    38

    View Slide

  39. COMMUTATIVITY
    X · Y = Y · X
    39

    View Slide

  40. IDEMPOTENCE
    X · X = X
    40

    View Slide

  41. BOUNDED JOIN
    S E M I L A T T I C E S
    41

    View Slide

  42. 42
    b a c
    a, b a, c
    a, b, c
    SET; MERGE: UNION
    b, c

    View Slide

  43. 43
    3 5 7
    5 7
    7
    INC NAT; MERGE: MAX

    View Slide

  44. 44
    F F T
    F T
    T
    BOOL; MERGE: OR

    View Slide

  45. STATE BASED
    CVRDT EXAMPLE
    THE OR-SET
    45

    View Slide

  46. [ [{1, a}], [] ] [ [{1, a}], [] ]

    View Slide

  47. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}], [{1, a}] ]

    View Slide

  48. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}, {2, a}], [{1, a}] ]

    View Slide

  49. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}, {2, a}], [{1, a}] ]
    [ [{1, a}, {2, a}], [{1, a}] ]

    View Slide

  50. [ [{1, a}], [] ] [ [{1, a}], [] ]

    View Slide

  51. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}, {2, b}], [] ]

    View Slide

  52. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}, {2, b}], [] ]
    [ [{1, a}], [{1, a}] ]

    View Slide

  53. [ [{1, a}], [] ] [ [{1, a}], [] ]
    [ [{1, a}, {2, b}], [] ]
    [ [{1, a}], [{1, a}] ]
    [ [{1, a}, {2, b}], [{1, a}] ]

    View Slide

  54. WHY
    54

    View Slide

  55. CONSENSUS
    IS HARD
    55

    View Slide

  56. AVOID COORDINATION
    56

    View Slide

  57. WEAK CONSISTENCY
    HIGHER AVAILABILITY
    57

    View Slide

  58. CONCLUSION
    58

    View Slide

  59. YOU’RE BUILDING
    A DISTRIBUTED SYSTEM
    59

    View Slide

  60. OR-SET IS
    ONE EXAMPLE
    60

    View Slide

  61. COUNTERS
    MAPS SETS
    REGISTERS
    BOOLEANS
    61

    View Slide

  62. LOOK AT
    EXISTING WORK
    62

    View Slide

  63. 63
    PAPEC 2014

    View Slide

  64. SYNCFREE.LIP6.FR
    64

    View Slide

  65. THINKDISTRIBUTED.IO
    65

    View Slide

  66. FIN
    66

    View Slide