Convergent/Divergent

 Convergent/Divergent

Presented at EmberConf 2014

3e09fee7b359be847ed5fa48f524a3d3?s=128

Christopher Meiklejohn

March 26, 2014
Tweet

Transcript

  1. CONVERGENT DIVERGENT ⊤ FAITS PRÉCIS 1

  2. @ C M E I K CMEIKLEJOHN B A S

    H O GEORGIA TECH TDISTRIBUTED 2
  3. SYNCFREE 3 Funded by the European Union

  4. THE MOTIVATION SOME THEORY A SOLUTION 4

  5. MOTIVATION EMBER DEV RICON 2013 5

  6. INTRODUCTION 6

  7. YOU’RE BUILDING A DISTRIBUTED SYSTEM 7

  8. PUSH STATE TO CLIENT 8

  9. REGARDLESS OF DATABASE ISOLATION 9

  10. REGARDLESS OF PUSHING STATE TO CLIENTS 10

  11. DROPPED MESSAGES REORDERED MESSAGES RACE CONDITIONS PARTIAL FAILURES CUSTOM MERGES

    11
  12. DROPPED MESSAGES REORDERED MESSAGES TCP INCAST TCP SLOW-START NAGLE’S ALGORITHM

    12
  13. RACE CONDITIONS PARTIAL FAILURES 13

  14. CUSTOM MERGES 14

  15. DISTRIBUTED S Y S T E M S T H

    E O R Y 15
  16. CONSISTENCY 16

  17. The data consistency model specifies a contract between programmer and

    system, wherein the system guarantees that if the programmer follows the rules, memory will be consistent and the results of memory operations will be predictable. ! Wikipedia, Consistency Model 17
  18. STRICT LINEAR
 IZABLE 18

  19. EVEN
 TUAL 19

  20. CAUSAL 20

  21. SAFETY VS L I V E N E S S

    21
  22. CONSENSUS 22

  23. A fundamental problem in distributed computing is to achieve overall

    system reliability in the presence of a number of faulty processes. This often requires processes to agree on some data value that is needed during computation. Examples of applications of consensus include whether to commit a transaction to a database, agreeing on the identity of a leader, state machine replication, and atomic broadcasts. ! Wikipedia, Consensus 23
  24. TERMINATION
 AGREE MENT V A L I D I T

    Y 24
  25. T W O GENERALS 25

  26. BYZANTINE GENERALS 26

  27. 2 PC 3 PC PAXOS R A F T 27

  28. CAUSALITY 28

  29. TIME CLOCKS AND THE ORDERING OF EVENTS IN A DISTRIBUTED

    SYSTEM 29
  30. VECTOR CLOCKS 30

  31. VERSION VECTORS 31

  32. CHARRON - B O S T 32

  33. D O T T E D VERSION VECTORS 33

  34. CONFLICT FREE R E P L I C A T

    E D DATA TYPES 34
  35. C O N V E R G E N T

    C O M M U T A T I V E R E P L I C A T E D DATA TYPES 35
  36. TWO FLAVORS STATE BASED OPERATIONS BASED 36

  37. MONOTONICITY 37

  38. ASSOCIATIVITY (X · Y) · Z = X · (Y

    · Z) 38
  39. COMMUTATIVITY X · Y = Y · X 39

  40. IDEMPOTENCE X · X = X 40

  41. BOUNDED JOIN S E M I L A T T

    I C E S 41
  42. 42 b a c a, b a, c a, b,

    c SET; MERGE: UNION b, c
  43. 43 3 5 7 5 7 7 INC NAT; MERGE:

    MAX
  44. 44 F F T F T T BOOL; MERGE: OR

  45. STATE BASED CVRDT EXAMPLE THE OR-SET 45

  46. [ [{1, a}], [] ] [ [{1, a}], [] ]

  47. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ]
  48. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ]
  49. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ]
  50. [ [{1, a}], [] ] [ [{1, a}], [] ]

  51. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}, {2, b}], [] ]
  52. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}, {2, b}], [] ] [ [{1, a}], [{1, a}] ]
  53. [ [{1, a}], [] ] [ [{1, a}], [] ]

    [ [{1, a}, {2, b}], [] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, b}], [{1, a}] ]
  54. WHY 54

  55. CONSENSUS IS HARD 55

  56. AVOID COORDINATION 56

  57. WEAK CONSISTENCY HIGHER AVAILABILITY 57

  58. CONCLUSION 58

  59. YOU’RE BUILDING A DISTRIBUTED SYSTEM 59

  60. OR-SET IS ONE EXAMPLE 60

  61. COUNTERS MAPS SETS REGISTERS BOOLEANS 61

  62. LOOK AT EXISTING WORK 62

  63. 63 PAPEC 2014

  64. SYNCFREE.LIP6.FR 64

  65. THINKDISTRIBUTED.IO 65

  66. FIN 66