Convergent/Divergent

Presented at EmberConf 2014

Transcript

1. CONVERGENT DIVERGENT ⊤ FAITS PRÉCIS 1
2. @ C M E I K CMEIKLEJOHN B A S H O GEORGIA TECH TDISTRIBUTED 2
3. SYNCFREE 3 Funded by the European Union
4. THE MOTIVATION SOME THEORY A SOLUTION 4
5. MOTIVATION EMBER DEV RICON 2013 5
6. INTRODUCTION 6
7. YOU’RE BUILDING A DISTRIBUTED SYSTEM 7
8. PUSH STATE TO CLIENT 8
9. REGARDLESS OF DATABASE ISOLATION 9
10. REGARDLESS OF PUSHING STATE TO CLIENTS 10
11. DROPPED MESSAGES REORDERED MESSAGES RACE CONDITIONS PARTIAL FAILURES CUSTOM MERGES 11
12. DROPPED MESSAGES REORDERED MESSAGES TCP INCAST TCP SLOW-START NAGLE’S ALGORITHM 12
13. RACE CONDITIONS PARTIAL FAILURES 13
14. CUSTOM MERGES 14
15. DISTRIBUTED S Y S T E M S T H E O R Y 15
16. CONSISTENCY 16
17. The data consistency model specifies a contract between programmer and system, wherein the system guarantees that if the programmer follows the rules, memory will be consistent and the results of memory operations will be predictable. ! Wikipedia, Consistency Model 17
18. STRICT LINEAR
 IZABLE 18
19. EVEN
 TUAL 19
20. CAUSAL 20
21. SAFETY VS L I V E N E S S 21
22. CONSENSUS 22
23. A fundamental problem in distributed computing is to achieve overall system reliability in the presence of a number of faulty processes. This often requires processes to agree on some data value that is needed during computation. Examples of applications of consensus include whether to commit a transaction to a database, agreeing on the identity of a leader, state machine replication, and atomic broadcasts. ! Wikipedia, Consensus 23
24. TERMINATION
 AGREE MENT V A L I D I T Y 24
25. T W O GENERALS 25
26. BYZANTINE GENERALS 26
27. 2 PC 3 PC PAXOS R A F T 27
28. CAUSALITY 28
29. TIME CLOCKS AND THE ORDERING OF EVENTS IN A DISTRIBUTED SYSTEM 29
30. VECTOR CLOCKS 30
31. VERSION VECTORS 31
32. CHARRON - B O S T 32
33. D O T T E D VERSION VECTORS 33
34. CONFLICT FREE R E P L I C A T E D DATA TYPES 34
35. C O N V E R G E N T C O M M U T A T I V E R E P L I C A T E D DATA TYPES 35
36. TWO FLAVORS STATE BASED OPERATIONS BASED 36
37. MONOTONICITY 37
38. ASSOCIATIVITY (X · Y) · Z = X · (Y · Z) 38
39. COMMUTATIVITY X · Y = Y · X 39
40. IDEMPOTENCE X · X = X 40
41. BOUNDED JOIN S E M I L A T T I C E S 41
42. 42 b a c a, b a, c a, b, c SET; MERGE: UNION b, c
43. 43 3 5 7 5 7 7 INC NAT; MERGE: MAX
44. 44 F F T F T T BOOL; MERGE: OR
45. STATE BASED CVRDT EXAMPLE THE OR-SET 45
46. [ [{1, a}], [] ] [ [{1, a}], [] ]
47. [ [{1, a}], [] ] [ [{1, a}], [] ] [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ]
48. [ [{1, a}], [] ] [ [{1, a}], [] ] [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ]
49. [ [{1, a}], [] ] [ [{1, a}], [] ] [ [{1, a}], [{1, a}] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ] [ [{1, a}, {2, a}], [{1, a}] ]
50. [ [{1, a}], [] ] [ [{1, a}], [] ]
51. [ [{1, a}], [] ] [ [{1, a}], [] ] [ [{1, a}, {2, b}], [] ]
52. [ [{1, a}], [] ] [ [{1, a}], [] ] [ [{1, a}, {2, b}], [] ] [ [{1, a}], [{1, a}] ]
53. [ [{1, a}], [] ] [ [{1, a}], [] ] [ [{1, a}, {2, b}], [] ] [ [{1, a}], [{1, a}] ] [ [{1, a}, {2, b}], [{1, a}] ]
54. WHY 54
55. CONSENSUS IS HARD 55
56. AVOID COORDINATION 56
57. WEAK CONSISTENCY HIGHER AVAILABILITY 57
58. CONCLUSION 58
59. YOU’RE BUILDING A DISTRIBUTED SYSTEM 59
60. OR-SET IS ONE EXAMPLE 60
61. COUNTERS MAPS SETS REGISTERS BOOLEANS 61
62. LOOK AT EXISTING WORK 62
63. 63 PAPEC 2014
64. SYNCFREE.LIP6.FR 64
65. THINKDISTRIBUTED.IO 65
66. FIN 66