Save 37% off PRO during our Black Friday Sale! »

The Final Causal Frontier

The Final Causal Frontier

Cyberspace… the final causal frontier. These are the voyages of the software enterprise. Its continuing mission: to explore strange new datacenters…to seek out new sites and new applications…to boldly code where no coder has gone before!

Ce461b57b6a1f65ce5b6cc1c124673e3?s=128

Sean Cribbs

April 23, 2015
Tweet

Transcript

  1. T H E F I N A L C A

    U S A L F R O N T I E R S E A N C R I B B S C R A F T C O N F E R E N C E , 2 3 A P R I L 2 0 1 5
  2. None
  3. M O B I L E O F F L

    I N E W E B G L O B A L P O P S I N T E R N E T O F T H I N G S
  4. M O B I L E O F F L

    I N E W E B G L O B A L P O P S I N T E R N E T O F T H I N G S C L I E N T S E R V E R
  5. C R D T S A R E A S

    O L U T I O N
  6. C O R E C O N C E P

    T S
  7. M O N O T O N I C I

    T Y A N D L AT T I C E S C O R E C O N C E P T S :
  8. M O N O T O N I C I

    T Y
  9. B O U N D E D J O I

    N S E M I - L AT T I C E S
  10. B O U N D E D J O I

    N S E M I - L AT T I C E S A ⊔ B ⊥
  11. B O U N D E D J O I

    N S E M I - L AT T I C E S A ⊔ B ⊥ mappend mempty
  12. B O U N D E D J O I

    N S E M I - L AT T I C E S A ⊔ B ⊥ A ⊔ B = C 㱺 A⊑C and B⊑C mappend mempty
  13. V E R S I O N V E C

    T O R S A N D D O T S C O R E C O N C E P T S :
  14. D E T E C T I O N O

    F M U T U A L I N C O N S I S T E N C Y I N D I S T R I B U T E D S Y S T E M S D . S T O T T PA R K E R J R , E T A L , 1 9 8 3
  15. V E R S I O N V E C

    T O R S A B C
  16. V E R S I O N V E C

    T O R S A B C
  17. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1
  18. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ]
  19. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ]
  20. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ]
  21. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ]
  22. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ]
  23. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ] A B C B : 1 C : 1 [ A:0, B:1, C:1 ]
  24. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ] A B C B : 1 C : 1 [ A:0, B:1, C:1 ] Dominates
  25. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ] B A C A : 1 B : 1 C : 1 C : 2 [ A:1, B:1, C:2 ] Dominates
  26. V E R S I O N V E C

    T O R S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ] B A C A : 1 B : 1 C : 1 C : 2 [ A:1, B:1, C:2 ] Dominates Concurrent
  27. V E R S I O N V E C

    T O R S & D O T S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ]
  28. V E R S I O N V E C

    T O R S & D O T S A B C A : 1 A : 2 B : 1 C : 1 [ A:2, B:1, C:1 ] C : 2 DOT
  29. [ A:2, B:1, C:2 ] V E R S I

    O N V E C T O R S & D O T S A B C C A : 1 A : 2 B : 1 C : 1 C : 2 DOT
  30. C O N F L I C T- F R

    E E R E P L I C AT E D D ATAT Y P E S C O R E C O N C E P T S :
  31. C O N V E R G E N T

    S TAT E / 
 C O M M U TAT I V E O P E R AT I O N S
  32. S T R O N G E V E N

    T U A L C O N S I S T E N C Y
  33. T H E S TAT E O F T H

    E A R T C A T C H I N G U P W I T H
  34. L AY E R E D A R C H

    I T E C T U R E D I A G R A M K E R N E L A P P L I C AT I O N S U S E R S PA C E
  35. L AY E R E D A R C H

    I T E C T U R E D I A G R A M K E R N E L A P P L I C AT I O N S U S E R S PA C E M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S
  36. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S
  37. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S D Y N A M O
  38. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S D Y N A M O M V / LW W
  39. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S D Y N A M O M V / LW W A D H O C / A P P E N D O N LY
  40. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S
  41. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B
  42. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S
  43. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S O R S W O T
  44. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S O R S W O T M A P
  45. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S O R S W O T M A P S W I F T C L O U D
  46. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S O R S W O T M A P S W I F T C L O U D Δ - C R D T S
  47. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S O R S W O T M A P S W I F T C L O U D Δ - C R D T S I N V C R D T S
  48. L AY E R E D A R C H

    I T E C T U R E D I A G R A M M I D D L E WA R E A P P L I C AT I O N S D ATAT Y P E S T R C B D V V S E T S O R S W O T M A P S W I F T C L O U D L A S P Δ - C R D T S I N V C R D T S
  49. N E W D ATAT Y P E S S

    E T S , R E G I S T E R S , M A P S , D E LTA S , I N VA R I A N T S
  50. O P T I M I Z E D O

    R - S E T B I E N I U S A , E T A L . O B S E R V E D - R E M O V E S E T W I T H O U T T O M B S T O N E S
  51. O R - S E T G R O W

    S Q U I C K LY
  52. O R - S E T G R O W

    S Q U I C K LY [ (i, Ai , Ri ) , (j, Aj , Rj ), … ]
  53. O R - S E T G R O W

    S Q U I C K LY [ (i, Ai , Ri ) , (j, Aj , Rj ), … ] []
  54. O R - S E T G R O W

    S Q U I C K LY [ (i, Ai , Ri ) , (j, Aj , Rj ), … ] [] [(1, {a}, {})] A D D 1
  55. O R - S E T G R O W

    S Q U I C K LY [ (i, Ai , Ri ) , (j, Aj , Rj ), … ] [] [(1, {a}, {})] A D D 1 [(1, {a}, {}), (2, {b}, {})] A D D 2
  56. O R - S E T G R O W

    S Q U I C K LY [ (i, Ai , Ri ) , (j, Aj , Rj ), … ] [] [(1, {a}, {})] A D D 1 [(1, {a}, {}), (2, {b}, {})] A D D 2 [(1, {a}, {a}), (2, {b}, {})] R E M O V E 1
  57. O P T I M I Z E D O

    R - S E T ( O R S W O T )
  58. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones VV ⊠ [ (i, Ai ), (j, Aj ), …]
  59. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones • Merge is more complicated VV ⊠ [ (i, Ai ), (j, Aj ), …]
  60. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones • Merge is more complicated • General technique! VV ⊠ [ (i, Ai ), (j, Aj ), …]
  61. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones • Merge is more complicated • General technique! VV ⊠ [ (i, Ai ), (j, Aj ), …] [] ⊠ []
  62. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones • Merge is more complicated • General technique! VV ⊠ [ (i, Ai ), (j, Aj ), …] [] ⊠ [] [a:1] ⊠ [(1, {a:1})] A D D 1
  63. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones • Merge is more complicated • General technique! VV ⊠ [ (i, Ai ), (j, Aj ), …] [] ⊠ [] [a:1] ⊠ [(1, {a:1})] A D D 1 [a:1,b:1] ⊠ [(1, {a:1}), (2, {b:1})] A D D 2
  64. O P T I M I Z E D O

    R - S E T ( O R S W O T ) • Version vector plus latest dot replaces unique ids and tombstones • Merge is more complicated • General technique! VV ⊠ [ (i, Ai ), (j, Aj ), …] [] ⊠ [] [a:1] ⊠ [(1, {a:1})] A D D 1 [a:1,b:1] ⊠ [(1, {a:1}), (2, {b:1})] A D D 2 [a:1,b:2] ⊠ [(2, {b:1})] R E M O V E 1
  65. D O T T E D V E R S

    I O N V E C T O R S E T S A L M E I D A , E T A L . S C A L A B L E A N D A C C U R A T E C A U S A L I T Y T R A C K I N G F O R E V E N T U A L LY C O N S I S T E N T S T O R E S
  66. P R O B L E M S W I

    T H R E G I S T E R S • Last-Write-Wins : loses data • Causal history MV : write-identifier explosion • Client-identified MV : actor explosion • Server-identified MV : sibling explosion
  67. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+
  68. a : 3 / C a r r i e

    D O T T E D V E R S I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d a : 2 / B o b [ a : 4 , b : 5 ]
  69. a : 3 / C a r r i e

    [ a : 3 , b : 2 ] D O T T E D V E R S I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d a : 2 / B o b [ a : 4 , b : 5 ] E u g e n e
  70. [ a : 3 , b : 2 ] D

    O T T E D V E R S I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 4 , b : 5 ] E u g e n e
  71. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] a : 5 / E u g e n e
  72. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 2 / B o b a : 3 / C a r r i e [ a : 4 , b : 1 ] b : 1 / A l i c e a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e
  73. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 2 / B o b a : 3 / C a r r i e [ a : 4 , b : 1 ] a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e
  74. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 2 / B o b a : 3 / C a r r i e [ a : 4 , b : 1 ] a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e
  75. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 2 / B o b a : 3 / C a r r i e [ a : 4 , b : 1 ] a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e
  76. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 2 / B o b a : 3 / C a r r i e [ a : 4 , b : 1 ] a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e
  77. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 3 / C a r r i e [ a : 4 , b : 1 ] a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e
  78. D O T T E D V E R S

    I O N V E C T O R S E T S • Version vector with dots on values • Writes remove “seen” dots • Similar optimization and merge to the ORSWOT • Default KV behavior in Riak 2.0+ a : 4 / D a v i d [ a : 5 , b : 5 ] [ a : 3 , b : 2 ] b : 2 / F re d a : 3 / C a r r i e [ a : 4 , b : 1 ] a : 3 / C a r r i e a : 4 / D a v i d a : 5 / E u g e n e [ a : 4 , b : 2 ]
  79. R I A K 2 . 0 M A P

    B R O W N E T A L . R I A K D T M A P : A C O N V E R G E N T, C O M P O S A B L E R E P L I C A T E D D I C T I O N A RY
  80. R I A K ’ S M A P •

    Recursive dictionary for other state-based CRDTs: 
 counters, sets, registers, flags, maps • Reset-remove semantic on concurrent update/remove • Atomic batch updates to multiple contained types • Similar optimizations/merge to ORSWOT: 
 top-level version vector, per-field dots
  81. D E LTA S TAT E - B A S

    E D C R D T S A L M E I D A , S H O K E R , B A Q U E R O . E F F I C I E N T S TA T E - B A S E D C R D T S B Y D E C O M P O S I T I O N
  82. D E LTA S TAT E - B A S

    E D C R D T S • Combines best of operation- and state-based CRDTS:
 Small payloads over unreliable channels • Delta-mutator produces minimal state representing change • Deltas can be collected into batches
  83. B O U N D E D C O U

    N T E R S B A L E G A S , E T A L . E N F O R C I N G N U M E R I C I N VA R I A N T S O N E V E N T U A L C O N S I S T E N T C L O U D S T O R E S
  84. B O U N D E D C O U

    N T E R S • “Comprehensive” mentions NN-counters may require coordination • Local decrements never exceed local increments • Rights can be transferred to peers (escrow) • Transfer can be coordinated lazily - it’s still a lattice!
  85. B O U N D E D C O U

    N T E R S • “Comprehensive” mentions NN-counters may require coordination • Local decrements never exceed local increments • Rights can be transferred to peers (escrow) • Transfer can be coordinated lazily - it’s still a lattice! A B C {10,0} — 0 0 {5,4} 0 — 0 {30,10} 0 0 —
  86. B O U N D E D C O U

    N T E R S • “Comprehensive” mentions NN-counters may require coordination • Local decrements never exceed local increments • Rights can be transferred to peers (escrow) • Transfer can be coordinated lazily - it’s still a lattice! A B C {10,0} — 0 0 {5,4} 0 — 0 {30,10} 0 0 — A B C {10,0} — 5 0 {5,8} 0 — 0 {30,10} 0 0 — B : D E C R E M E N T 4
  87. M I D D L E WA R E P

    U R E O P - B A S E D , TA G G E D R E L I A B L E C A U S A L B R O A D C A S T
  88. P U R E O P - B A S

    E D C R D T S , 
 TA G G E D R E L I A B L E C A U S A L B R O A D C A S T M A K I N G O P E R A T I O N - B A S E D C R D T S O P E R A T I O N - B A S E D . B A Q U E R O , A L M E I D A , S H O K E R
  89. P U R E O P - B A S

    E D C R D T S • Current operation-based CRDTs conflate delivery concerns with type semantics • Focus on minimal pure state and direct operations
 e.g. integer counter with direct add/subtract • Op-log moved to middleware layer to ensure reliable delivery
  90. TA G G E D R E L I A

    B L E C A U S A L B R O A D C A S T • Uses vector clock per message • Causal stability - all future messages will have dominant clocks • Partially-ordered log of unapplied operations • Semantic compaction of PO-log
  91. A P P L I C AT I O N

    S S W I F T C L O U D , L A S P
  92. S W I F T C L O U D

    Z A W I R S K I , E T A L . S W I F T C L O U D : FA U LT- T O L E R A N T G E O - R E P L I C A T I O N I N T E G R A T E D A L L T H E WA Y T O T H E C L I E N T M A C H I N E
  93. S W I F T C L O U D

    • Introduces causal+ consistency 
 causal snapshot isolation, mergeable transactions • Clients can execute completely offline and synchronize later • Rich Java API with many included CRDTs
  94. L A S P : L AT T I C

    E P R O C E S S I N G M E I K L E J O H N & VA N R O Y. L A S P : A L A N G U A G E F O R D I S T R I B U T E D , C O O R D I N A T I O N - F R E E P R O G R A M M I N G
  95. L A S P : L AT T I C

    E P R O C E S S I N G • Based on deterministic dataflow programming • Functional: map, filter, fold • Set: product, union, intersection • Safety: monotonic read
  96. None
  97. None
  98. None
  99. T H E F I N A L C A

    U S A L F R O N T I E R W H A T ’ S N E X T O N
  100. T H E F I N A L C A

    U S A L F R O N T I E R • Applications at the edge • Increased robustness • Reduced metadata and communication • Larger applications
  101. A C K N O W L E D G

    M E N T S • Basho: Russell Brown, Christopher Meiklejohn • SyncFree FP7: Marc Shapiro, Nuno Preguiça, Carlos Baquero, Annette Bieniusa, João Leitão, Peter Van Roy, Valter Balegas • NASA Astronomy Picture of the Day (APOD)
  102. T H A N K S