Slide 1
Slide 1 text
No content
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
List (sequence, array) CRDTs: WOOT, Treedoc, Logoot,
RGA, Causal Trees, LSEQ, …
Slide 35
Slide 35 text
List (sequence, array) CRDTs: WOOT, Treedoc, Logoot,
RGA, Causal Trees, LSEQ, …
Emulating “move” as “delete-and-reinsert”:
concurrent moves of the same item
à duplication
Slide 36
Slide 36 text
Concurrent move of the same item to different
positions — what should happen?
Slide 37
Slide 37 text
Converge to one of the destination positions
(pick one arbitrarily but deterministically)
Slide 38
Slide 38 text
“pick one arbitrarily”
= last-writer wins register!
posphone joe
:= “head of the list”
merge
posphone joe
== “head of the list”
posphone joe
:= “after buy milk”
Slide 39
Slide 39 text
List CRDT with move operation
posphone joe
:= “after buy milk”
need one register per list item
state = AWSet({ (v1
, LWWRegister(p1
)),
(v2
, LWWRegister(p2
)),
… })
need a stable way of referencing list positions
Treedoc: path through binary tree
Logoot: list of (integer, replicaID) pairs
RGA: s4vector
Causal Trees: logical timestamp
etc…
Composition of any list CRDT + AWSet +
LWWRegister = another CRDT
Slide 40
Slide 40 text
Moving ranges of elements
Slide 41
Slide 41 text
Moving ranges of elements
Slide 42
Slide 42 text
Desired outcome
Slide 43
Slide 43 text
Actual outcome
Fixing this: an open problem!
Slide 44
Slide 44 text
No content
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
Concurrent moves of same node
Slide 47
Slide 47 text
Concurrent moves of same node
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
Moving A into B, and B into A
Slide 51
Slide 51 text
Moving A into B, and B into A
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
No content
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
No content
Slide 56
Slide 56 text
No content
Slide 57
Slide 57 text
No content
Slide 58
Slide 58 text
No content
Slide 59
Slide 59 text
No content
Slide 60
Slide 60 text
No content
Slide 61
Slide 61 text
No content
Slide 62
Slide 62 text
No content
Slide 63
Slide 63 text
No content
Slide 64
Slide 64 text
No content
Slide 65
Slide 65 text
No content
Slide 66
Slide 66 text
No content
Slide 67
Slide 67 text
File size File size (gzipped)
Full document history,
JSON format
146,406,415 bytes 6,132,895 bytes
Full document history,
custom binary format
695,298 bytes 302,067 bytes
Document history with
cursor movements omitted
570,992 bytes 214,889 bytes
CRDT document with
editing history omitted
228,153 bytes 114,821 bytes
CRDT document with
tombstones removed
154,418 bytes 63,249 bytes
Baseline: plain text with
no CRDT metadata
104,852 bytes 27,569 bytes
48%
48%
150%
22%
211 x
Benchmark data: keystroke-by-keystroke editing trace of a text file (LaTeX source of
a research paper) containing 182,315 single-character insertions, 77,463 single-
character deletions, and 102,049 cursor movements.
Compressing CRDT metadata in Automerge
Slide 68
Slide 68 text
No content
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
No content
Slide 72
Slide 72 text
No content
Slide 73
Slide 73 text
No content
Slide 74
Slide 74 text
No content
Slide 75
Slide 75 text
Text editing CRDTs:
Logoot: Stéphane Weiss, Pascal Urso, and Pascal Molli: “Logoot: A Scalable
Optimistic Replication Algorithm for Collaborative Editing on P2P
Networks,” ICDCS 2009.
LSEQ: Brice Nédelec, Pascal Molli, Achour Mostefaoui, and Emmanuel
Desmontils: “LSEQ: an Adaptive Structure for Sequences in Distributed
Collaborative Editing,” DocEng 2013.
RGA: Hyun-Gul Roh, Myeongjae Jeon, Jin-Soo Kim, and Joonwon Lee:
“Replicated abstract data types: Building blocks for collaborative
applications,” Journal of Parallel and Distributed Computing, 71(3):354–368,
2011.
Treedoc: Nuno Preguiça, Joan Manuel Marques, Marc Shapiro, and Mihai Letia: “A
Commutative Replicated Data Type for Cooperative Editing,” ICDCS 2009.
WOOT: Gérald Oster, Pascal Urso, Pascal Molli, and Abdessamad Imine: “Data
consistency for P2P collaborative editing,” CSCW 2006.
Astrong
: Hagit Attiya, Sebastian Burckhardt, Alexey Gotsman, Adam Morrison,
Hongseok Yang, and Marek Zawirski: “Specification and Complexity of
Collaborative Text Editing,” PODC 2016.
Slide 76
Slide 76 text
More details in these related publications:
Interleaving anomaly: Martin Kleppmann, Victor B. F. Gomes, Dominic P. Mulligan,
and Alastair R. Beresford: “Interleaving anomalies in collaborative text editors”.
PaPoC 2019.
Proof of no interleaving in RGA: Martin Kleppmann, Victor B F Gomes, Dominic P
Mulligan, and Alastair R Beresford: “OpSets: Sequential Specifications for
Replicated Datatypes,” https://arxiv.org/abs/1805.04263, May 2018.
Moving list items: Martin Kleppmann: “Moving Elements in List CRDTs”. PaPoC 2020.
Move operation in CRDT trees: Martin Kleppmann, Dominic P. Mulligan, Victor B. F.
Gomes, and Alastair R. Beresford: “A highly-available move operation for
replicated trees and distributed filesystems”. Preprint,
https://martin.kleppmann.com/papers/move-op.pdf
Reducing metadata overhead: Martin Kleppmann: “Experiment: columnar data
encoding for Automerge”, 2019. https://github.com/automerge/automerge-
perf/blob/master/columnar/README.md
Local-first software: Martin Kleppmann, Adam Wiggins, Peter van Hardenberg, and
Mark McGranaghan: “Local-first software: You own your data, in spite of the
cloud”. Onward! 2019. https://www.inkandswitch.com/local-first.html
Slide 77
Slide 77 text
Thanks!
• Martin’s email: [email protected]
• Martin on Twitter: https://twitter.com/martinkl
• Martin’s book: https://dataintensive.net/
• CRDT resources: https://crdt.tech/
• Automerge: https://github.com/automerge/automerge
Thank you to these organisations for supporting this work!