Syncing data across user devices for distributed collaboration

Slides from a presentation given on 12 July 2019 at the Hydra distributed computing conference, St. Petersburg, Russia.
https://martin.kleppmann.com/2019/07/11/hydra-distributed.html

Abstract:

Google Docs and similar web apps are very convenient if several people need to work together on a document: all users can simultaneously edit the shared document without having to send files back and forth by email. From a distributed systems point of view, real-time collaboration is a replication problem: each user has a replica of the shared document on their device, which they can modify locally, and any changes are sent over the network to their collaborators.

Users may edit a document while offline, causing their replica to become temporarily inconsistent with the others. As they re-synchronise with others, conflicts need to be resolved so that all collaborators end up in a consistent state. Conflict-free Replicated Data Types (CRDTs), first proposed in 2011, are increasingly being used to merge different users’ versions of a document in collaborative software.

This talk will give an update on the latest research on CRDTs, and our open source implementation Automerge. We will also discuss efforts to build “local-first software”, a new generation of collaborative software, on top of these technologies.

Transcript

26. https://killedbygoogle.com
28. Cuneiform script on clay tablet, ca. 3000 BCE (image from Wikimedia Commons)
50. For details, see our paper at https://doi.org/10.1145/3133933 Victor B. F. Gomes, Martin Kleppmann, Dominic P. Mulligan, and Alastair R. Beresford: Verifying Strong Eventual Consistency in Distributed Systems. PACMPL 1(OOPSLA), 2017.
89. Trellis, a Trello clone based on Automerge: https://github.com/automerge/trellis Joint work with Orion Henry, Peter van Hardenberg, Roshan Choxi, and Adam Wiggins.
91. Pixelpusher, a collaborative pixel art editor: https://github.com/automerge/pixelpusher Created by Javier Valencia, Jeff Peterson, Peter van Hardenberg, and Jim Pick.
95. MPL, a WebRTC network layer for Automerge: https://github.com/automerge/mpl Joint work with Orion Henry, Peter van Hardenberg, Roshan Choxi, and Adam Wiggins.
96. Hypermerge, a peer-to-peer network layer: https://github.com/automerge/hypermerge Created by Jim Pick, Jeff Peterson, and Peter van Hardenberg.
98. Resources • Martin’s email: mk428@cl.cam.ac.uk • Martin on Twitter: @martinkl • Martin’s book: http://dataintensive.net/ • Local-first software: https://www.inkandswitch.com/local-first.html • Automerge: https://github.com/automerge/automerge • Trellis: https://github.com/automerge/trellis • Pixelpusher: https://github.com/automerge/pixelpusher • PushPin: https://inkandswitch.github.io/pushpin/ • Capstone: https://www.inkandswitch.com/capstone-manuscript.html • Hypermerge: https://github.com/automerge/hypermerge • Dat / Hypercore: https://datproject.org/ • Proving CRDTs correct: https://doi.org/10.1145/3133933 • JSON CRDT: http://arxiv.org/abs/1608.03960
99. Available from Питер book stall Book signing 11:30 in Discussion Zone 1