Automerge: a new foundation for collaboration software

Slides from a talk given on 25 Nov 2021 at the University of Cambridge SRG Seminars
https://martin.kleppmann.com/2021/11/25/srg-seminars.html
https://www.youtube.com/watch?v=Qytg0Ibet2E

Abstract:

Software for real-time collaboration, such as Google Docs, Overleaf, Figma, or Trello, has enabled many people around the world to continue working remotely during the pandemic. While this software is very valuable, it is also fragile because it relies on centralised, trusted cloud services. If the company providing the cloud software goes out of business, or decides to suspend your account, the software stops working, and you are locked out of all of the documents and data you ever created with that software.

Local-first software is an effort to make collaboration software less dependent on cloud services, and Automerge is an open-source library for realising local-first software. Automerge provides a shared JSON-like data structure that several users can update concurrently, and which automatically merges those updates into a consistent result. It provides data formats for efficiently storing this data and syncing it between users. It seamlessly supports both offline work and live real-time collaboration while online.

Automerge is the result of years of research on Conflict-free Replicated Data Types (CRDTs). In this talk we will explore both the theoretical foundations of CRDTs, their practical applications, and the ongoing research in this area.

Transcript

2. Thank you to my supporters Crowdfunding supporters include: Ably, Adrià Arcarons, Chet Corcos, Macrometa, Mintter, David Pollak, RelationalAI, SoftwareMill, Talent Formation Network, Adam Wiggins https://www.patreon.com/martinkl
126. File size JSON (uncompressed) 146,406,415 bytes JSON + gzip 6,132,895 bytes Custom binary format 302,067 bytes 20 x Benchmark data: keystroke-by-keystroke editing trace of a text file (LaTeX source of a research paper) containing 182,315 single-character insertions, 77,463 single-character deletions, and 102,049 cursor movements. Total: 361,827 operations. Compressing CRDT metadata in Automerge 24 x ≈ 0.8 bytes per operation!
127. File size File size (gzipped) Full document history, JSON format 146,406,415 bytes 6,132,895 bytes Full document history, custom binary format 695,298 bytes 302,067 bytes Document history with cursor movements omitted 570,992 bytes 214,889 bytes CRDT document with editing history omitted 228,153 bytes 114,821 bytes CRDT document with tombstones removed 154,418 bytes 63,249 bytes Baseline: plain text with no CRDT metadata 104,852 bytes 27,569 bytes 48% 48% 150% 22% 211 x Benchmark data: keystroke-by-keystroke editing trace of a text file (LaTeX source of a research paper) containing 182,315 single-character insertions, 77,463 single- character deletions, and 102,049 cursor movements. Compressing CRDT metadata in Automerge
136. Resources Automerge https://github.com/automerge/automerge Publications https://martin.kleppmann.com/#publications Email martin@kleppmann.com Twitter @martinkl Book http://dataintensive.net/ Support me https://www.patreon.com/martinkl Huge thanks to my collaborators and the Automerge community and contributors, especially Peter van Hardenberg, Orion Henry, Alex Good, Andrew Jeffery, Herb Caudill, and many others!