Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automerge: a new foundation for collaboration software

Automerge: a new foundation for collaboration software

Slides from a talk given on 25 Nov 2021 at the University of Cambridge SRG Seminars
https://martin.kleppmann.com/2021/11/25/srg-seminars.html
https://www.youtube.com/watch?v=Qytg0Ibet2E

Abstract:

Software for real-time collaboration, such as Google Docs, Overleaf, Figma, or Trello, has enabled many people around the world to continue working remotely during the pandemic. While this software is very valuable, it is also fragile because it relies on centralised, trusted cloud services. If the company providing the cloud software goes out of business, or decides to suspend your account, the software stops working, and you are locked out of all of the documents and data you ever created with that software.

Local-first software is an effort to make collaboration software less dependent on cloud services, and Automerge is an open-source library for realising local-first software. Automerge provides a shared JSON-like data structure that several users can update concurrently, and which automatically merges those updates into a consistent result. It provides data formats for efficiently storing this data and syncing it between users. It seamlessly supports both offline work and live real-time collaboration while online.

Automerge is the result of years of research on Conflict-free Replicated Data Types (CRDTs). In this talk we will explore both the theoretical foundations of CRDTs, their practical applications, and the ongoing research in this area.

Martin Kleppmann

November 25, 2021
Tweet

More Decks by Martin Kleppmann

Other Decks in Research

Transcript

  1. Thank you to my supporters
    Crowdfunding supporters include: Ably, Adrià Arcarons, Chet
    Corcos, Macrometa, Mintter, David Pollak, RelationalAI,
    SoftwareMill, Talent Formation Network, Adam Wiggins
    https://www.patreon.com/martinkl

    View full-size slide

  2. File size
    JSON (uncompressed) 146,406,415 bytes
    JSON + gzip 6,132,895 bytes
    Custom binary format 302,067 bytes
    20 x
    Benchmark data: keystroke-by-keystroke editing trace of a text file (LaTeX
    source of a research paper) containing 182,315 single-character insertions,
    77,463 single-character deletions, and 102,049 cursor movements.
    Total: 361,827 operations.
    Compressing CRDT metadata in Automerge
    24 x
    ≈ 0.8 bytes per operation!

    View full-size slide

  3. File size File size (gzipped)
    Full document history,
    JSON format
    146,406,415 bytes 6,132,895 bytes
    Full document history,
    custom binary format
    695,298 bytes 302,067 bytes
    Document history with
    cursor movements omitted
    570,992 bytes 214,889 bytes
    CRDT document with
    editing history omitted
    228,153 bytes 114,821 bytes
    CRDT document with
    tombstones removed
    154,418 bytes 63,249 bytes
    Baseline: plain text with
    no CRDT metadata
    104,852 bytes 27,569 bytes
    48%
    48%
    150%
    22%
    211 x
    Benchmark data: keystroke-by-keystroke editing trace of a text file (LaTeX source of
    a research paper) containing 182,315 single-character insertions, 77,463 single-
    character deletions, and 102,049 cursor movements.
    Compressing CRDT metadata in Automerge

    View full-size slide

  4. Resources
    Automerge https://github.com/automerge/automerge
    Publications https://martin.kleppmann.com/#publications
    Email [email protected]
    Twitter @martinkl
    Book http://dataintensive.net/
    Support me https://www.patreon.com/martinkl
    Huge thanks to my collaborators and the Automerge community
    and contributors, especially Peter van Hardenberg, Orion Henry,
    Alex Good, Andrew Jeffery, Herb Caudill, and many others!

    View full-size slide