Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mayday, we're syncing!

Mayday, we're syncing!

We live in the time of mobile-first development, where many developers keep facing the same problem over and over again: How do we synchronize users’ data across their multitude of devices? Phones can be temporarily offline, tablets only work with WiFi, and browsers may crash. Under no circumstances should users have to resolve merge conflicts, but they do expect history and rollbacks. Can we satisfy all these constraints? “Conflict-free replicated datatypes” are the solution. A rather young technology, poised to solve synchronisation issues once and for all. Come along to a stroll through research & practice of this emerging ecosystem.

Lars Hupel

June 21, 2022
Tweet

More Decks by Lars Hupel

Other Decks in Programming

Transcript

  1. Mayday, we're syncing! REACT VIENNA / 2022-06-21 LARS HUPEL @LARSR_H

    LUCAS DOHMEN @MOONBEAMLABS
  2. None
  3. None
  4. When everything's in the cloud • • • ... we

    are dependent on the Internet ... we are limited in how we can work ... we are excluding people
  5. Working Offline? • • in the Olden Days, we didn't

    always have a connection local file storage, manual sync
  6. Offline-first is the standard for development • • • we

    like working offline no one wants to use Subversion nowadays why do we ask our users for connectivity?
  7. None
  8. Hello world! Hello world Hello, world!

  9. None
  10. None
  11. None
  12. Lattices • • mathematical structure abstract definition of ordering

  13. interface Lattice<T> { join(left: T, right: T): T; lteq(left: T,

    right: T): boolean; } // 1. join(x, y) == join(y, x) // 2. join(x, x) == x // 3. join(x, join(y, z)) == join(join(x, y), z) // ...
  14. https://commons.wikimedia.org/wiki/File:Lattice_of_the_divisibility_of_60.svg

  15. const divisionLattice: Lattice<number> = { join(left: number, right: number) {

    return leastCommonMultiple(left, right); }, lteq(left: number, right: number) { return right % left == 0; } }
  16. CRDTs Conflict-Free Replicated Datatypes

  17. Example: G-Sets {} {Skyr} {Cereal} {Skyr, Grapefruit} {Skyr, Cereal, Grapefruit}

  18. Tombstones • • deletion is not monotonic ☹️ solution: mark

    as deleted
  19. Example: 2P-Sets {Skyr, Cereal, Grapefruit} {Skyr, Cereal, Grapefruit} {Skyr, Apple,

    Cereal, Grapefruit} {Skyr, Cereal, Grapefruit, Tea} {Skyr, Apple, Cereal, Grapefruit, Tea}
  20. Vector Clocks • • • each of the n devices

    gets its own timestamp the global timestamp for all devices is composed of those n timestamps it represents the last known state
  21. Vector Clocks [1, 0] [2, 0] [0, 1] [0, 0]

    [2, 1] [T₁, T₂]
  22. T₁ = 0 T₂ = 0 Example: MV-Register x =

    0 x = 1 T₁ = 1 T₂ = 0 👩 x = 0 T₁ = 0 T₂ = 0 x = 1 T₁ = 1 T₂ = 0 x = 1 T₁ = 1 T₂ = 0 x = 2 T₁ = 2 T₂ = 0 x = 3 T₁ = 1 T₂ = 1 💣 update sync update update x = 0 🧑
  23. “Any two object replicas of a CvRDT eventually converge, assuming

    the system transmits payload infinitely often between pairs of replicas over eventually-reliable point-to-point channels.”
  24. None
  25. None
  26. Automerge • • • JavaScript Library for Browser & Node

    offers JSON documents as datatype 💡they are CRDTs
  27. Usage Requests pro Sekunde doc1 = Automerge.from({ todos: [] })

    doc2 = Automerge.change(doc1, 'Add cereal', doc => { doc.todos.push({ text: 'Cereal', done: false }) }) doc3 = Automerge.change(doc1, 'Add skyr', doc => { doc.todos.push({ text: 'Skyr', done: false }) }) doc4 = Automerge.merge(doc2, doc3) // contains traces of cereal & skyr
  28. Demo Shopping List

  29. None
  30. https://github.com/innoq/groceries

  31. Server CRDT Browser List as HTML POST the new item

  32. Server CRDT Browser Changes via SSE POST the changeset CRDT

  33. Subscribe via SSE const source = new EventSource("/stream"); source.addEventListener("message", (ev)

    => { list.applyChanges(ev.data); });
  34. Push Changes class List { constructor() { // ... this.changes

    = []; this.interval = setInterval(this.pushChanges.bind(this), 1000); } }
  35. Push Changes async pushChanges() { try { await fetch("/", {

    method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ changes: this.changes }), }); this.changes = []; } catch(e) { /* ... */ } }
  36. More Features • • • • • • other datatypes

    Counter Text (collaborative editing) Peritext table P2P with Hypermerge
  37. Should the app work offline Do whatever you want <3

    Read or write? Check out Service Workers There is no way around CSR & CRDTs No Yes only read both
  38. Q&A Lucas Dohmen lucas.dohmen@innoq.com @moonbeamlabs www.innoq.com innoQ Deutschland GmbH Krischerstr.

    100 40789 Monheim +49 2173 333660 Ohlauer Str. 43 10999 Berlin Ludwigstr. 180E 63067 Offenbach Kreuzstr. 16 80331 München Hermannstr. 13 20095 Hamburg Erftstr. 15-17 50672 Köln Königstorgraben 11 90402 Nürnberg Lars Hupel lars.hupel@innoq.com @larsr_h