Mayday, we're syncing!

We live in the time of mobile-first development, where many developers keep facing the same problem over and over again: How do we synchronize users’ data across their multitude of devices? Phones can be temporarily offline, tablets only work with WiFi, and browsers may crash. Under no circumstances should users have to resolve merge conflicts, but they do expect history and rollbacks. Can we satisfy all these constraints? “Conflict-free replicated datatypes” are the solution. A rather young technology, poised to solve synchronisation issues once and for all. Come along to a stroll through research & practice of this emerging ecosystem.

Transcript

1. Mayday, we're syncing! REACT VIENNA / 2022-06-21 LARS HUPEL @LARSR_H LUCAS DOHMEN @MOONBEAMLABS
4. When everything's in the cloud • • • ... we are dependent on the Internet ... we are limited in how we can work ... we are excluding people
5. Working Offline? • • in the Olden Days, we didn't always have a connection local file storage, manual sync
6. Offline-first is the standard for development • • • we like working offline no one wants to use Subversion nowadays why do we ask our users for connectivity?
8. Hello world! Hello world Hello, world!
12. Lattices • • mathematical structure abstract definition of ordering
13. interface Lattice<T> { join(left: T, right: T): T; lteq(left: T, right: T): boolean; } // 1. join(x, y) == join(y, x) // 2. join(x, x) == x // 3. join(x, join(y, z)) == join(join(x, y), z) // ...
14. https://commons.wikimedia.org/wiki/File:Lattice_of_the_divisibility_of_60.svg
15. const divisionLattice: Lattice<number> = { join(left: number, right: number) { return leastCommonMultiple(left, right); }, lteq(left: number, right: number) { return right % left == 0; } }
16. CRDTs Conflict-Free Replicated Datatypes
17. Example: G-Sets {} {Skyr} {Cereal} {Skyr, Grapefruit} {Skyr, Cereal, Grapefruit}
18. Tombstones • • deletion is not monotonic ☹️ solution: mark as deleted
19. Example: 2P-Sets {Skyr, Cereal, Grapefruit} {Skyr, Cereal, Grapefruit} {Skyr, Apple, Cereal, Grapefruit} {Skyr, Cereal, Grapefruit, Tea} {Skyr, Apple, Cereal, Grapefruit, Tea}
20. Vector Clocks • • • each of the n devices gets its own timestamp the global timestamp for all devices is composed of those n timestamps it represents the last known state
21. Vector Clocks [1, 0] [2, 0] [0, 1] [0, 0] [2, 1] [T₁, T₂]
22. T₁ = 0 T₂ = 0 Example: MV-Register x = 0 x = 1 T₁ = 1 T₂ = 0 👩 x = 0 T₁ = 0 T₂ = 0 x = 1 T₁ = 1 T₂ = 0 x = 1 T₁ = 1 T₂ = 0 x = 2 T₁ = 2 T₂ = 0 x = 3 T₁ = 1 T₂ = 1 💣 update sync update update x = 0 🧑
23. “Any two object replicas of a CvRDT eventually converge, assuming the system transmits payload infinitely often between pairs of replicas over eventually-reliable point-to-point channels.”
26. Automerge • • • JavaScript Library for Browser & Node offers JSON documents as datatype 💡they are CRDTs
27. Usage Requests pro Sekunde doc1 = Automerge.from({ todos: [] }) doc2 = Automerge.change(doc1, 'Add cereal', doc => { doc.todos.push({ text: 'Cereal', done: false }) }) doc3 = Automerge.change(doc1, 'Add skyr', doc => { doc.todos.push({ text: 'Skyr', done: false }) }) doc4 = Automerge.merge(doc2, doc3) // contains traces of cereal & skyr
28. Demo Shopping List
30. https://github.com/innoq/groceries
31. Server CRDT Browser List as HTML POST the new item
32. Server CRDT Browser Changes via SSE POST the changeset CRDT
33. Subscribe via SSE const source = new EventSource("/stream"); source.addEventListener("message", (ev) => { list.applyChanges(ev.data); });
34. Push Changes class List { constructor() { // ... this.changes = []; this.interval = setInterval(this.pushChanges.bind(this), 1000); } }
35. Push Changes async pushChanges() { try { await fetch("/", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ changes: this.changes }), }); this.changes = []; } catch(e) { /* ... */ } }
36. More Features • • • • • • other datatypes Counter Text (collaborative editing) Peritext table P2P with Hypermerge
37. Should the app work offline Do whatever you want <3 Read or write? Check out Service Workers There is no way around CSR & CRDTs No Yes only read both
38. Q&A Lucas Dohmen lucas.dohmen@innoq.com @moonbeamlabs www.innoq.com innoQ Deutschland GmbH Krischerstr. 100 40789 Monheim +49 2173 333660 Ohlauer Str. 43 10999 Berlin Ludwigstr. 180E 63067 Offenbach Kreuzstr. 16 80331 München Hermannstr. 13 20095 Hamburg Erftstr. 15-17 50672 Köln Königstorgraben 11 90402 Nürnberg Lars Hupel lars.hupel@innoq.com @larsr_h