Tweet
Tweet

Slide 1

Slide 1 text

YATA: COLLABORATIVE DOCS AND HOW TO MAKE THEM FAST

Slide 2

Slide 2 text

INTRODUCTION Bartosz Sypytkowski @Horusiath b.sypytkowski@gmail.com bartoszsypytkowski.com

Slide 3

Slide 3 text

— When do we need CRDTs — YATA: conflict resolution for arrays and maps — Update merge & split — Optimizations AGENDA

Slide 4

Slide 4 text

YJS.DEV

Slide 5

Slide 5 text

PROBLEM STATEMENT COLLABORATIVE TEXT EDITOR

Slide 6

Slide 6 text

COLLABORATIVE TEXT EDITOR Alice a Carol a Bob a

Slide 7

Slide 7 text

COLLABORATIVE TEXT EDITOR Alice a Carol a Bob a insert(1, ‘b’) insert(1, ‘c’)

Slide 8

Slide 8 text

COLLABORATIVE TEXT EDITOR Alice a Carol ac Bob ab

Slide 9

Slide 9 text

COLLABORATIVE TEXT EDITOR Alice a Carol ac Bob ab insert(1, ‘b’) insert(1, ‘c’) insert(1, ‘b’) insert(1, ‘c’)

Slide 10

Slide 10 text

COLLABORATIVE TEXT EDITOR Alice abc Carol abc Bob acb

Slide 11

Slide 11 text

MAYBE JUST SYNCHRONIZE VIA CENTRAL SERVER?

Slide 12

Slide 12 text

SERVER-DRIVEN TEXT EDITOR Alice a Carol a Bob a server a

Slide 13

Slide 13 text

SERVER-DRIVEN TEXT EDITOR Alice a Carol a Bob a server a insert(1, ‘b’) insert(1, ‘c’)

Slide 14

Slide 14 text

SERVER-DRIVEN TEXT EDITOR Alice a Carol a Bob a server a insert(1, ‘b’) insert(1, ‘c’) insert(1, ‘b’) insert(1, ‘c’) E1 E2

Slide 15

Slide 15 text

SERVER-DRIVEN TEXT EDITOR Alice a Carol a Bob a server ab insert(1, ‘b’) insert(1, ‘b’) E2 insert(1, ‘b’)

Slide 16

Slide 16 text

SERVER-DRIVEN TEXT EDITOR Alice ab Carol ab Bob ab server acb insert(1, ‘c’) insert(1, ‘c’) insert(1, ‘c’)

Slide 17

Slide 17 text

SERVER-DRIVEN TEXT EDITOR Alice acb Carol acb Bob acb server acb

Slide 18

Slide 18 text

COMPAR ING VECTOR CLOCKS ISSUES 1. Latency 2. Online-only / Network issues 3. Characters interleaving 4. Server: bottleneck & point of failure

Slide 19

Slide 19 text

FIELDS SHARING SIMILAR PROBLEMS 1. Collaborative text editors 2. Cross-continental data replication 3. Vehicle apps 4. Remote areas 5. One-account/multi-device sync

Slide 20

Slide 20 text

CONFLICT AVOIDANCE “Let the majority decide on the correct order.” Decisions are made by quorum

Slide 21

Slide 21 text

CONFLICT AVOIDANCE CONFLICT RESOLUTION “Let the majority decide on the correct order.” “Given enough context everyone should come to the same conclusion.” Decisions are made by quorum Decisions are made individually

Slide 22

Slide 22 text

CONFLICT RESOLUTION IT’S ALL ABOUT PRESERVING THE ORIGINAL INTENT

Slide 23

Slide 23 text

SEMANTICS MATTER 1. Insert characters one after another 2. Fixing a typo 3. Move an element 4. Move a range of elements 1. Insert characters at positions X, X+1, X+2, etc. 2. Insert/remove character at position X 3. Remove an element then re-insert it 4. Delete then re-insert range of elements

Slide 24

Slide 24 text

DON’T USE INDEXES FOR POSITIONING THEIR MEANING CHANGES OVER TIME

Slide 25

Slide 25 text

LAMPORT CLOCK A:1 Unique peer ID Sequence Number

Slide 26

Slide 26 text

CONTEXT-AWARE INSERTION insert_between(A:1, B:1, (B:2, ‘b’))

Slide 27

Slide 27 text

CONTEXT-AWARE INSERTION insert_between(A:1, B:1, (B:2, ‘b’)) inserted block ID inserted block content

Slide 28

Slide 28 text

CONTEXT-AWARE INSERTION insert_between(A:1, B:1, (B:2, ‘b’)) left neighbor right neighbor inserted block ID inserted block content

Slide 29

Slide 29 text

CONFLICT RESOLUTION

Slide 30

Slide 30 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING CONFLICT RESOLUTION

Slide 31

Slide 31 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING CONFLICT RESOLUTION

Slide 32

Slide 32 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING CONFLICT RESOLUTION ?

Slide 33

Slide 33 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING CONFLICT RESOLUTION Use block IDs to skip over blocks with lower precedence

Slide 34

Slide 34 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING CONFLICT RESOLUTION

Slide 35

Slide 35 text

YATA FOR MAPS

Slide 36

Slide 36 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” JSON Document state YMap “key” A:1 ENTRY INSERTION YMAP

Slide 37

Slide 37 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” JSON Document state YMap “key” A:1 ENTRY INSERTION YMAP ymap.set(‘key’, ‘b’)

Slide 38

Slide 38 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” JSON Document state YMap “key” A:1 ENTRY INSERTION YMAP ymap.set(‘key’, ‘b’) A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” JSON Create new block representing insert operation

Slide 39

Slide 39 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” JSON Document state YMap “key” A:2 ENTRY INSERTION YMAP ymap.set(‘key’, ‘b’) A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” JSON Insert block at the end of “key”’s sequence of values

Slide 40

Slide 40 text

A:1 ID NULL LEFT NULL RIGHT CONTENT length = 1 DELETED Document state YMap “key” A:2 ENTRY INSERTION YMAP ymap.set(‘key’, ‘b’) A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” JSON Tombstone all blocks for “key”’s values except the latest one

Slide 41

Slide 41 text

MOVING ELEMENTS

Slide 42

Slide 42 text

WHY DON’T JUST USE REMOVE + INSERT?

Slide 43

Slide 43 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING MOVING ELEMENTS

Slide 44

Slide 44 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING MOVING ELEMENTS doc.move(1..2, 0)

Slide 45

Slide 45 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING MOVING ELEMENTS doc.move(1..2, 0)

Slide 46

Slide 46 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING MOVING ELEMENTS doc.move(1..2, 0) A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED Create new block representing move operation

Slide 47

Slide 47 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING MOVING ELEMENTS doc.move(1..2, 0) A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED Range 1..2 maps onto continuous sequence of blocks from A:2 to B:2

Slide 48

Slide 48 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING MOVING ELEMENTS doc.move(1..2, 0) A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED Destination index 0 suggests to insert this block before A:1

Slide 49

Slide 49 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING Document state B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED MOVING ELEMENTS

Slide 50

Slide 50 text

Document state MOVING ELEMENTS doc.move(1..2, 0) A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED Mark moved elements

Slide 51

Slide 51 text

HOW ARE WE GOING TO READ THAT?

Slide 52

Slide 52 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR

Slide 53

Slide 53 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack A:3 A:2..B:2 Move frame informs if we’re currently within moved range context

Slide 54

Slide 54 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack A:3 A:2..B:2 Jump to the beginning of moved range

Slide 55

Slide 55 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack A:3 A:2..B:2 “b”

Slide 56

Slide 56 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack A:3 A:2..B:2 “b” “c”

Slide 57

Slide 57 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack A:3 A:2..B:2 “b” “c” We reached the end of a current move frame

Slide 58

Slide 58 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack “b” “c”

Slide 59

Slide 59 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack “b” “c” “a”

Slide 60

Slide 60 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR Move stack “b” “c” “a” Skip over moved blocks that aren’t part of a current move frame

Slide 61

Slide 61 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR “b” “c” “a”

Slide 62

Slide 62 text

READING MOVED ELEMENTS Document state A:3 ID NULL LEFT A:1 RIGHT CONTENT (A:2..B:2) MOVED NULL MOVED A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING NULL MOVED A:2 ID A:1 LEFT B:1 RIGHT CONTENT “b” STRING A:3 MOVED B:2 ID A:1 LEFT B:1 RIGHT CONTENT “c” STRING A:3 MOVED B:1 ID A:1 LEFT NULL RIGHT CONTENT “d” STRING NULL MOVED ITERATOR “b” “c” “a” “d”

Slide 63

Slide 63 text

OPTIMIZATIONS 1. KNOW YOUR ENVIRONMENT

Slide 64

Slide 64 text

KNOW THE DIFFERENCE Peer to Peer Client / Server Examples Yjs/Yrs, Automerge RiakDB, AntidoteDB, DynamoDB Ops / sec. few* (related to single user activity) > 1000 ops / sec Collaborators unknown, limited control known, under full control Network / connections heterogenous, unreliable Homogenous, fairly stable Data volume fits in memory (hopefully) greater than disk

Slide 65

Slide 65 text

OPTIMIZATIONS 2. REDUCE NUMBER OF OPERATIONS

Slide 66

Slide 66 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” STRING Document state

Slide 67

Slide 67 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” STRING Document state Both blocks have sequential IDs

Slide 68

Slide 68 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” STRING Document state Block was intended to be placed sequentially

Slide 69

Slide 69 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “ab” STRING Document state Block A:1 is responsible for holding 2 elements now (range from A:1 to A:2)

Slide 70

Slide 70 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “ab” STRING Document state A:1 ID NULL LEFT NULL RIGHT CONTENT “a” STRING A:2 ID A:1 LEFT NULL RIGHT CONTENT “b” STRING Document state These two representations are logically equivalent

Slide 71

Slide 71 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “ab” STRING A:3 ID A:2 LEFT NULL RIGHT CONTENT “c” STRING Document state Next block ID = last block ID + last block length insert_between(A:2, NULL, (A:3, ‘c’))

Slide 72

Slide 72 text

OPTIMIZATIONS BLOCK MERGING A:1 ID NULL LEFT NULL RIGHT CONTENT “ab” STRING A:3 ID A:2 LEFT NULL RIGHT CONTENT “c” STRING Document state Next block ID = last block ID + last block length insert_between(A:2, NULL, (A:3, ‘c’)) Left neighbor point to last ID not block ID

Slide 73

Slide 73 text

WHAT IF WE WANT TO INSERT ELEMENT WITHIN BLOCK?

Slide 74

Slide 74 text

A:1 ID NULL LEFT NULL RIGHT CONTENT “helo” STRING A:5 ID A:3 LEFT A:4 RIGHT CONTENT “l” STRING Document state insert_between(A:3, A:4, (A:5, ‘l’)) OPTIMIZATIONS BLOCK SPLITTING

Slide 75

Slide 75 text

A:4 ID A:3 LEFT NULL RIGHT CONTENT “o” STRING A:5 ID A:3 LEFT A:4 RIGHT CONTENT “l” STRING Document state insert_between(A:3, A:4, (A:5, ‘l’)) OPTIMIZATIONS BLOCK SPLITTING A:1 ID NULL LEFT NULL RIGHT CONTENT “hel” STRING Split blocks to create space

Slide 76

Slide 76 text

A:4 ID A:3 LEFT NULL RIGHT CONTENT “o” STRING A:5 ID A:3 LEFT A:4 RIGHT CONTENT “l” STRING Document state OPTIMIZATIONS BLOCK SPLITTING A:1 ID NULL LEFT NULL RIGHT CONTENT “hel” STRING

Slide 77

Slide 77 text

OPTIMIZATIONS 3. IMPROVE BLOCK INSERTION TIME

Slide 78

Slide 78 text

PREFER SEQUENTIAL ACCESS WHENEVER POSSIBLE

Slide 79

Slide 79 text

DOCUMENT BLOCK STRUCTURE UNDER THE HOOD A B C Clients Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Client block list Root types “name” BRANCH START Pointer to a CRDT list head

Slide 80

Slide 80 text

DOCUMENT BLOCK STRUCTURE UNDER THE HOOD A B C Clients Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Client block list Root types “name” BRANCH START Pointer to a CRDT list head New operation is always appended to the end

Slide 81

Slide 81 text

DOCUMENT BLOCK STRUCTURE UNDER THE HOOD A B C Clients Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Client block list Root types “name” BRANCH START Pointer to a CRDT list head Finding block by ID (e.g. C:2) is done by binary search

Slide 82

Slide 82 text

IT’S ALSO BETTER FOR SYNCHRONIZING DATA

Slide 83

Slide 83 text

DELTA REPLICATION A B C Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START A B C Block store A:1 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START Alice Bob

Slide 84

Slide 84 text

DELTA REPLICATION A B C Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START A B C Block store A:1 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START Alice Bob Bob is missing some of the updates from Alice

Slide 85

Slide 85 text

DELTA REPLICATION A B C Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START A B C Block store A:1 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START Alice Bob Bob creates a vector clock of his most recent updates A:2 B:3 C:4

Slide 86

Slide 86 text

DELTA REPLICATION A B C Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START A B C Block store A:1 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START Alice Bob Alice compares Bob’s vector clock against her own known state A:2 B:3 C:4

Slide 87

Slide 87 text

DELTA REPLICATION A B C Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START A B C Block store A:1 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START Alice Bob Alice produces a delta with blocks that Bob is missing A A:2 A:3

Slide 88

Slide 88 text

DELTA REPLICATION A B C Block store A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3 Root types “name” BRANCH START Block store Root types “name” BRANCH START Alice Bob Bob applies incoming updates on his side A A:2 A:3 A B C A:1 A:2 A:3 B:1 B:2 C:1 C:2 C:3

Slide 89

Slide 89 text

OPTIMIZATIONS 4. INSERTING ELEMENTS CONSECUTIVELY

Slide 90

Slide 90 text

OPTIMIZATIONS CACHING LATEST POSITION let doc = Doc::new(); let txt = doc.transact().get_text(“text”); txt.insert(&mut doc.transact(), 0, “hello”); txt.insert(&mut doc.transact(), 5, “world”);

Slide 91

Slide 91 text

MAPPING BETWEEN USER INDEX AND BLOCK ID INTRODUCES COST

Slide 92

Slide 92 text

OPTIMIZATIONS CURSORS let doc = Doc::new(); let txt = doc.transact().get_text(“text”); let mut cursor = txt.seek(0); cursor.insert(&mut doc.transact(), “hello”); cursor.insert(&mut doc.transact(), “world”);

Slide 93

Slide 93 text

SUMMARY

Slide 94

Slide 94 text

— Yjs/Yrs: https://crates.io/crates/yrs — Automerge: https://automerge.org/ — Ditto.live: https://www.ditto.live/ — RiakDB: https://riak.com/ — Amazon DynamoDB: https://aws.amazon.com/dynamodb/ — AntidoteDB: https://www.antidotedb.eu/ CRDT PROJECTS

Slide 95

Slide 95 text

— CRDTs deep dive: https://bartoszsypytkowski.com/tag/crdt/ — List of aggregated CRDT articles: https://crdt.tech — Making CRDTs faster: https://josephg.com/blog/crdts-go-brrr/ REFERENCES

Slide 96

Slide 96 text

THANK YOU