Building a Collaborative Text Editor

Have you ever clicked "Save" on a document, and caused a coworker to lose hours of work? Or spent more time coordinating "Who has permission to edit" in Slack than you did actually writing the document?

Google-Docs-style collaboration used to be nice to have. Now, it's expected. Companies like Quip and Coda have based their business on real-time collaboration features. Atom and Sublime Text have collaborative editing plugins. Even Confluence now supports collaborative editing!

In this talk, you'll learn how to build a great collaborative experience, based on solid fundamental ideas.

Transcript

1. @justinweiss #collabeditor Building a Collaborative Text Editor Justin Weiss
3. justinweiss.com
8. @justinweiss #collabeditor Collaborate instead of coordinate
10. https://xkcd.com/1597/
11. @justinweiss #collabeditor Just make changes vs Lock, change, unlock
12. @justinweiss #collabeditor In theory, it’s broken. In practice, it… mostly works?
13. @justinweiss #collabeditor You don’t have to be right, you have to be consistent
14. hello “hello”
15. byehello hello “hello” byehello bye “bye” “bye” “hello”
16. Photo by Hanny Naibaho on Unsplash
17. @justinweiss #collabeditor Our editor should be responsive and consistent
18. hello @@ -1 +1 @@ -hello +hello, world
19. hello { type: :insert, position: 5, text: “, world” } hello, world
20. hello { type: :insert, position: 5, text: “, world” } hello, world
21. hello hello, world Client A hello hello, world Client B insert “, world” @ 5 insert “, world” @ 5
22. Client A Client B
23. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5
24. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 insert c, 0
25. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1
26. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert c, 0 insert r, 1
27. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 c r a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert c, 0 insert r, 1 insert r, 1
28. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 c r a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert c, 0 insert r, 1 insert r, 1 ≠
29. @justinweiss #collabeditor Operational Transformation
30. Operational Transformation
31. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1
32. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 insert r, ?
33. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 insert r, 2
34. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 c a t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 insert r, 2 c a r t 0 1 2 3 4 5 insert c, 0
35. @justinweiss #collabeditor How do you change an operation?
36. @justinweiss #collabeditor How can two operations have the same effect, when run in different orders?
37. a t 0 1 2 3 4 5
38. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 c
39. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 ? ? ? ? 0 1 2 3 4 5 insert c, 0 insert r, 1 c
40. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 ? ? ? ? 0 1 2 3 4 5 insert c, 0 insert r, 1 c
41. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c
42. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
43. @justinweiss #collabeditor test “insert vs insert” do top = { type: :insert, text: “c”, position: 0 } left = { type: :insert, text: “r”, position: 1 } bottom, right = transform(top, left) assert_equal { type: insert, text: “c”, position: 0 }, bottom assert_equal { type: insert, text: “r”, position: 2 }, right end
44. @justinweiss #collabeditor document = “ca” top = { type: :insert, text: “r”, position: 2, client_id: 1 } left = { type: :insert, text: “p”, position: 2, client_id: 2 } # Should our document be “carp” or “capr”? # left is server, server always wins transform(top, left, false) How do you break ties?
45. @justinweiss #collabeditor document = “ca” top = { type: :insert, text: “r”, position: 2, client_id: 1 } left = { type: :insert, text: “p”, position: 2, client_id: 2 } # left is server, server always wins transform(top, left, false) # bigger client_id wins transform(top, left, top.client_id > left.client_id) How do you break ties?
46. @justinweiss #collabeditor test “insert vs insert” do top = { type: :insert, text: “c”, position: 0 } left = { type: :insert, text: “r”, position: 1 } bottom, right = transform(top, left) assert_equal { type: insert, text: “c”, position: 0 }, bottom assert_equal { type: insert, text: “r”, position: 2 }, right end
47. @justinweiss #collabeditor def transform(top, left, win_tiebreakers = false) bottom = transform_operation(top, left, win_tiebreakers), right = transform_operation(left, top, !win_tiebreakers) [bottom, right] end a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
48. @justinweiss #collabeditor def transform(top, left, win_tiebreakers = false) bottom = transform_operation(top, left, win_tiebreakers), right = transform_operation(left, top, !win_tiebreakers) [bottom, right] end a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
49. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position: 1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
50. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position: 1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
51. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position: 1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
52. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position: 1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
53. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position: 1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
54. @justinweiss #collabeditor Transformations should be functional
55. @justinweiss #collabeditor apply([op1, T(op2, op1)]) == apply([op2, T(op1, op2)]) # or, in other words: bottom = transform_operation(top, left, true) right = transform_operation(left, top, false) document.apply([top, right]) == document.apply([left, bottom]) Transform Property 1
56. @justinweiss #collabeditor apply([op1, T(op2, op1)]) == apply([op2, T(op1, op2)]) # or, in other words: bottom = transform_operation(top, left, true) right = transform_operation(left, top, false) document.apply([top, right]) == document.apply([left, bottom]) Transform Property 1 a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
57. @justinweiss #collabeditor apply([op1, T(op2, op1)]) == apply([op2, T(op1, op2)]) # or, in other words: bottom = transform_operation(top, left, true) right = transform_operation(left, top, false) document.apply([top, right]) == document.apply([left, bottom]) Transform Property 1 a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
58. @justinweiss #collabeditor Math makes testing easy! • Take a document • Generate random operations • Transform them • Apply them, to the document, according to TP1 • Are the documents equal?
59. a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
61. @justinweiss #collabeditor
63. a t a t a t e c a t e insert c, 0 (server) insert e, 2 (client 2) c insert e, 3 (client 2) insert c, 0 (server) a t a t a r t c a r t insert c, 0 (server) insert r, 1 (client 1) c insert r, 2 (client 1) insert c, 0 (server)
64. @justinweiss #collabeditor Transformation Function Take two operations, that happen simultaneously, and make them operations, that happen one after another.
65. @justinweiss #collabeditor Two pieces: Transformation functions (How do you transform?) Control algorithm (What do you transform?)
66. @justinweiss #collabeditor Talking to a server? You can use a version number!
67. @justinweiss #collabeditor > document { content: “at”, version: 2 } > operation { type: :insert, text: “r”, position: 1, version: 2 }
68. @justinweiss #collabeditor > top { type: :insert, text: “c”, position: 0, version: 2 } > left { type: :insert, text: “r”, position: 1, version: 2 }
71. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] bottom[0] r t s 1 2 3 4 5 top[1] c ? c h a r 0 1 2 3 4 5 left[1] ? r s 1 2 3 4 5 bottom[1] c ? c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 ? c right[1] 0 0 0
72. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c 0 0 0
73. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] transformed top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 transformed top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
74. a r t 1 2 3 4 5 r t 0 1 2 3 4 5 a r 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] transformed top[0] transf
75. r t 0 1 2 3 4 5 c r 0 1 2 3 4 5 c transformed left[0] op[0] r t s 1 2 3 4 5 top[1] c right[0] r s 1 2 3 4 5 transformed top[1] c 0 0
76. @justinweiss #collabeditor An Integrating, Transformation-Oriented Approach to Concurrency Control and Undo in Group Editors “In fact, it took us a long time to detect this flaw. Our solution became evident, however, as we had the idea of visualizing transformations as grid-like diagrams.”
77. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c 0 0 0
78. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] bottom[0] r t s 1 2 3 4 5 top[1] c ? c h a r 0 1 2 3 4 5 left[1] ? r s 1 2 3 4 5 bottom[1] c ? c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 ? c right[1] 0 0 0
79. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
80. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
81. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
82. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
83. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] transformed top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 transformed top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
84. @justinweiss #collabeditor def transform(left, top) left = Array(left) top = Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.first, true) bottom = transform_operation(top.first, left.first, false) return [Array(right), Array(bottom)] end
85. @justinweiss #collabeditor def transform(left, top) left = Array(left) top = Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.firs bottom = transform_operation(top.first, left.fir return [Array(right), Array(bottom)] end right = []
86. @justinweiss #collabeditor top = Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.first, true) bottom = transform_operation(top.first, left.first, false) return [Array(right), Array(bottom)] end right = [] bottom = [] left.each do |left_op| bottom.clear top.each do |top_op|
87. @justinweiss #collabeditor right = [] bottom = [] left.each do |left_op| bottom.clear top.each do |top_op| new_left, new_top = transform(left_op, top_op) left_op = new_left bottom.concat(new_top) end right = right.concat(left_op) top = bottom end
88. c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
89. @justinweiss #collabeditor right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
90. @justinweiss #collabeditor right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
91. @justinweiss #collabeditor right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
92. @justinweiss #collabeditor right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
93. @justinweiss #collabeditor right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
94. @justinweiss #collabeditor left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end [right, bottom] end
95. @justinweiss #collabeditor def transform(left, top) left = Array(left) top = Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.first, true) bottom = transform_operation(top.first, left.first, false) return [Array(right), Array(bottom)] end right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end [right, bottom] end
96. @justinweiss #collabeditor Control algorithms are generic
97. @justinweiss #collabeditor Operations and Transformations are specific
98. @justinweiss #collabeditor More specific operations mean better intent… But also many, many more transformations
99. @justinweiss #collabeditor What decisions make collaboration easier?
100. @justinweiss #collabeditor Speak in terms of actions, not state
102. @justinweiss #collabeditor Keep your state linear
103. @justinweiss #collabeditor Keep your state linear # “easy” state = [{:text, “a”, :bold}, {:text, “b”}] # not as easy state = [{ :paragraph, [ {:bold, [ {:text, “a”}]}]}]
104. @justinweiss #collabeditor Keep your state linear state = [{:enter, :p}, {:text, “a”}, {:exit, :p}]
105. document paragraph paragraph “Hello, world!” “Hi” “there!”
106. document paragraph paragraph “Hello, world!” “Hi” “there!” [0] [0, 0] [1] [1, 0] [1, 1]
107. @justinweiss #collabeditor Use transformable data: strings, numbers, arrays
108. @justinweiss #collabeditor Putting it all together
109. hello v1 Client hello v1 Server
110. hello v1 Client hello v1 Server hello! v1* insert “!”, 6, v1 insert “!”, 6, v1
111. hello v1 Client hello v1 Server hello! v2 insert “!”, 6, v1 hello! v2 insert “!”, 6, v1 acknowledged, v2
112. hello v1 Client heello v2 Server hello! v1* insert “!”, 6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1
113. hello v1 Client heello v2 Server hello! v1* insert “!”, 6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1 heello! v2* insert “e”, 1
114. hello v1 Client heello v2 Server hello! v1* insert “!”, 6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1 heello! v3 insert “e”, 1 heello! v3 insert “!”, 7, v2 acknowledged, v3
115. hello v1 Client heello v2 Server hello! v1* insert “!”, 6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1 heello! v3 insert “e”, 1 heello! v3 insert “!”, 7, v2 acknowledged, v3
116. @justinweiss #collabeditor What else do you need?
119. @justinweiss #collabeditor • Client 1: “Car” - Document: “Car” • Client 2: “t” - Document: “Cart” • Client 1: Undo! • Should the document be “Car” or should it be “t”?
120. @justinweiss #collabeditor Local Undo: undo my last change Global Undo: undo anyone’s last change
121. @justinweiss #collabeditor Collaborative editing • Responsiveness • Consistency • Intention-preserving • Cursor synchronizing • Local undo
122. H e l l o 0 1 2 3 4 5 Position: 1
123. H e l l o 0 1 2 3 4 5 Position: 5
124. H e l l o 0 1 2 3 4 5 { state: “Hello”, version: 2, position: 5, cursors: [ { client_id: 3, position: 2 } ] }
125. H e l l o 0 1 2 3 4 5 { state: “Hello”, version: 2, position: 5, cursors: [ { client_id: 3, position: 2 } ] }
126. c a r t 0 1 2 3 4 5 { state: “Hello”, version: 2, position: 1, cursors: [ { client_id: 2, position: 2 } ] }
127. c h a r t 0 1 2 3 4 5 { state: “Hello”, version: 2, position: 2, cursors: [ { client_id: 2, position: ? } ] }
128. c h a r t 0 1 2 3 4 5 { state: “Hello”, version: 2, position: 2, cursors: [ { client_id: 2, position: 3 } ] } transform_operation( [:cursor, 2], [:insert, “h”, 1] ) = [:cursor, 3]
129. @justinweiss #collabeditor When you perform an operation: • Apply it to your document • Send it to the server • Transform all cursors against it
130. @justinweiss #collabeditor transform(cursor, op) is just like transform(insert_text_op, op)
131. @justinweiss #collabeditor Which document version did the cursor come from?
132. @justinweiss #collabeditor You can’t apply a v2 cursor to a v1 document (but you can save it for later)
133. @justinweiss #collabeditor You can’t apply a v1 cursor to a v2 document (but you can transform it up to v2) (or just ignore it)
134. @justinweiss #collabeditor Same version? You’re all clear!
135. @justinweiss #collabeditor Same version? You’re all clear! (maybe)
136. c h a r t 0 1 2 3 4 5 v15* [insert h, 1] c a r t 0 1 2 3 4 5 v15 [] cursor, 3
137. c h a r t 0 1 2 3 4 5 v15* [insert h, 1] c a r t 0 1 2 3 4 5 v15 [] cursor, 3 transform([:cursor, 3], [:insert, “h”, 1]) # => [:cursor, 4]
138. @justinweiss #collabeditor You can send your cursor at any time (as long as all your operations are acknowledged)
139. c h a r t 0 1 2 3 4 5 v15* [insert h, 1] c a r t 0 1 2 3 4 5 v15 [] cursor, 5?
140. @justinweiss #collabeditor How does undo work?
141. @justinweiss #collabeditor undo([:insert, “a”, 3]) # => [:remove, “a”, 3] redo([:remove, “a”, 3]) # => [:insert, “a”, 3]
142. @justinweiss #collabeditor Operations have inverses doc.apply([op, invert(op)]) # => doc
143. @justinweiss #collabeditor doc.undos # => [op1’] doc.apply([op2]) doc.undos # => [op2’, op1’] doc.undo doc.undos # => [op1’]
144. @justinweiss #collabeditor def apply(op, save: true) perform(op) undos.push(op.invert) if save end
145. @justinweiss #collabeditor def undo op = undos.pop apply(op, save: false) redos.push(op.invert) end
146. @justinweiss #collabeditor Let’s break it!
147. c a r t 0 1 2 3 4 5 6 undos c a r t s 0 1 2 3 4 5 6 remove s, 4 undos insert s, 4
148. c a r t s 0 1 2 3 4 5 6 remove s, 4 undos c h a r t 0 1 2 3 4 5 s 6 remove s, 4 undos insert h, 1
149. c a r t s 0 1 2 3 4 5 6 remove s, 4 undos c h a r t 0 1 2 3 4 5 s 6 remove s, 5 undos insert h, 1
150. c h a r t 0 1 2 3 4 5 s 6 remove s, 5 undos c h a r t 0 1 2 3 4 5 6 undos undo!
151. @justinweiss #collabeditor Whenever you receive an operation, you have to transform undo/redo stacks
152. @justinweiss #collabeditor def transform_stacks(remote_op) self.undos, _ = transform(self.undos, remote_op) self.redos, _ = transform(self.redos, remote_op) end
153. @justinweiss #collabeditor Local undo • When you perform an operation: • Invert it, push it on the undo stack • When you receive an operation: • Transform the undo / redo stack against it • When you undo an operation: • Pop it off the stack, apply it, and send it to the server
154. @justinweiss #collabeditor Putting it all together
155. @justinweiss #collabeditor Document • Array of “things” • Version • Cursor • Remote cursors • Undo / Redo stacks
156. @justinweiss #collabeditor Operations • Type - :insert_text, :remove_text, etc. • Version • Data
157. @justinweiss #collabeditor Transformation functions t_op1 = transform_operation(op1, op2, win_ties)
158. @justinweiss #collabeditor Control algorithm top = server.operations_since?(left.version) right, bottom = transform(left, top)
159. @justinweiss #collabeditor Cursor transformations send_cursor(document.position) doc.cursors << received_cursor(cursor) transform_cursor(cursor, operations)
160. @justinweiss #collabeditor doc.undos << invert([:insert, “a”, 3]) doc.transform_stacks(operations) doc.undo
161. @justinweiss #collabeditor When you perform an operation: • Apply it to your document • Transform all cursors against it • Send it to the server • Add its inverse to the undo stack • Send your cursor… eventually
162. @justinweiss #collabeditor When you receive an operation: • Transform it against operations you haven’t sent • Apply their transformed operations, 
 and send your transformed operations • Transform all cursors against it • Transform your undo / redo stacks against it
163. @justinweiss #collabeditor When you change your cursor position • Send it • But only if you don’t have any unacknowledged operations!
164. @justinweiss #collabeditor When you receive a cursor: • Older version? Transform to your version, or ignore it • Same version? Transform against unacknowledged operations • Newer version? Hold onto it until you see that version, or ignore it
165. @justinweiss #collabeditor Demo: justinweiss-editor.herokuapp.com
166. @justinweiss #collabeditor It’s not perfect…
167. @justinweiss #collabeditor • You need a server • Weird edge cases with undo / redo • For certain apps, other collaboration methods might be easier
168. @justinweiss #collabeditor Peer-to-peer? Plain text only? Look into CRDTs.
169. @justinweiss #collabeditor Open Source • ShareDB: Control algorithm and server as a library • CRDTs: • Y.js / Gun.js: Flexible libraries for rich data types • Automerge: Automatically sync JSON data
170. Photo by Christin Hume on Unsplash
172. @justinweiss #collabeditor justin@justinweiss.com https://blog.aha.io/text-editor http://justinweiss-editor.herokuapp.com