Building a Collaborative Text Editor

Building a Collaborative Text Editor

Have you ever clicked "Save" on a document, and caused a coworker to lose hours of work? Or spent more time coordinating "Who has permission to edit" in Slack than you did actually writing the document?

Google-Docs-style collaboration used to be nice to have. Now, it's expected. Companies like Quip and Coda have based their business on real-time collaboration features. Atom and Sublime Text have collaborative editing plugins. Even Confluence now supports collaborative editing!

In this talk, you'll learn how to build a great collaborative experience, based on solid fundamental ideas.

Ac54c2b179cd4c54305846de2cb22ba1?s=128

Justin Weiss

April 18, 2018
Tweet

Transcript

  1. @justinweiss #collabeditor Building a Collaborative Text Editor Justin Weiss

  2. None
  3. justinweiss.com

  4. None
  5. None
  6. None
  7. None
  8. @justinweiss #collabeditor Collaborate instead of coordinate

  9. None
  10. https://xkcd.com/1597/

  11. @justinweiss #collabeditor Just make changes vs Lock, change, unlock

  12. @justinweiss #collabeditor In theory, it’s broken. In practice, it… mostly

    works?
  13. @justinweiss #collabeditor You don’t have to be right, you have

    to be consistent
  14. hello “hello”

  15. byehello hello “hello” byehello bye “bye” “bye” “hello”

  16. Photo by Hanny Naibaho on Unsplash

  17. @justinweiss #collabeditor Our editor should be responsive and consistent

  18. hello @@ -1 +1 @@ -hello +hello, world

  19. hello { type: :insert, position: 5, text: “, world” }

    hello, world
  20. hello { type: :insert, position: 5, text: “, world” }

    hello, world
  21. hello hello, world Client A hello hello, world Client B

    insert “, world” @ 5 insert “, world” @ 5
  22. Client A Client B

  23. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5
  24. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 insert c, 0
  25. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1
  26. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert c, 0 insert r, 1
  27. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 c r a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert c, 0 insert r, 1 insert r, 1
  28. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 c r a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert c, 0 insert r, 1 insert r, 1 ≠
  29. @justinweiss #collabeditor Operational Transformation

  30. Operational Transformation

  31. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1
  32. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 insert r, ?
  33. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 insert r, 2
  34. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 c a t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 insert r, 2 c a r t 0 1 2 3 4 5 insert c, 0
  35. @justinweiss #collabeditor How do you change an operation?

  36. @justinweiss #collabeditor How can two operations have the same effect,

    when run in different orders?
  37. a t 0 1 2 3 4 5

  38. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 c
  39. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 a r t 0 1 2 3 4 5 ? ? ? ? 0 1 2 3 4 5 insert c, 0 insert r, 1 c
  40. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 a r t 0 1 2 3 4 5 ? ? ? ? 0 1 2 3 4 5 insert c, 0 insert r, 1 c
  41. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c
  42. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
  43. @justinweiss #collabeditor test “insert vs insert” do top = {

    type: :insert, text: “c”, position: 0 } left = { type: :insert, text: “r”, position: 1 } bottom, right = transform(top, left) assert_equal { type: insert, text: “c”, position: 0 }, bottom assert_equal { type: insert, text: “r”, position: 2 }, right end
  44. @justinweiss #collabeditor document = “ca” top = { type: :insert,

    text: “r”, position: 2, client_id: 1 } left = { type: :insert, text: “p”, position: 2, client_id: 2 } # Should our document be “carp” or “capr”? # left is server, server always wins transform(top, left, false) How do you break ties?
  45. @justinweiss #collabeditor document = “ca” top = { type: :insert,

    text: “r”, position: 2, client_id: 1 } left = { type: :insert, text: “p”, position: 2, client_id: 2 } # left is server, server always wins transform(top, left, false) # bigger client_id wins transform(top, left, top.client_id > left.client_id) How do you break ties?
  46. @justinweiss #collabeditor test “insert vs insert” do top = {

    type: :insert, text: “c”, position: 0 } left = { type: :insert, text: “r”, position: 1 } bottom, right = transform(top, left) assert_equal { type: insert, text: “c”, position: 0 }, bottom assert_equal { type: insert, text: “r”, position: 2 }, right end
  47. @justinweiss #collabeditor def transform(top, left, win_tiebreakers = false) bottom =

    transform_operation(top, left, win_tiebreakers), right = transform_operation(left, top, !win_tiebreakers) [bottom, right] end a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
  48. @justinweiss #collabeditor def transform(top, left, win_tiebreakers = false) bottom =

    transform_operation(top, left, win_tiebreakers), right = transform_operation(left, top, !win_tiebreakers) [bottom, right] end a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
  49. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position:

    1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
  50. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position:

    1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
  51. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position:

    1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
  52. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position:

    1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
  53. @justinweiss #collabeditor # ours: { type: :insert, text: “r”, position:

    1 } # theirs: { type: :insert, text: “c”, position: 0 } def transform_operation(ours, theirs, win_tiebreakers) # TODO: handle other kinds of operations transformed_op = ours.dup if ours[:position] > theirs[:position] || (ours[:position] == theirs[:position] && !win_tiebreakers ) transformed_op[:position] = transformed_op[:position] + theirs[:text].length end transformed_op end
  54. @justinweiss #collabeditor Transformations should be functional

  55. @justinweiss #collabeditor apply([op1, T(op2, op1)]) == apply([op2, T(op1, op2)]) #

    or, in other words: bottom = transform_operation(top, left, true) right = transform_operation(left, top, false) document.apply([top, right]) == document.apply([left, bottom]) Transform Property 1
  56. @justinweiss #collabeditor apply([op1, T(op2, op1)]) == apply([op2, T(op1, op2)]) #

    or, in other words: bottom = transform_operation(top, left, true) right = transform_operation(left, top, false) document.apply([top, right]) == document.apply([left, bottom]) Transform Property 1 a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
  57. @justinweiss #collabeditor apply([op1, T(op2, op1)]) == apply([op2, T(op1, op2)]) #

    or, in other words: bottom = transform_operation(top, left, true) right = transform_operation(left, top, false) document.apply([top, right]) == document.apply([left, bottom]) Transform Property 1 a t 0 1 2 3 4 5 a t 0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
  58. @justinweiss #collabeditor Math makes testing easy! • Take a document

    • Generate random operations • Transform them • Apply them, to the document, according to TP1 • Are the documents equal?
  59. a t 0 1 2 3 4 5 a t

    0 1 2 3 4 5 a r t 0 1 2 3 4 5 c a r t 0 1 2 3 4 5 insert c, 0 insert r, 1 c insert r, 2 insert c, 0 insert c, 0
  60. None
  61. @justinweiss #collabeditor

  62. None
  63. a t a t a t e c a t

    e insert c, 0 (server) insert e, 2 (client 2) c insert e, 3 (client 2) insert c, 0 (server) a t a t a r t c a r t insert c, 0 (server) insert r, 1 (client 1) c insert r, 2 (client 1) insert c, 0 (server)
  64. @justinweiss #collabeditor Transformation Function Take two operations, that happen simultaneously,

    and make them operations, that happen one after another.
  65. @justinweiss #collabeditor Two pieces: Transformation functions (How do you transform?)

    Control algorithm (What do you transform?)
  66. @justinweiss #collabeditor Talking to a server? You can use a

    version number!
  67. @justinweiss #collabeditor > document { content: “at”, version: 2 }

    > operation { type: :insert, text: “r”, position: 1, version: 2 }
  68. @justinweiss #collabeditor > top { type: :insert, text: “c”, position:

    0, version: 2 } > left { type: :insert, text: “r”, position: 1, version: 2 }
  69. None
  70. None
  71. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] bottom[0] r t s 1 2 3 4 5 top[1] c ? c h a r 0 1 2 3 4 5 left[1] ? r s 1 2 3 4 5 bottom[1] c ? c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 ? c right[1] 0 0 0
  72. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c 0 0 0
  73. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] transformed top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 transformed top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
  74. a r t 1 2 3 4 5 r t

    0 1 2 3 4 5 a r 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] transformed top[0] transf
  75. r t 0 1 2 3 4 5 c r

    0 1 2 3 4 5 c transformed left[0] op[0] r t s 1 2 3 4 5 top[1] c right[0] r s 1 2 3 4 5 transformed top[1] c 0 0
  76. @justinweiss #collabeditor An Integrating, Transformation-Oriented Approach to Concurrency Control and

    Undo in Group Editors “In fact, it took us a long time to detect this flaw. Our solution became evident, however, as we had the idea of visualizing transformations as grid-like diagrams.”
  77. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c 0 0 0
  78. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] bottom[0] r t s 1 2 3 4 5 top[1] c ? c h a r 0 1 2 3 4 5 left[1] ? r s 1 2 3 4 5 bottom[1] c ? c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 ? c right[1] 0 0 0
  79. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
  80. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
  81. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
  82. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
  83. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] transformed top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 transformed top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
  84. @justinweiss #collabeditor def transform(left, top) left = Array(left) top =

    Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.first, true) bottom = transform_operation(top.first, left.first, false) return [Array(right), Array(bottom)] end
  85. @justinweiss #collabeditor def transform(left, top) left = Array(left) top =

    Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.firs bottom = transform_operation(top.first, left.fir return [Array(right), Array(bottom)] end right = []
  86. @justinweiss #collabeditor top = Array(top) return [left, top] if left.empty?

    || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.first, true) bottom = transform_operation(top.first, left.first, false) return [Array(right), Array(bottom)] end right = [] bottom = [] left.each do |left_op| bottom.clear top.each do |top_op|
  87. @justinweiss #collabeditor right = [] bottom = [] left.each do

    |left_op| bottom.clear top.each do |top_op| new_left, new_top = transform(left_op, top_op) left_op = new_left bottom.concat(new_top) end right = right.concat(left_op) top = bottom end
  88. c a r t 0 1 2 3 4 5

    r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c transformed left[0] bottom[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 bottom[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c right[1] 0 0 0
  89. @justinweiss #collabeditor right = [] bottom = [] left.each do

    |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
  90. @justinweiss #collabeditor right = [] bottom = [] left.each do

    |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
  91. @justinweiss #collabeditor right = [] bottom = [] left.each do

    |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
  92. @justinweiss #collabeditor right = [] bottom = [] left.each do

    |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end
  93. @justinweiss #collabeditor right = [] bottom = [] left.each do

    |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end c a r t 0 1 2 3 4 5 r t 0 1 2 3 4 5 c a r 0 1 2 3 4 5 c r 0 1 2 3 4 5 top[0] left[0] c right[0] top[0] r t s 1 2 3 4 5 top[1] c right[0] c h a r 0 1 2 3 4 5 left[1] bottom[0] r s 1 2 3 4 5 top[1] c right[1] c h r 0 1 2 3 4 5 h r s 1 2 3 4 5 bottom[1] c transformed left[1] 0 0 0
  94. @justinweiss #collabeditor left.each do |left_op| bottom = [] top.each do

    |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end [right, bottom] end
  95. @justinweiss #collabeditor def transform(left, top) left = Array(left) top =

    Array(top) return [left, top] if left.empty? || top.empty? if left.length == 1 && top.length == 1 right = transform_operation(left.first, top.first, true) bottom = transform_operation(top.first, left.first, false) return [Array(right), Array(bottom)] end right = [] bottom = [] left.each do |left_op| bottom = [] top.each do |top_op| right_op, bottom_op = transform(left_op, top_op) left_op = right_op bottom.concat(bottom_op) end right.concat(left_op) top = bottom end [right, bottom] end
  96. @justinweiss #collabeditor Control algorithms are generic

  97. @justinweiss #collabeditor Operations and Transformations are specific

  98. @justinweiss #collabeditor More specific operations mean better intent… But also

    many, many more transformations
  99. @justinweiss #collabeditor What decisions make collaboration easier?

  100. @justinweiss #collabeditor Speak in terms of actions, not state

  101. None
  102. @justinweiss #collabeditor Keep your state linear

  103. @justinweiss #collabeditor Keep your state linear # “easy” state =

    [{:text, “a”, :bold}, {:text, “b”}] # not as easy state = [{ :paragraph, [ {:bold, [ {:text, “a”}]}]}]
  104. @justinweiss #collabeditor Keep your state linear state = [{:enter, :p},

    {:text, “a”}, {:exit, :p}]
  105. document paragraph paragraph “Hello, world!” “Hi” “there!”

  106. document paragraph paragraph “Hello, world!” “Hi” “there!” [0] [0, 0]

    [1] [1, 0] [1, 1]
  107. @justinweiss #collabeditor Use transformable data: strings, numbers, arrays

  108. @justinweiss #collabeditor Putting it all together

  109. hello v1 Client hello v1 Server

  110. hello v1 Client hello v1 Server hello! v1* insert “!”,

    6, v1 insert “!”, 6, v1
  111. hello v1 Client hello v1 Server hello! v2 insert “!”,

    6, v1 hello! v2 insert “!”, 6, v1 acknowledged, v2
  112. hello v1 Client heello v2 Server hello! v1* insert “!”,

    6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1
  113. hello v1 Client heello v2 Server hello! v1* insert “!”,

    6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1 heello! v2* insert “e”, 1
  114. hello v1 Client heello v2 Server hello! v1* insert “!”,

    6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1 heello! v3 insert “e”, 1 heello! v3 insert “!”, 7, v2 acknowledged, v3
  115. hello v1 Client heello v2 Server hello! v1* insert “!”,

    6, v1 insert “!”, 6, v1 Nope. v1 was actually insert “e”, 1 heello! v3 insert “e”, 1 heello! v3 insert “!”, 7, v2 acknowledged, v3
  116. @justinweiss #collabeditor What else do you need?

  117. None
  118. None
  119. @justinweiss #collabeditor • Client 1: “Car” - Document: “Car” •

    Client 2: “t” - Document: “Cart” • Client 1: Undo! • Should the document be “Car” or should it be “t”?
  120. @justinweiss #collabeditor Local Undo: undo my last change Global Undo:

    undo anyone’s last change
  121. @justinweiss #collabeditor Collaborative editing • Responsiveness • Consistency • Intention-preserving

    • Cursor synchronizing • Local undo
  122. H e l l o 0 1 2 3 4

    5 Position: 1
  123. H e l l o 0 1 2 3 4

    5 Position: 5
  124. H e l l o 0 1 2 3 4

    5 { state: “Hello”, version: 2, position: 5, cursors: [ { client_id: 3, position: 2 } ] }
  125. H e l l o 0 1 2 3 4

    5 { state: “Hello”, version: 2, position: 5, cursors: [ { client_id: 3, position: 2 } ] }
  126. c a r t 0 1 2 3 4 5

    { state: “Hello”, version: 2, position: 1, cursors: [ { client_id: 2, position: 2 } ] }
  127. c h a r t 0 1 2 3 4

    5 { state: “Hello”, version: 2, position: 2, cursors: [ { client_id: 2, position: ? } ] }
  128. c h a r t 0 1 2 3 4

    5 { state: “Hello”, version: 2, position: 2, cursors: [ { client_id: 2, position: 3 } ] } transform_operation( [:cursor, 2], [:insert, “h”, 1] ) = [:cursor, 3]
  129. @justinweiss #collabeditor When you perform an operation: • Apply it

    to your document • Send it to the server • Transform all cursors against it
  130. @justinweiss #collabeditor transform(cursor, op) is just like transform(insert_text_op, op)

  131. @justinweiss #collabeditor Which document version did the cursor come from?

  132. @justinweiss #collabeditor You can’t apply a v2 cursor to a

    v1 document (but you can save it for later)
  133. @justinweiss #collabeditor You can’t apply a v1 cursor to a

    v2 document (but you can transform it up to v2) (or just ignore it)
  134. @justinweiss #collabeditor Same version? You’re all clear!

  135. @justinweiss #collabeditor Same version? You’re all clear! (maybe)

  136. c h a r t 0 1 2 3 4

    5 v15* [insert h, 1] c a r t 0 1 2 3 4 5 v15 [] cursor, 3
  137. c h a r t 0 1 2 3 4

    5 v15* [insert h, 1] c a r t 0 1 2 3 4 5 v15 [] cursor, 3 transform([:cursor, 3], [:insert, “h”, 1]) # => [:cursor, 4]
  138. @justinweiss #collabeditor You can send your cursor at any time

    (as long as all your operations are acknowledged)
  139. c h a r t 0 1 2 3 4

    5 v15* [insert h, 1] c a r t 0 1 2 3 4 5 v15 [] cursor, 5?
  140. @justinweiss #collabeditor How does undo work?

  141. @justinweiss #collabeditor undo([:insert, “a”, 3]) # => [:remove, “a”, 3]

    redo([:remove, “a”, 3]) # => [:insert, “a”, 3]
  142. @justinweiss #collabeditor Operations have inverses doc.apply([op, invert(op)]) # => doc

  143. @justinweiss #collabeditor doc.undos # => [op1’] doc.apply([op2]) doc.undos # =>

    [op2’, op1’] doc.undo doc.undos # => [op1’]
  144. @justinweiss #collabeditor def apply(op, save: true) perform(op) undos.push(op.invert) if save

    end
  145. @justinweiss #collabeditor def undo op = undos.pop apply(op, save: false)

    redos.push(op.invert) end
  146. @justinweiss #collabeditor Let’s break it!

  147. c a r t 0 1 2 3 4 5

    6 undos c a r t s 0 1 2 3 4 5 6 remove s, 4 undos insert s, 4
  148. c a r t s 0 1 2 3 4

    5 6 remove s, 4 undos c h a r t 0 1 2 3 4 5 s 6 remove s, 4 undos insert h, 1
  149. c a r t s 0 1 2 3 4

    5 6 remove s, 4 undos c h a r t 0 1 2 3 4 5 s 6 remove s, 5 undos insert h, 1
  150. c h a r t 0 1 2 3 4

    5 s 6 remove s, 5 undos c h a r t 0 1 2 3 4 5 6 undos undo!
  151. @justinweiss #collabeditor Whenever you receive an operation, you have to

    transform undo/redo stacks
  152. @justinweiss #collabeditor def transform_stacks(remote_op) self.undos, _ = transform(self.undos, remote_op) self.redos,

    _ = transform(self.redos, remote_op) end
  153. @justinweiss #collabeditor Local undo • When you perform an operation:

    • Invert it, push it on the undo stack • When you receive an operation: • Transform the undo / redo stack against it • When you undo an operation: • Pop it off the stack, apply it, and send it to the server
  154. @justinweiss #collabeditor Putting it all together

  155. @justinweiss #collabeditor Document • Array of “things” • Version •

    Cursor • Remote cursors • Undo / Redo stacks
  156. @justinweiss #collabeditor Operations • Type - :insert_text, :remove_text, etc. •

    Version • Data
  157. @justinweiss #collabeditor Transformation functions t_op1 = transform_operation(op1, op2, win_ties)

  158. @justinweiss #collabeditor Control algorithm top = server.operations_since?(left.version) right, bottom =

    transform(left, top)
  159. @justinweiss #collabeditor Cursor transformations send_cursor(document.position) doc.cursors << received_cursor(cursor) transform_cursor(cursor, operations)

  160. @justinweiss #collabeditor doc.undos << invert([:insert, “a”, 3]) doc.transform_stacks(operations) doc.undo

  161. @justinweiss #collabeditor When you perform an operation: • Apply it

    to your document • Transform all cursors against it • Send it to the server • Add its inverse to the undo stack • Send your cursor… eventually
  162. @justinweiss #collabeditor When you receive an operation: • Transform it

    against operations you haven’t sent • Apply their transformed operations, 
 and send your transformed operations • Transform all cursors against it • Transform your undo / redo stacks against it
  163. @justinweiss #collabeditor When you change your cursor position • Send

    it • But only if you don’t have any unacknowledged operations!
  164. @justinweiss #collabeditor When you receive a cursor: • Older version?

    Transform to your version, or ignore it • Same version? Transform against unacknowledged operations • Newer version? Hold onto it until you see that version, or ignore it
  165. @justinweiss #collabeditor Demo: justinweiss-editor.herokuapp.com

  166. @justinweiss #collabeditor It’s not perfect…

  167. @justinweiss #collabeditor • You need a server • Weird edge

    cases with undo / redo • For certain apps, other collaboration methods might be easier
  168. @justinweiss #collabeditor Peer-to-peer? Plain text only? Look into CRDTs.

  169. @justinweiss #collabeditor Open Source • ShareDB: Control algorithm and server

    as a library • CRDTs: • Y.js / Gun.js: Flexible libraries for rich data types • Automerge: Automatically sync JSON data
  170. Photo by Christin Hume on Unsplash

  171. None
  172. @justinweiss #collabeditor justin@justinweiss.com https://blog.aha.io/text-editor http://justinweiss-editor.herokuapp.com