Automerge: Making servers optional for real-time collaboration

Slides from a talk given at Craft Conference, Budapest, Hungary, 10 May 2018
http://martin.kleppmann.com/2018/05/10/craft-conf.html
and at J on the Beach, Málaga, Spain, 24 May 2018
https://jonthebeach.com/speakers/4/Martin+Kleppmann

Abstract:

Once upon a time, we used software that ran on our own computers, that worked offline, and that stored its data in files on the local disk. Then we decided to put it all in the cloud. We gained some great features: real-time collaboration, like in Google Docs, for example. But we also lost control of our own data, and became dependent on far-away servers to allow us to access the data that we created.

Automerge is part of an effort to get the best of both worlds. It is a JavaScript library for building real-time collaborative applications. However, apps built with Automerge also work offline, storing data locally, and synchronise their data with collaborators whenever a network is available. And although you can use it with servers, you don’t have to: synchronisation also works peer-to-peer, or via any network you choose.

In this talk we will explore how Automerge deals with different users independently modifying shared data in a collaborative application (hint: by merging the changes… automatically!), how it achieves consistency in highly distributed settings, and where it is heading in the future.

Resources:

Automerge: https://github.com/automerge/automerge
Trellis: https://github.com/automerge/trellis
Pixelpusher: https://github.com/automerge/pixelpusher
MPL (WebRTC layer): https://github.com/automerge/mpl
Hypermerge: https://github.com/automerge/hypermerge
Dat / Hypercore: https://datproject.org/
Proving CRDTs correct: https://doi.org/10.1145/3133933
JSON CRDT: http://arxiv.org/abs/1608.03960
My book: http://dataintensive.net/

Transcript

1. View Slide

2. View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. View Slide

23. View Slide

24. View Slide

25. View Slide

26. View Slide

27. View Slide

28. View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. For details, see our paper at https://doi.org/10.1145/3133933
    Victor B. F. Gomes, Martin Kleppmann, Dominic P. Mulligan, and Alastair R. Beresford:
    Verifying Strong Eventual Consistency in Distributed Systems. PACMPL 1(OOPSLA), 2017.
    

    View Slide

48. View Slide

49. View Slide

50. View Slide

51. Trellis, a Trello clone based on Automerge: https://github.com/automerge/trellis
    Joint work with Orion Henry, Peter van Hardenberg, Roshan Choxi, and Adam Wiggins.
    

    View Slide

52. View Slide

53. Pixelpusher, a collaborative pixel art editor: https://github.com/automerge/pixelpusher
    Created by Javier Valencia, Jeff Peterson, Peter van Hardenberg, and Jim Pick.
    

    View Slide

54. View Slide

55. View Slide

56. MPL, a WebRTC network layer for Automerge: https://github.com/automerge/mpl
    Joint work with Orion Henry, Peter van Hardenberg, Roshan Choxi, and Adam Wiggins.
    

    View Slide

57. Hypermerge, a peer-to-peer network layer: https://github.com/automerge/hypermerge
    Created by Jim Pick, Jeff Peterson, and Peter van Hardenberg.
    

    View Slide

58. View Slide

59. View Slide

60. View Slide

61. View Slide

62. View Slide

63. View Slide

64. View Slide

65. View Slide

66. View Slide

67. View Slide

68. View Slide

69. View Slide

70. View Slide

71. View Slide

72. View Slide

73. View Slide

74. View Slide

75. View Slide

76. View Slide

77. View Slide

78. View Slide

79. View Slide

80. View Slide

81. View Slide

82. View Slide

83. View Slide

84. View Slide

85. View Slide

86. View Slide

87. View Slide

88. View Slide

89. View Slide

90. View Slide

91. View Slide

92. View Slide

93. View Slide

94. View Slide

95. View Slide

96. View Slide

97. View Slide

98. View Slide

99. View Slide

100. Resources
     • Automerge: https://github.com/automerge/automerge
     • Trellis: https://github.com/automerge/trellis
     • Pixelpusher: https://github.com/automerge/pixelpusher
     • MPL (WebRTC layer): https://github.com/automerge/mpl
     • Hypermerge: https://github.com/automerge/hypermerge
     • Dat / Hypercore: https://datproject.org/
     • Proving CRDTs correct: https://doi.org/10.1145/3133933
     • JSON CRDT: http://arxiv.org/abs/1608.03960
     • My book: http://dataintensive.net/
     

     View Slide

101. View Slide