livedb-rethinkdb: Building collaborative apps with ShareJS and RethinkDB

Transcript

1. livedb-rethinkdb:
   Building collaborative apps with
   ShareJS and RethinkDB
   Jorge Silva · @thejsj · Developer Evangelist @ RethinkDB
   

   View Slide

2. Overview
   What is Operational Transformation?
   ShareJS: A Node.js library for OT
   RethinkDB: A database for realtime apps
   livedb-rethinkdb: RethinkDB + ShareJS
   

   View Slide

3. Text here...
   

   View Slide

4. Text here... Text here...
   User #1 User #2
   

   View Slide

5. Text here... Text here...
   User #1 User #2
   How could these two users collaborate?
   

   View Slide

6. Every time a user makes an edit,
   push an updated document
   

   View Slide

7. abc
   User #1 User #2
   ‘abc’
   Server
   abc
   

   View Slide

8. abc
   User #1 User #2
   ‘abc’
   ‘abc’
   abc
   Server
   abc
   

   View Slide

9. abc
   User #1 User #2
   ‘xyz’
   xyz
   1ms later:
   Server
   xyz
   ‘abc’
   

   View Slide

10. xyz
    User #1 User #2
    ‘abc’
    ‘xyz’
    xyz
    ‘abc’
    ‘xyz’
    Server
    xyz
    

    View Slide

11. This system doesn’t take latency
    into consideration
    

    View Slide

12. What if, instead of sending a completely updated
    document, we just sent the user’s actions?
    

    View Slide

13. User #1 User #2
    abc
    { op: [1, ‘bc’] }
    a
    Server
    abc
    

    View Slide

14. abc
    User #1 User #2
    abc
    { op: [1, ‘bc’] } { op: [1, ‘bc’] }
    Server
    abc
    

    View Slide

15. User #1 User #2
    { op: [3, ‘efg’] }
    012abc
    1ms later:
    { op: [0, ‘012’ ] }
    abcefg
    Server
    012efgabc
    

    View Slide

16. 012efgabc
    User #1 User #2
    012abcefg
    { op: [3, ‘efg’] } { op: [0, ‘012’ ] }
    Server
    012efgabc
    

    View Slide

17. This strategy is not able to accomodate
    the user’s intentions
    User #2’s changes are not inserted at the
    right cursor position.
    

    View Slide

18. Merely sending operations to the server
    doesn’t solve our problem.
    We need to tranform our operations.
    

    View Slide

19. What is Operational Transformation?
    

    View Slide

20. A class of algorithms that do multi-site realtime
    concurrency.
    Gives you eventual consistency
    - between multiple users
    - without retries
    - without errors
    - without any data being overwritten
    http://sharejs.org/
    

    View Slide

21. User #1 User #2
    { op: [3, ‘def’] }
    012abc
    1ms later:
    { op: [0, ‘012’] }
    abcdef
    Server
    012abcdef
    { op: [0, ‘012’] } { op: [6, ‘def’] }
    Transformed into:
    Transformed into:
    

    View Slide

22. 012abc
    User #1 User #2
    012abcdef
    Server
    012abcdef
    { op: [0, ‘012’] }
    Not Applied
    Transformed into:
    { op: [0, ‘012’] }
    { op: [0, ‘012’] }
    

    View Slide

23. 012abcdef
    User #1 User #2
    abcdef
    Server
    012abcdef
    { op: [6, ‘def’] }
    { op: [6, ‘def’] }
    Transformed into:
    { op: [6, ‘def’] }
    Not Applied
    

    View Slide

24. Possible applications
    Collaborative text and code editors
    Realtime wikis
    Any type of concurent text editing
    Examples: Google Docs, hackpad, etc.
    

    View Slide

25. OT is an important tool for
    building realtime applications
    

    View Slide

26. How can I use this in
    my own projects?
    

    View Slide

27. Enter
    ShareJS
    “ShareJS is an Operational Transform
    library for NodeJS & browsers. It lets
    you easily do live concurrent editing in
    your app.”
    

    View Slide

28. Demo
    http://livedb-rethinkdb.thejsj.com/
    

    View Slide

29. // WebSocket Connection
    var ws = new WebSocket(‘ws://localhost:8005’);
    var shareJS = new window.sharejs.Connection(ws);
    // Textarea
    var textareaDoc = shareJS.get(‘documents’, ‘helloworld’);
    textareaDoc.subscribe();
    // Wait for document to be ready
    textareaDoc.whenReady(function () {
    setTimeout(function () {
    if (textareaDoc.type && textareaDoc.type.name === ‘text’) {
    // Attach textarea to document
    var elem = document.getElementById(‘helloworld’);
    textareaDoc.attachTextarea(elem);
    }
    });
    });
    ShareJS on the client
    https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/client/index.html#L102-L124
    

    View Slide

30. // WebSocket Connection
    var ws = new WebSocket(‘ws://localhost:8005’);
    var shareJS = new window.sharejs.Connection(ws);
    // Textarea
    var textareaDoc = shareJS.get(‘documents’, ‘helloworld’);
    textareaDoc.subscribe();
    // Wait for document to be ready
    textareaDoc.whenReady(function () {
    setTimeout(function () {
    if (textareaDoc.type && textareaDoc.type.name === ‘text’) {
    // Attach textarea to document
    var elem = document.getElementById(‘helloworld’);
    textareaDoc.attachTextarea(elem);
    }
    });
    });
    ShareJS on the client
    https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/client/index.html#L102-L124
    

    View Slide

31. var sharejs = require(‘share’);
    var livedb = require(‘livedb’);
    // Connect to the database
    var db = require(‘livedb-rethinkdb’)({
    host: ‘localhost’,
    port: 28015,
    db: ‘sharejs’
    });
    var backend = livedb.client(db);
    // Attach livedb instances to ShareJS
    var share = sharejs.server.createClient({
    backend: backend
    });
    ShareJS on the server: database
    https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/livedb-client.js
    

    View Slide

32. var sharejs = require(‘share’);
    var livedb = require(‘livedb’);
    // Connect to the database
    var db = require(‘livedb-rethinkdb’)({
    host: ‘localhost’,
    port: 28015,
    db: ‘sharejs’
    });
    var backend = livedb.client(db);
    // Attach livedb instances to ShareJS
    var share = sharejs.server.createClient({
    backend: backend
    });
    ShareJS on the server: database
    https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/livedb-client.js
    

    View Slide

33. var wss = new WebSocketServer();
    var Duplex = require(‘stream’).Duplex;
    var share = sharejs.server.createClient({
    backend: backend
    });
    // On socket connection
    wss.on(‘connection’, function () {
    var stream = new Duplex({ objectMode: true });
    stream._write = function (chunk, encoding, callback) {
    client.send(JSON.stringify(chunk));
    return callback();
    };
    client.on(‘message’, function (data) {
    return stream.push(JSON.parse(data));
    });
    return share.listen(stream);
    });
    ShareJS on the server
    https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/socket-handler.js#L8-L49
    

    View Slide

34. var wss = new WebSocketServer();
    var Duplex = require(‘stream’).Duplex;
    var share = sharejs.server.createClient({
    backend: backend
    });
    // On socket connection
    wss.on(‘connection’, function () {
    var stream = new Duplex({ objectMode: true });
    stream._write = function (chunk, encoding, callback) {
    client.send(JSON.stringify(chunk));
    return callback();
    };
    client.on(‘message’, function (data) {
    return stream.push(JSON.parse(data));
    });
    return share.listen(stream);
    });
    ShareJS on the server
    https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/socket-handler.js#L8-L49
    

    View Slide

35. Open source database for building realtime
    applications
    NoSQL database that stores schemaless JSON
    documents
    Distributed database that is easy to scale
    What is RethinkDB?
    

    View Slide

36. Subscribe to change notifications from
    database queries
    No more polling – the database pushes changes
    to your app
    Reduce the amount of plumbing needed to
    stream live updates
    Built for realtime
    

    View Slide

37. var r = require(‘rethinkdb’);
    r
    .db(‘share_js’)
    .table(‘documents’)
    .filter({ name: ‘helloworld’ })
    .run(conn)
    .then(function (cursor) {
    // Go through every row returned by the query
    cursor.each(function (row) {
    console.log(row);
    });
    });
    Changefeeds
    

    View Slide

38. var r = require(‘rethinkdb’);
    r
    .db(‘share_js’)
    .table(‘documents’)
    .filter({ name: ‘helloworld’ })
    .run(conn)
    .then(function (cursor) {
    // Go through every row returned by the query
    cursor.each(function (row) {
    console.log(row);
    });
    });
    Changefeeds
    

    View Slide

39. var r = require(‘rethinkdb’);
    r
    .db(‘share_js’)
    .table(‘documents’)
    .filter({ name: ‘helloworld’ })
    .changes()
    .run(conn)
    .then(function (cursor) {
    // Gets fired every time a row changes
    cursor.each(function (row) {
    console.log(row);
    });
    });
    Changefeeds
    

    View Slide

40. ShareJS & RethinkDB
    npm install sharejs livedb livedb-rethinkdb --save
    

    View Slide

41. Access documents
    

    View Slide

42. Access every single operation
    

    View Slide

43. Listen for new operations
    

    View Slide

44. Try it!
    https://github.com/thejsj/sharejs-rethinkdb-example
    

    View Slide

45. ShareJS
    http://sharejs.org/ | https://github.com/share/ShareJS
    RethinkDB
    http://rethinkdb.com/ | @rethinkdb
    livedb-rethinkdb
    https://github.com/thejsj/livedb-rethinkdb
    Twitter
    @thejsj
    Questions?
    

    View Slide