RethinkDB Training Course

A three-hour RethinkDB training workshop that I presented on Platzi. https://courses.platzi.com/courses/rethinkdb-databases/

Transcript

1. RethinkDB
   Distributed Databases
   

   View Slide

2. Ryan Paul
   RethinkDB
   Evangelist
   @segphault
   

   View Slide

3. Introduction
   What is RethinkDB?
   

   View Slide

4. What is RethinkDB?
   • Open source database for building
   realtime web applications
   • NoSQL database that
   stores schemaless JSON documents
   • Distributed database that is easy to
   scale
   

   View Slide

5. Built for Realtime Apps
   • Subscribe to change notifications
   from database queries
   • No more polling — the database
   pushes changes to your app
   • Reduce the amount of plumbing
   needed to stream live updates
   

   View Slide

6. Power and Convenience
   • Highly expressive query language
   • Relational features like table joins
   • Powerful admin UI with point-
   and-click cluster management
   

   View Slide

7. Introduction to ReQL
   RethinkDB Query Language
   

   View Slide

8. Introduction to ReQL
   • ReQL embeds natively into your
   programming language
   • Compose ReQL queries by
   chaining commands
   • ReQL queries are executed on the
   server
   

   View Slide

9. Sample ReQL Queries
   r.table("users")
   .filter(r.row("age").gt(30))
   r.table("users")
   .pluck("last_name")
   .distinct().count()
   r.table("fellowship")
   .filter({species: "hobbit"})
   .update({species: "halfling"})
   

   View Slide

10. Anatomy of a ReQL Query
    r.table("users")
    .pluck("last_name")
    .distinct().count()
    Number of unique last names
    

    View Slide

11. Anatomy of a ReQL Query
    r.table("users")
    .pluck("last_name")
    .distinct().count()
    Access a database table
    

    View Slide

12. Anatomy of a ReQL Query
    r.table("users")
    .pluck("last_name")
    .distinct().count()
    Isolate a document property
    

    View Slide

13. Anatomy of a ReQL Query
    r.table("users")
    .pluck("last_name")
    .distinct().count()
    Consolidate duplicate values
    

    View Slide

14. Anatomy of a ReQL Query
    r.table("users")
    .pluck("last_name")
    .distinct().count()
    Display the number of items
    

    View Slide

15. ReQL Commands
    • Transformations: map, orderBy, skip, limit, slice
    • Aggregations: group, reduce, count, sum, avg,
    min, max, distinct, contains
    • Documents: row, pluck, without, merge,
    append, difference, keys, hasFields, spliceAt
    • Writing: insert, update, replace, delete
    • Control: forEach, range, branch, do, coerceTo,
    expr
    

    View Slide

16. DEMO
    

    View Slide

17. Secondary Indexes
    • Queries performed against
    indexes are much faster
    • Can index on a single property,
    multiple fields, or arbitrary ReQL
    expressions
    

    View Slide

18. Querying an Index
    r.table("fellowship")
    .indexCreate("species")
    r.table("fellowship")
    .getAll("human", {index: "species"})
    Find all humans in “fellowship”
    

    View Slide

19. Anonymous Functions
    r.range(5).map(function(i) {
    return i.mul(2);
    })
    Multiply each value by 2
    You can pass anonymous functions
    to commands like map and reduce:
    

    View Slide

20. Understanding ReQL
    • Anonymous function must return a valid
    ReQL expression
    • Client driver translates ReQL queries into
    wire protocol
    • Can’t mix local application functionality into
    ReQL queries
    • In JS use e.g. the mul and gt commands
    instead of the normal operators
    

    View Slide

21. The r.row command
    Multiply each value by 2
    You can often use r.row instead of
    an anonymous function:
    r.range(5).map(r.row.mul(2))
    

    View Slide

22. Grouping
    r.table("songs")
    .group("artist")
    .count()
    Number of songs by each artist
    Use group to collect records into
    groups by a shared property:
    

    View Slide

23. Grouping
    [
    {
    "group": "Morcheeba",
    "reduction": 10
    },
    {
    "group": "Pink Floyd",
    "reduction": 145
    },
    ...
    ]
    

    View Slide

24. Grouping
    • Commands chained after group
    will operate on the contents of the
    group
    • Use the ungroup command to
    operate on the grouped output
    

    View Slide

25. Ungroup
    r.table("songs")
    .group("artist").count()
    .ungroup().max("reduction")
    Artist with the most songs
    

    View Slide

26. Additional ReQL Features
    • Geospatial indexing for location-
    based queries
    • Support for storing binary objects
    • Date and time functions for time data
    • An http command for fetching
    remote JSON data
    

    View Slide

27. Realtime Updates
    Working with Changefeeds
    

    View Slide

28. Subscribe to change notifications
    on database queries
    Changefeeds
    

    View Slide

29. r.table("users").changes()
    Changefeeds
    Track changes on the users table
    

    View Slide

30. Changefeeds
    • The changes command returns a
    cursor that receives updates
    • Each update includes the new and
    old value of the modified record
    

    View Slide

31. Changefeeds
    r.table("users")
    .filter({name: "Bob"}).delete()
    Changefeed output:
    {
    new_val: null,
    old_val: {
    id: '362ae837-2e29-4695-adef-4fa415138f90',
    name: 'Bob',
    ...
    }
    }
    

    View Slide

32. Changefeeds
    r.table("players")
    .orderBy({index: r.desc("score")})
    .limit(3).changes()
    Track top three players by score
    Chain the changes command
    to an actual ReQL query:
    

    View Slide

33. Changefeeds
    r.table("table").get(ID).changes()
    r.table("table").between(X, Y).changes()
    r.table("table").filter(CONDITION).changes()
    r.table("table").map(FN).changes()
    r.table("table").min(INDEX).changes()
    r.table("table").max(INDEX).changes()
    r.table("table").orderBy(CONDITION)
    .limit(N).changes()
    Commands that currently
    work with changefeeds:
    

    View Slide

34. DEMO
    

    View Slide

35. Building Web Apps
    Using RethinkDB in Node
    

    View Slide

36. Client Driver
    • Use a RethinkDB client driver to
    access the database in your app
    • Official drivers available for Ruby,
    Python, and JavaScript
    • Third-party drivers available for
    other languages like Go and Clojure
    

    View Slide

37. >
    Client Driver
    Install the JS client driver from
    NPM in your Node.js project:
    $ npm install rethinkdb --save
    

    View Slide

38. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    Add Bob to the “users” table
    

    View Slide

39. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    Import the RethinkDB module
    

    View Slide

40. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    Connect to the database
    

    View Slide

41. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    ReQL query that inserts a record
    

    View Slide

42. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    Run the query on a connection
    

    View Slide

43. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    Close connection when operation is complete
    

    View Slide

44. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    });
    Display query response
    

    View Slide

45. Client Driver
    var r = require("rethinkdb");
    r.connect().then(function(conn) {
    return r.table("users")
    .insert({name: "Bob"}).run(conn)
    .finally(function() { conn.close(); });
    }).then(function(output) {
    console.log(output);
    }).error(function(err) {
    console.log("Failed:", err);
    });
    Handle errors emitted by Promise
    

    View Slide

46. Query Composition
    • ReQL embeds natively in your
    programming language
    • Pass around ReQL expressions like
    any other code
    • You can assign ReQL expressions to
    variables or store them in functions
    

    View Slide

47. Query Composition
    var old = r.row("time").lt(r.now());
    r.table("events")
    .filter(old)
    .delete().run();
    Delete events that are in the past
    Store subexpressions in
    variables for reuse:
    

    View Slide

48. Query Composition
    function olderThan(t) {
    return r.table("events")
    .filter(r.row("time").lt(t));
    }
    olderThan(r.now()).delete().run();
    Delete events that are in the past
    Encapsulate query logic in
    functions for reuse:
    

    View Slide

49. Using Express
    • Express is a Node.js framework for
    building web applications
    • It does URL routing, parameter
    parsing, and response handling
    • Easy to use Express to make
    simple REST APIS
    

    View Slide

50. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Serve rows with specified species
    

    View Slide

51. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Instantiate an Express app
    

    View Slide

52. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Serve app on desired port
    

    View Slide

53. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Define GET request with URL route
    

    View Slide

54. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Perform RethinkDB query
    

    View Slide

55. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Return query output JSON to user
    

    View Slide

56. Using Express
    var app = require("express")();
    var r = require("rethinkdb");
    app.listen(8090);
    console.log("App listening on port 8090");
    app.get("/fellowship/species/:species", function(req, res) {
    r.connect().then(function(conn) {
    return r.table("fellowship")
    .filter({species: req.params.species}).run(conn)
    .finally(function() { conn.close(); });
    })
    .then(function(cursor) { return cursor.toArray(); })
    .then(function(output) { res.json(output); })
    .error(function(err) {
    res.status(500).json({err: err});
    });
    });
    Emit a 500 error if the query fails
    

    View Slide

57. Using Changefeeds
    r.connect().then(function(c) {
    return r.table("fellowship")
    .changes().run(c);
    })
    .then(function(cursor) {
    cursor.each(function(err, item) {
    console.log(item);
    });
    });
    Display every change on the “fellowship” table
    

    View Slide

58. Using Changefeeds
    r.connect().then(function(c) {
    return r.table("fellowship")
    .changes().run(c);
    })
    .then(function(cursor) {
    cursor.each(function(err, item) {
    console.log(item);
    });
    });
    Attach a changefeed to the table
    

    View Slide

59. Using Changefeeds
    r.connect().then(function(c) {
    return r.table("fellowship")
    .changes().run(c);
    })
    .then(function(cursor) {
    cursor.each(function(err, item) {
    console.log(item);
    });
    });
    Iterate over every value passed into the cursor
    

    View Slide

60. Using Changefeeds
    r.connect().then(function(c) {
    return r.table("fellowship")
    .changes().run(c);
    })
    .then(function(cursor) {
    cursor.each(function(err, item) {
    console.log(item);
    });
    });
    Display received changes in the console
    

    View Slide

61. Using Socket.io
    • Powerful framework for realtime
    client/server communication
    • Supports WebSockets, long
    polling, and other transports
    • Lets you send JSON messages
    between your app and frontend
    

    View Slide

62. Using Socket.io
    var sockio = require("socket.io");
    var app = require("express")();
    var r = require("rethinkdb");
    var io = sockio.listen(app.listen(8090));
    r.connect().then(function(conn) {
    return r.table("players")
    .orderBy({index: r.desc("score")})
    .limit(5).changes().run(conn);
    })
    .then(function(cursor) {
    cursor.each(function(err, data) {
    io.sockets.emit("update", data);
    });
    });
    Broadcast score changes over Socket.io
    

    View Slide

63. Using Socket.io
    var sockio = require("socket.io");
    var app = require("express")();
    var r = require("rethinkdb");
    var io = sockio.listen(app.listen(8090));
    r.connect().then(function(conn) {
    return r.table("players")
    .orderBy({index: r.desc("score")})
    .limit(5).changes().run(conn);
    })
    .then(function(cursor) {
    cursor.each(function(err, data) {
    io.sockets.emit("update", data);
    });
    });
    Load the Socket.io module
    

    View Slide

64. Using Socket.io
    var sockio = require("socket.io");
    var app = require("express")();
    var r = require("rethinkdb");
    var io = sockio.listen(app.listen(8090));
    r.connect().then(function(conn) {
    return r.table("players")
    .orderBy({index: r.desc("score")})
    .limit(5).changes().run(conn);
    })
    .then(function(cursor) {
    cursor.each(function(err, data) {
    io.sockets.emit("update", data);
    });
    });
    Instantiate Socket.io server
    

    View Slide

65. Using Socket.io
    var sockio = require("socket.io");
    var app = require("express")();
    var r = require("rethinkdb");
    var io = sockio.listen(app.listen(8090));
    r.connect().then(function(conn) {
    return r.table("players")
    .orderBy({index: r.desc("score")})
    .limit(5).changes().run(conn);
    })
    .then(function(cursor) {
    cursor.each(function(err, data) {
    io.sockets.emit("update", data);
    });
    });
    Attach a changefeed to the query
    

    View Slide

66. Using Socket.io
    var sockio = require("socket.io");
    var app = require("express")();
    var r = require("rethinkdb");
    var io = sockio.listen(app.listen(8090));
    r.connect().then(function(conn) {
    return r.table("players")
    .orderBy({index: r.desc("score")})
    .limit(5).changes().run(conn);
    })
    .then(function(cursor) {
    cursor.each(function(err, data) {
    io.sockets.emit("update", data);
    });
    });
    Broadcast updates to all Socket.io connections
    

    View Slide

67. Socket.io Frontend
    
    
    Real-time web app
    
    <br/>var socket = io.connect();<br/>socket.on("update", function(data) {<br/>console.log("Update:", data);<br/>});<br/>Receive Socket.io updates on frontend<br/>

    View Slide

68. Socket.io Frontend
    
    
    Real-time web app
    
    <br/>var socket = io.connect();<br/>socket.on("update", function(data) {<br/>console.log("Update:", data);<br/>});<br/>Load the Socket.io client script<br/>

    View Slide

69. Socket.io Frontend
    
    
    Real-time web app
    
    <br/>var socket = io.connect();<br/>socket.on("update", function(data) {<br/>console.log("Update:", data);<br/>});<br/>Connect to the Socket.io server<br/>

    View Slide

70. Socket.io Frontend
    
    
    Real-time web app
    
    <br/>var socket = io.connect();<br/>socket.on("update", function(data) {<br/>console.log("Update:", data);<br/>});<br/>Create handler for “update” messages<br/>

    View Slide

71. Socket.io Frontend
    
    
    Real-time web app
    
    <br/>var socket = io.connect();<br/>socket.on("update", function(data) {<br/>console.log("Update:", data);<br/>});<br/>Display update in browser console<br/>

    View Slide

72. DEMO
    

    View Slide

73. Cluster Configuration
    Sharding and replication
    

    View Slide

74. Sharding and Replication
    • RethinkDB is designed for clustering and
    easy scalability
    • To add a new server to the cluster, just
    launch it with the join option
    • Configure sharding and replication per table
    • Any feature that works with a single
    database will work in a sharded cluster
    

    View Slide

75. Add a Server to a Cluster
    $ rethinkdb --join server:29015
    >
    

    View Slide

76. Cluster Configuration
    • Interactively configure your cluster
    with the web UI
    • Programmatically configure your
    cluster with simple ReQL commands
    • Fine-grained cluster control via full
    ReQL access to system tables
    

    View Slide

77. DEMO
    

    View Slide

78. Reconfigure Command
    r.table("users")
    .reconfigure({shards: 2, replicas: 1})
    Configure table with 2 shards and 1 replica
    The reconfigure command
    changes table settings:
    

    View Slide

79. Config Command
    r.table("users").config()
    {
    id: "31c92680-f70c-4a4b-a49e-b238eb12c023",
    name: "users",
    db: "mydatabase",
    primary_key: "id",
    shards: [
    {primary_replica: "a", "replicas": ["a", "b"]},
    {primary_replica: "d", "replicas": ["c", "d"]}
    ],
    write_acks: "majority",
    durability: "hard"
    }
    

    View Slide

80. System Tables
    • table_config: table configurations,
    including sharding and replication
    • server_config: server names and
    tags
    • db_config: database UUIDs and
    names
    

    View Slide

81. System Tables
    • current_issues: cluster errors and
    problems
    • stats: statistics for the cluster
    • logs: server logs and messages
    • jobs: currently-running operations
    

    View Slide

82. Scalability Tricks
    • useOutdated: reduce query time
    by not guaranteeing that you use
    the latest data
    • durability: can set to soft if you
    don’t want to wait for writes to be
    written to disk
    

    View Slide

83. Additional Resources
    • RethinkDB website:

    http://rethinkdb.com
    • RethinkDB cookbook:

    http://rethinkdb.com/docs/cookbook
    • RethinkDB installation:

    http://rethinkdb.com/docs/install/
    

    View Slide