RethinkDB Training Course

RethinkDB Training Course

A three-hour RethinkDB training workshop that I presented on Platzi. https://courses.platzi.com/courses/rethinkdb-databases/

Ba70f10866cbab0bc6b9b1d547ef8015?s=128

Ryan Paul

March 14, 2015
Tweet

Transcript

  1. RethinkDB Distributed Databases

  2. Ryan Paul RethinkDB Evangelist @segphault

  3. Introduction What is RethinkDB?

  4. What is RethinkDB? • Open source database for building realtime

    web applications • NoSQL database that stores schemaless JSON documents • Distributed database that is easy to scale
  5. Built for Realtime Apps • Subscribe to change notifications from

    database queries • No more polling — the database pushes changes to your app • Reduce the amount of plumbing needed to stream live updates
  6. Power and Convenience • Highly expressive query language • Relational

    features like table joins • Powerful admin UI with point- and-click cluster management
  7. Introduction to ReQL RethinkDB Query Language

  8. Introduction to ReQL • ReQL embeds natively into your programming

    language • Compose ReQL queries by chaining commands • ReQL queries are executed on the server
  9. Sample ReQL Queries r.table("users") .filter(r.row("age").gt(30)) r.table("users") .pluck("last_name") .distinct().count() r.table("fellowship") .filter({species:

    "hobbit"}) .update({species: "halfling"})
  10. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Number of

    unique last names
  11. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Access a

    database table
  12. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Isolate a

    document property
  13. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Consolidate duplicate

    values
  14. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Display the

    number of items
  15. ReQL Commands • Transformations: map, orderBy, skip, limit, slice •

    Aggregations: group, reduce, count, sum, avg, min, max, distinct, contains • Documents: row, pluck, without, merge, append, difference, keys, hasFields, spliceAt • Writing: insert, update, replace, delete • Control: forEach, range, branch, do, coerceTo, expr
  16. DEMO

  17. Secondary Indexes • Queries performed against indexes are much faster

    • Can index on a single property, multiple fields, or arbitrary ReQL expressions
  18. Querying an Index r.table("fellowship") .indexCreate("species") r.table("fellowship") .getAll("human", {index: "species"}) Find

    all humans in “fellowship”
  19. Anonymous Functions r.range(5).map(function(i) { return i.mul(2); }) Multiply each value

    by 2 You can pass anonymous functions to commands like map and reduce:
  20. Understanding ReQL • Anonymous function must return a valid ReQL

    expression • Client driver translates ReQL queries into wire protocol • Can’t mix local application functionality into ReQL queries • In JS use e.g. the mul and gt commands instead of the normal operators
  21. The r.row command Multiply each value by 2 You can

    often use r.row instead of an anonymous function: r.range(5).map(r.row.mul(2))
  22. Grouping r.table("songs") .group("artist") .count() Number of songs by each artist

    Use group to collect records into groups by a shared property:
  23. Grouping [ { "group": "Morcheeba", "reduction": 10 }, { "group":

    "Pink Floyd", "reduction": 145 }, ... ]
  24. Grouping • Commands chained after group will operate on the

    contents of the group • Use the ungroup command to operate on the grouped output
  25. Ungroup r.table("songs") .group("artist").count() .ungroup().max("reduction") Artist with the most songs

  26. Additional ReQL Features • Geospatial indexing for location- based queries

    • Support for storing binary objects • Date and time functions for time data • An http command for fetching remote JSON data
  27. Realtime Updates Working with Changefeeds

  28. Subscribe to change notifications on database queries Changefeeds

  29. r.table("users").changes() Changefeeds Track changes on the users table

  30. Changefeeds • The changes command returns a cursor that receives

    updates • Each update includes the new and old value of the modified record
  31. Changefeeds r.table("users") .filter({name: "Bob"}).delete() Changefeed output: { new_val: null, old_val:

    { id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bob', ... } }
  32. Changefeeds r.table("players") .orderBy({index: r.desc("score")}) .limit(3).changes() Track top three players by

    score Chain the changes command to an actual ReQL query:
  33. Changefeeds r.table("table").get(ID).changes() r.table("table").between(X, Y).changes() r.table("table").filter(CONDITION).changes() r.table("table").map(FN).changes() r.table("table").min(INDEX).changes() r.table("table").max(INDEX).changes() r.table("table").orderBy(CONDITION) .limit(N).changes()

    Commands that currently work with changefeeds:
  34. DEMO

  35. Building Web Apps Using RethinkDB in Node

  36. Client Driver • Use a RethinkDB client driver to access

    the database in your app • Official drivers available for Ruby, Python, and JavaScript • Third-party drivers available for other languages like Go and Clojure
  37. > Client Driver Install the JS client driver from NPM

    in your Node.js project: $ npm install rethinkdb --save
  38. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Add Bob to the “users” table
  39. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Import the RethinkDB module
  40. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Connect to the database
  41. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); ReQL query that inserts a record
  42. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Run the query on a connection
  43. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Close connection when operation is complete
  44. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Display query response
  45. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")

    .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }).error(function(err) { console.log("Failed:", err); }); Handle errors emitted by Promise
  46. Query Composition • ReQL embeds natively in your programming language

    • Pass around ReQL expressions like any other code • You can assign ReQL expressions to variables or store them in functions
  47. Query Composition var old = r.row("time").lt(r.now()); r.table("events") .filter(old) .delete().run(); Delete

    events that are in the past Store subexpressions in variables for reuse:
  48. Query Composition function olderThan(t) { return r.table("events") .filter(r.row("time").lt(t)); } olderThan(r.now()).delete().run();

    Delete events that are in the past Encapsulate query logic in functions for reuse:
  49. Using Express • Express is a Node.js framework for building

    web applications • It does URL routing, parameter parsing, and response handling • Easy to use Express to make simple REST APIS
  50. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Serve rows with specified species
  51. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Instantiate an Express app
  52. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Serve app on desired port
  53. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Define GET request with URL route
  54. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Perform RethinkDB query
  55. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Return query output JSON to user
  56. Using Express var app = require("express")(); var r = require("rethinkdb");

    app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Emit a 500 error if the query fails
  57. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {

    cursor.each(function(err, item) { console.log(item); }); }); Display every change on the “fellowship” table
  58. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {

    cursor.each(function(err, item) { console.log(item); }); }); Attach a changefeed to the table
  59. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {

    cursor.each(function(err, item) { console.log(item); }); }); Iterate over every value passed into the cursor
  60. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {

    cursor.each(function(err, item) { console.log(item); }); }); Display received changes in the console
  61. Using Socket.io • Powerful framework for realtime client/server communication •

    Supports WebSockets, long polling, and other transports • Lets you send JSON messages between your app and frontend
  62. Using Socket.io var sockio = require("socket.io"); var app = require("express")();

    var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Broadcast score changes over Socket.io
  63. Using Socket.io var sockio = require("socket.io"); var app = require("express")();

    var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Load the Socket.io module
  64. Using Socket.io var sockio = require("socket.io"); var app = require("express")();

    var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Instantiate Socket.io server
  65. Using Socket.io var sockio = require("socket.io"); var app = require("express")();

    var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Attach a changefeed to the query
  66. Using Socket.io var sockio = require("socket.io"); var app = require("express")();

    var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Broadcast updates to all Socket.io connections
  67. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>

    var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Receive Socket.io updates on frontend
  68. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>

    var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Load the Socket.io client script
  69. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>

    var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Connect to the Socket.io server
  70. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>

    var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Create handler for “update” messages
  71. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>

    var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Display update in browser console
  72. DEMO

  73. Cluster Configuration Sharding and replication

  74. Sharding and Replication • RethinkDB is designed for clustering and

    easy scalability • To add a new server to the cluster, just launch it with the join option • Configure sharding and replication per table • Any feature that works with a single database will work in a sharded cluster
  75. Add a Server to a Cluster $ rethinkdb --join server:29015

    >
  76. Cluster Configuration • Interactively configure your cluster with the web

    UI • Programmatically configure your cluster with simple ReQL commands • Fine-grained cluster control via full ReQL access to system tables
  77. DEMO

  78. Reconfigure Command r.table("users") .reconfigure({shards: 2, replicas: 1}) Configure table with

    2 shards and 1 replica The reconfigure command changes table settings:
  79. Config Command r.table("users").config() { id: "31c92680-f70c-4a4b-a49e-b238eb12c023", name: "users", db: "mydatabase",

    primary_key: "id", shards: [ {primary_replica: "a", "replicas": ["a", "b"]}, {primary_replica: "d", "replicas": ["c", "d"]} ], write_acks: "majority", durability: "hard" }
  80. System Tables • table_config: table configurations, including sharding and replication

    • server_config: server names and tags • db_config: database UUIDs and names
  81. System Tables • current_issues: cluster errors and problems • stats:

    statistics for the cluster • logs: server logs and messages • jobs: currently-running operations
  82. Scalability Tricks • useOutdated: reduce query time by not guaranteeing

    that you use the latest data • durability: can set to soft if you don’t want to wait for writes to be written to disk
  83. Additional Resources • RethinkDB website:
 http://rethinkdb.com • RethinkDB cookbook:
 http://rethinkdb.com/docs/cookbook

    • RethinkDB installation:
 http://rethinkdb.com/docs/install/