RethinkDB Training Course

A three-hour RethinkDB training workshop that I presented on Platzi. https://courses.platzi.com/courses/rethinkdb-databases/

Transcript

1. RethinkDB Distributed Databases
2. Ryan Paul RethinkDB Evangelist @segphault
3. Introduction What is RethinkDB?
4. What is RethinkDB? • Open source database for building realtime web applications • NoSQL database that stores schemaless JSON documents • Distributed database that is easy to scale
5. Built for Realtime Apps • Subscribe to change notifications from database queries • No more polling — the database pushes changes to your app • Reduce the amount of plumbing needed to stream live updates
6. Power and Convenience • Highly expressive query language • Relational features like table joins • Powerful admin UI with point- and-click cluster management
7. Introduction to ReQL RethinkDB Query Language
8. Introduction to ReQL • ReQL embeds natively into your programming language • Compose ReQL queries by chaining commands • ReQL queries are executed on the server
9. Sample ReQL Queries r.table("users") .filter(r.row("age").gt(30)) r.table("users") .pluck("last_name") .distinct().count() r.table("fellowship") .filter({species: "hobbit"}) .update({species: "halfling"})
10. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Number of unique last names
11. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Access a database table
12. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Isolate a document property
13. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Consolidate duplicate values
14. Anatomy of a ReQL Query r.table("users") .pluck("last_name") .distinct().count() Display the number of items
15. ReQL Commands • Transformations: map, orderBy, skip, limit, slice • Aggregations: group, reduce, count, sum, avg, min, max, distinct, contains • Documents: row, pluck, without, merge, append, difference, keys, hasFields, spliceAt • Writing: insert, update, replace, delete • Control: forEach, range, branch, do, coerceTo, expr
16. DEMO
17. Secondary Indexes • Queries performed against indexes are much faster • Can index on a single property, multiple fields, or arbitrary ReQL expressions
18. Querying an Index r.table("fellowship") .indexCreate("species") r.table("fellowship") .getAll("human", {index: "species"}) Find all humans in “fellowship”
19. Anonymous Functions r.range(5).map(function(i) { return i.mul(2); }) Multiply each value by 2 You can pass anonymous functions to commands like map and reduce:
20. Understanding ReQL • Anonymous function must return a valid ReQL expression • Client driver translates ReQL queries into wire protocol • Can’t mix local application functionality into ReQL queries • In JS use e.g. the mul and gt commands instead of the normal operators
21. The r.row command Multiply each value by 2 You can often use r.row instead of an anonymous function: r.range(5).map(r.row.mul(2))
22. Grouping r.table("songs") .group("artist") .count() Number of songs by each artist Use group to collect records into groups by a shared property:
23. Grouping [ { "group": "Morcheeba", "reduction": 10 }, { "group": "Pink Floyd", "reduction": 145 }, ... ]
24. Grouping • Commands chained after group will operate on the contents of the group • Use the ungroup command to operate on the grouped output
25. Ungroup r.table("songs") .group("artist").count() .ungroup().max("reduction") Artist with the most songs
26. Additional ReQL Features • Geospatial indexing for location- based queries • Support for storing binary objects • Date and time functions for time data • An http command for fetching remote JSON data
27. Realtime Updates Working with Changefeeds
28. Subscribe to change notifications on database queries Changefeeds
29. r.table("users").changes() Changefeeds Track changes on the users table
30. Changefeeds • The changes command returns a cursor that receives updates • Each update includes the new and old value of the modified record
31. Changefeeds r.table("users") .filter({name: "Bob"}).delete() Changefeed output: { new_val: null, old_val: { id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bob', ... } }
32. Changefeeds r.table("players") .orderBy({index: r.desc("score")}) .limit(3).changes() Track top three players by score Chain the changes command to an actual ReQL query:
33. Changefeeds r.table("table").get(ID).changes() r.table("table").between(X, Y).changes() r.table("table").filter(CONDITION).changes() r.table("table").map(FN).changes() r.table("table").min(INDEX).changes() r.table("table").max(INDEX).changes() r.table("table").orderBy(CONDITION) .limit(N).changes() Commands that currently work with changefeeds:
34. DEMO
35. Building Web Apps Using RethinkDB in Node
36. Client Driver • Use a RethinkDB client driver to access the database in your app • Official drivers available for Ruby, Python, and JavaScript • Third-party drivers available for other languages like Go and Clojure
37. > Client Driver Install the JS client driver from NPM in your Node.js project: $ npm install rethinkdb --save
38. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Add Bob to the “users” table
39. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Import the RethinkDB module
40. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Connect to the database
41. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); ReQL query that inserts a record
42. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Run the query on a connection
43. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Close connection when operation is complete
44. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Display query response
45. Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users") .insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }).error(function(err) { console.log("Failed:", err); }); Handle errors emitted by Promise
46. Query Composition • ReQL embeds natively in your programming language • Pass around ReQL expressions like any other code • You can assign ReQL expressions to variables or store them in functions
47. Query Composition var old = r.row("time").lt(r.now()); r.table("events") .filter(old) .delete().run(); Delete events that are in the past Store subexpressions in variables for reuse:
48. Query Composition function olderThan(t) { return r.table("events") .filter(r.row("time").lt(t)); } olderThan(r.now()).delete().run(); Delete events that are in the past Encapsulate query logic in functions for reuse:
49. Using Express • Express is a Node.js framework for building web applications • It does URL routing, parameter parsing, and response handling • Easy to use Express to make simple REST APIS
50. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Serve rows with specified species
51. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Instantiate an Express app
52. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Serve app on desired port
53. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Define GET request with URL route
54. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Perform RethinkDB query
55. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Return query output JSON to user
56. Using Express var app = require("express")(); var r = require("rethinkdb"); app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Emit a 500 error if the query fails
57. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) { cursor.each(function(err, item) { console.log(item); }); }); Display every change on the “fellowship” table
58. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) { cursor.each(function(err, item) { console.log(item); }); }); Attach a changefeed to the table
59. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) { cursor.each(function(err, item) { console.log(item); }); }); Iterate over every value passed into the cursor
60. Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) { cursor.each(function(err, item) { console.log(item); }); }); Display received changes in the console
61. Using Socket.io • Powerful framework for realtime client/server communication • Supports WebSockets, long polling, and other transports • Lets you send JSON messages between your app and frontend
62. Using Socket.io var sockio = require("socket.io"); var app = require("express")(); var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Broadcast score changes over Socket.io
63. Using Socket.io var sockio = require("socket.io"); var app = require("express")(); var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Load the Socket.io module
64. Using Socket.io var sockio = require("socket.io"); var app = require("express")(); var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Instantiate Socket.io server
65. Using Socket.io var sockio = require("socket.io"); var app = require("express")(); var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Attach a changefeed to the query
66. Using Socket.io var sockio = require("socket.io"); var app = require("express")(); var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Broadcast updates to all Socket.io connections
67. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Receive Socket.io updates on frontend
68. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Load the Socket.io client script
69. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Connect to the Socket.io server
70. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Create handler for “update” messages
71. Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Display update in browser console
72. DEMO
73. Cluster Configuration Sharding and replication
74. Sharding and Replication • RethinkDB is designed for clustering and easy scalability • To add a new server to the cluster, just launch it with the join option • Configure sharding and replication per table • Any feature that works with a single database will work in a sharded cluster
75. Add a Server to a Cluster $ rethinkdb --join server:29015 >
76. Cluster Configuration • Interactively configure your cluster with the web UI • Programmatically configure your cluster with simple ReQL commands • Fine-grained cluster control via full ReQL access to system tables
77. DEMO
78. Reconfigure Command r.table("users") .reconfigure({shards: 2, replicas: 1}) Configure table with 2 shards and 1 replica The reconfigure command changes table settings:
79. Config Command r.table("users").config() { id: "31c92680-f70c-4a4b-a49e-b238eb12c023", name: "users", db: "mydatabase", primary_key: "id", shards: [ {primary_replica: "a", "replicas": ["a", "b"]}, {primary_replica: "d", "replicas": ["c", "d"]} ], write_acks: "majority", durability: "hard" }
80. System Tables • table_config: table configurations, including sharding and replication • server_config: server names and tags • db_config: database UUIDs and names
81. System Tables • current_issues: cluster errors and problems • stats: statistics for the cluster • logs: server logs and messages • jobs: currently-running operations
82. Scalability Tricks • useOutdated: reduce query time by not guaranteeing that you use the latest data • durability: can set to soft if you don’t want to wait for writes to be written to disk
83. Additional Resources • RethinkDB website:
 http://rethinkdb.com • RethinkDB cookbook:
 http://rethinkdb.com/docs/cookbook • RethinkDB installation:
 http://rethinkdb.com/docs/install/