Share
Embed
RethinkDB Training Course
A three-hour RethinkDB training workshop that I presented on Platzi. https://courses.platzi.com/courses/rethinkdb-databases/
Ryan Paul
March 14, 2015
Transcript
83 slides
-
What is RethinkDB? • Open source database for building realtime
web applications • NoSQL database that stores schemaless JSON documents • Distributed database that is easy to scale -
Built for Realtime Apps • Subscribe to change notifications from
database queries • No more polling — the database pushes changes to your app • Reduce the amount of plumbing needed to stream live updates -
Power and Convenience • Highly expressive query language • Relational
features like table joins • Powerful admin UI with point- and-click cluster management -
Introduction to ReQL • ReQL embeds natively into your programming
language • Compose ReQL queries by chaining commands • ReQL queries are executed on the server -
Sample ReQL Queries r.table("users") .filter(r.row("age").gt(30)) r.table("users") .pluck("last_name") .distinct().count() r.table("fellowship") .filter({species:
"hobbit"}) .update({species: "halfling"}) -
ReQL Commands • Transformations: map, orderBy, skip, limit, slice •
Aggregations: group, reduce, count, sum, avg, min, max, distinct, contains • Documents: row, pluck, without, merge, append, difference, keys, hasFields, spliceAt • Writing: insert, update, replace, delete • Control: forEach, range, branch, do, coerceTo, expr -
Secondary Indexes • Queries performed against indexes are much faster
• Can index on a single property, multiple fields, or arbitrary ReQL expressions -
Anonymous Functions r.range(5).map(function(i) { return i.mul(2); }) Multiply each value
by 2 You can pass anonymous functions to commands like map and reduce: -
Understanding ReQL • Anonymous function must return a valid ReQL
expression • Client driver translates ReQL queries into wire protocol • Can’t mix local application functionality into ReQL queries • In JS use e.g. the mul and gt commands instead of the normal operators -
The r.row command Multiply each value by 2 You can
often use r.row instead of an anonymous function: r.range(5).map(r.row.mul(2)) -
Grouping r.table("songs") .group("artist") .count() Number of songs by each artist
Use group to collect records into groups by a shared property: -
Grouping [ { "group": "Morcheeba", "reduction": 10 }, { "group":
"Pink Floyd", "reduction": 145 }, ... ] -
Grouping • Commands chained after group will operate on the
contents of the group • Use the ungroup command to operate on the grouped output -
Additional ReQL Features • Geospatial indexing for location- based queries
• Support for storing binary objects • Date and time functions for time data • An http command for fetching remote JSON data -
Changefeeds • The changes command returns a cursor that receives
updates • Each update includes the new and old value of the modified record -
Changefeeds r.table("users") .filter({name: "Bob"}).delete() Changefeed output: { new_val: null, old_val:
{ id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bob', ... } } -
Changefeeds r.table("players") .orderBy({index: r.desc("score")}) .limit(3).changes() Track top three players by
score Chain the changes command to an actual ReQL query: -
Changefeeds r.table("table").get(ID).changes() r.table("table").between(X, Y).changes() r.table("table").filter(CONDITION).changes() r.table("table").map(FN).changes() r.table("table").min(INDEX).changes() r.table("table").max(INDEX).changes() r.table("table").orderBy(CONDITION) .limit(N).changes()
Commands that currently work with changefeeds: -
Client Driver • Use a RethinkDB client driver to access
the database in your app • Official drivers available for Ruby, Python, and JavaScript • Third-party drivers available for other languages like Go and Clojure -
> Client Driver Install the JS client driver from NPM
in your Node.js project: $ npm install rethinkdb --save -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Add Bob to the “users” table -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Import the RethinkDB module -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Connect to the database -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); ReQL query that inserts a record -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Run the query on a connection -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Close connection when operation is complete -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }); Display query response -
Client Driver var r = require("rethinkdb"); r.connect().then(function(conn) { return r.table("users")
.insert({name: "Bob"}).run(conn) .finally(function() { conn.close(); }); }).then(function(output) { console.log(output); }).error(function(err) { console.log("Failed:", err); }); Handle errors emitted by Promise -
Query Composition • ReQL embeds natively in your programming language
• Pass around ReQL expressions like any other code • You can assign ReQL expressions to variables or store them in functions -
Query Composition var old = r.row("time").lt(r.now()); r.table("events") .filter(old) .delete().run(); Delete
events that are in the past Store subexpressions in variables for reuse: -
Query Composition function olderThan(t) { return r.table("events") .filter(r.row("time").lt(t)); } olderThan(r.now()).delete().run();
Delete events that are in the past Encapsulate query logic in functions for reuse: -
Using Express • Express is a Node.js framework for building
web applications • It does URL routing, parameter parsing, and response handling • Easy to use Express to make simple REST APIS -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Serve rows with specified species -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Instantiate an Express app -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Serve app on desired port -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Define GET request with URL route -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Perform RethinkDB query -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Return query output JSON to user -
Using Express var app = require("express")(); var r = require("rethinkdb");
app.listen(8090); console.log("App listening on port 8090"); app.get("/fellowship/species/:species", function(req, res) { r.connect().then(function(conn) { return r.table("fellowship") .filter({species: req.params.species}).run(conn) .finally(function() { conn.close(); }); }) .then(function(cursor) { return cursor.toArray(); }) .then(function(output) { res.json(output); }) .error(function(err) { res.status(500).json({err: err}); }); }); Emit a 500 error if the query fails -
Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {
cursor.each(function(err, item) { console.log(item); }); }); Display every change on the “fellowship” table -
Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {
cursor.each(function(err, item) { console.log(item); }); }); Attach a changefeed to the table -
Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {
cursor.each(function(err, item) { console.log(item); }); }); Iterate over every value passed into the cursor -
Using Changefeeds r.connect().then(function(c) { return r.table("fellowship") .changes().run(c); }) .then(function(cursor) {
cursor.each(function(err, item) { console.log(item); }); }); Display received changes in the console -
Using Socket.io • Powerful framework for realtime client/server communication •
Supports WebSockets, long polling, and other transports • Lets you send JSON messages between your app and frontend -
Using Socket.io var sockio = require("socket.io"); var app = require("express")();
var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Broadcast score changes over Socket.io -
Using Socket.io var sockio = require("socket.io"); var app = require("express")();
var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Load the Socket.io module -
Using Socket.io var sockio = require("socket.io"); var app = require("express")();
var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Instantiate Socket.io server -
Using Socket.io var sockio = require("socket.io"); var app = require("express")();
var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Attach a changefeed to the query -
Using Socket.io var sockio = require("socket.io"); var app = require("express")();
var r = require("rethinkdb"); var io = sockio.listen(app.listen(8090)); r.connect().then(function(conn) { return r.table("players") .orderBy({index: r.desc("score")}) .limit(5).changes().run(conn); }) .then(function(cursor) { cursor.each(function(err, data) { io.sockets.emit("update", data); }); }); Broadcast updates to all Socket.io connections -
Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>
var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Receive Socket.io updates on frontend -
Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>
var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Load the Socket.io client script -
Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>
var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Connect to the Socket.io server -
Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>
var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Create handler for “update” messages -
Socket.io Frontend <html> <head> <title>Real-time web app</title> <script src="/socket.io/socket.io.js"></script> <script>
var socket = io.connect(); socket.on("update", function(data) { console.log("Update:", data); }); Display update in browser console -
Sharding and Replication • RethinkDB is designed for clustering and
easy scalability • To add a new server to the cluster, just launch it with the join option • Configure sharding and replication per table • Any feature that works with a single database will work in a sharded cluster -
Cluster Configuration • Interactively configure your cluster with the web
UI • Programmatically configure your cluster with simple ReQL commands • Fine-grained cluster control via full ReQL access to system tables -
Reconfigure Command r.table("users") .reconfigure({shards: 2, replicas: 1}) Configure table with
2 shards and 1 replica The reconfigure command changes table settings: -
Config Command r.table("users").config() { id: "31c92680-f70c-4a4b-a49e-b238eb12c023", name: "users", db: "mydatabase",
primary_key: "id", shards: [ {primary_replica: "a", "replicas": ["a", "b"]}, {primary_replica: "d", "replicas": ["c", "d"]} ], write_acks: "majority", durability: "hard" } -
System Tables • table_config: table configurations, including sharding and replication
• server_config: server names and tags • db_config: database UUIDs and names -
System Tables • current_issues: cluster errors and problems • stats:
statistics for the cluster • logs: server logs and messages • jobs: currently-running operations -
Scalability Tricks • useOutdated: reduce query time by not guaranteeing
that you use the latest data • durability: can set to soft if you don’t want to wait for writes to be written to disk -
Additional Resources • RethinkDB website: http://rethinkdb.com • RethinkDB cookbook: http://rethinkdb.com/docs/cookbook
• RethinkDB installation: http://rethinkdb.com/docs/install/