Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Realtime data visualization with RethinkDB and Epoch

Ryan Paul
December 23, 2015

Realtime data visualization with RethinkDB and Epoch

Ryan Paul

December 23, 2015
Tweet

More Decks by Ryan Paul

Other Decks in Programming

Transcript

  1. What is RethinkDB? • Open source database for building realtime

    web applications • NoSQL database that stores schemaless JSON documents • Distributed database that is easy to scale
  2. Built for Realtime Apps • Subscribe to change notifications from

    database queries • No more polling — the database pushes changes to your app • Reduce the amount of plumbing needed to stream live updates
  3. Changefeeds • The changes command returns a cursor that receives

    updates • Each update includes the new and old value of the modified record
  4. r.table("users") .filter({name: "Bob"}).delete() Changefeed output: { new_val: null, old_val: {

    id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bob', ... } } Changefeeds
  5. What is Socket.io? • Realtime web framework • Supports communication

    between frontend and backend • Abstracts away network transport • Can send arbitrary JSON objects
  6. What is Socket.io? • Officially supported NodeJS server implementation •

    Client libraries for native mobile platforms and frontend web apps • Third-party server implementations for other languages
  7. Socket.io Server var server = require("http").createServer(); var io = require("socket.io")(server);

    server.listen(8000, () => console.log("Server started on port 8000")); io.on("connection", client => { client.emit("message", {text: "Hello World"}); });
  8. Socket.io Client var socket = io(); socket.on("message", data => {

    console.log("Received message:", data.text); });
  9. What is D3? • Library for building interactive data visualizations

    • Provides APIs for generating and manipulating SVG content • Extremely powerful but difficult to use
  10. Realtime Challenges • Potential for very frequent drawing updates •

    Need to remove old data while painting new data • Increments for your axes can change dynamically
  11. What is Epoch? • High-level library for realtime data visualizations

    • Builds on D3 but also uses <canvas> to boost performance • Supports a fixed range of visualization types
  12. Realtime Line Graph var chart = $("#chart").epoch({ type: "time.line", axes:

    ["left", "bottom"], data: [ {label: "Writes", values: [...]}, {label: "Reads", values: [...]} ] });
  13. Realtime Line Graph var server = require("http").createServer(); var io =

    require("socket.io")(server); var r = require("rethinkdbdash")(config.database); var bluebird = require("bluebird"); bluebird.coroutine(function*() { var stats = yield r.db("rethinkdb").table("stats") .get(["cluster"]).changes(); stats.each((err, item) => io.emit("stats", item)); })(); server.listen(8000, () => console.log("Server started on port 8000"));
  14. Realtime Line Graph function timestamp() { return (new Date).getTime() /

    1000; } var socket = io(); socket.on("stats", data => { var stats = data.new_val.query_engine; chart.push([ { time: timestamp(), y: stats.written_docs_per_sec}, { time: timestamp(), y: stats.read_docs_per_sec} ]); });
  15. Realtime Pie Graph var changes = yield r.table("votes").changes()("new_val"); changes.each((err, vote)

    => { io.to(`poll:${vote.poll}`).emit("vote", vote); }); ... io.on("connection", bluebird.coroutine(function*(client) { client.on("subscribe", (id, callback) => { r.table("polls").get(id).merge({ choices: r.row("choices").merge(c => ({ value: r.table("votes") .getAll(c("id"), {index: "choice"}).count() })) }).then(callback); client.join(`poll:${id}`); }); }));
  16. Realtime Pie Graph created: function() { var client = io();

    client.on("vote", this.updateVote); client.on("connect", () => client.emit("subscribe", this.id, this.setup)); }, methods: { setup: function(data) { this.poll = data; this.chart = $("#vote-pie").epoch({ type: "pie", inner: 90, data: data.choices}); }, updateVote: function(vote) { this.poll.choices.find(c => c.id == vote.choice).value += 1; this.chart.update(this.poll.choices); ...