RethinkDB Cluster Monitoring

Display RethinkDB cluster statistics in a live graph using Go, Socket.io, and Epoch.

Transcript

1. RethinkDB
   Cluster Monitoring
   

   View Slide

2. Ryan Paul
   RethinkDB
   Evangelist
   @segphault
   

   View Slide

3. View Slide

4. RethinkDB 1.16
   • Changefeeds
   • Cluster admin
   • System tables
   

   View Slide

5. Backend
   Go and RethinkDB
   

   View Slide

6. Watch cluster stats
   r.Db("rethinkdb").Table("stats").Changes()
   

   View Slide

7. "new_val": {
   "id": ["cluster"],
   "query_engine": {
   "client_connections": 10,
   "clients_active": 2,
   "queries_per_sec": 12.971126524000002,
   "read_docs_per_sec": 0,
   "written_docs_per_sec": 7.4114252490000005
   }
   }
   

   View Slide

8. Watch cluster stats
   r.Db("rethinkdb").Table("stats").
   Filter(r.Row.Field("id").AtIndex(0).Eq("cluster")).
   Changes()
   

   View Slide

9. server, _ := socketio.NewServer(nil)
   stats, _ := r.Db("rethinkdb").Table("stats").
   Filter(r.Row.Field("id").AtIndex(0).Eq("cluster")).
   Changes().Run(conn)
   go func() {
   var change r.WriteChanges
   for stats.Next(&change) {
   server.BroadcastTo(
   "monitor", "stats", change.NewValue)
   }
   }()
   Pass changes to Socket.io
   

   View Slide

10. Frontend
    Epoch and Polymer
    

    View Slide

11. Epoch
    A realtime charting library for building
    high performance visualizations.
    

    View Slide

12. Plot data with Epoch
    function timestamp() { return (new Date).getTime() / 1000; }
    var chart = $("#chart").epoch({
    type: "time.line",
    axes: ["left", "bottom"],
    data: [
    {label: "Writes", values: [{time: timestamp(), y: 0}]},
    {label: "Reads", values: [{time: timestamp(), y: 0}]}
    ]
    });
    var socket = io.connect();
    socket.on("stats", function(data) {
    cluster.stats = data.query_engine;
    chart.push([
    {time: timestamp(), y: cluster.stats.written_docs_per_sec},
    {time: timestamp(), y: cluster.stats.read_docs_per_sec}
    ]);
    });
    

    View Slide

13. Bonus
    IRC bot for issue notification
    

    View Slide

14. Initialize bot
    ircConf := irc.NewConfig("mybot")
    ircConf.Server = "localhost:6667"
    bot := irc.Client(ircConf)
    bot.HandleFunc("connected",
    func(conn *irc.Conn, line *irc.Line) {
    log.Println("Connected to IRC server")
    conn.Join("#mychannel")
    })
    

    View Slide

15. Attach changefeed
    type Issue struct { Description, Type string }
    issues, _ := r.Db("rethinkdb").Table("current_issues").
    Filter(r.Row.Field("critical").Eq(true)).
    Changes().Field("new_val").Run(db)
    go func() {
    var issue Issue
    for issues.Next(&issue) {
    if issue.Type != "" {
    text := strings.Split(issue.Description, "\n")[0]
    message := fmt.Sprintf("(%s) %s", issue.Type, text)
    bot.Privmsg("#mychannel", message)
    }
    }
    }()
    

    View Slide

16. Resources
    • http://rethinkdb.com/blog/realtime-cluster-monitoring/
    • https://github.com/rethinkdb/rethink-status/tree/go-backend
    • http://rethinkdb.com/docs/quickstart/
    Questions?
    

    View Slide