River JS

A talk about the River Node module. https://github.com/andykent/river
Presented at LNUG Feb 2012 meetup.

Transcript

1. RIVER https://github.com/andykent/river
2. A RECAP...
3. REALTIME DATA ANALYSIS OVER UNBOUNDED STREAMS
4. WHEN YOU’VE GOT A HOSEPIPE WHAT YOU WANT IS A CREEK
5. WHEN YOU WANT MORE THAN A CREEK WHAT YOU NEED IS A RIVER
6. RIVER IS A BACKWARDS DATABASE
7. DATABASE “SELECT COUNT(1) FROM events”
8. RIVER “SELECT COUNT(1) FROM events”
9. MORE THAN A BACKWARDS DATABASE
10. RIVER INSERT 1 “SELECT COUNT(1) FROM events.win:time(10)”
11. RIVER REMOVE 1 INSERT 2 “SELECT COUNT(1) FROM events.win:time(10)”
12. RIVER REMOVE 2 INSERT 1 “SELECT COUNT(1) FROM events.win:time(10)”
13. RIVER REMOVE 1 INSERT 0 “SELECT COUNT(1) FROM events.win:time(10)”
14. CONTEXTS river = require('river') ctx = river.createContext() ctx.push('coords', {x:1, y:2}) ctx.push('coords', {x:3, y:4})
15. QUERIES query = ctx.addQuery("SELECT x FROM coords") query.on("insert", console.log) query.on("remove", console.log)
16. EXAMPLE
17. TWEET SOURCES SELECT source, COUNT(1) AS i FROM tweets.win:length(1000) GROUP BY source HAVING i > 10
18. SELECT source, COUNT(1) AS i FROM tweets.win:length(1000) GROUP BY source HAVING i > 10 PARSED QUERY QUERY PLAN COMPILED QUERY
19. SELECT source, COUNT(1) AS i FROM tweets.win:length(1000) GROUP BY source HAVING i > 10 PARSED QUERY QUERY PLAN MINIFICATION LENGTH BASED REPLAY AGGREGATION PROJECTION FILTER STREAM SELECTION COMPILED QUERY
20. RATE LIMITING SELECT accounts.email AS contact_email, FLOOR(COUNT(1)/60) AS req_sec FROM requests.win:time(60) JOIN accounts ON requests.token = accounts.token GROUP BY requests.token HAVING req_count > 2
21. ESPER
22. THE FUTURE...
23. PATTERN MATCHING SELECT * FROM stream[a -> b -> c]
24. SERVER WRAPPER
25. PARTITIONING
26. PERSISTENCE
27. SQL-PARSER https://github.com/forward/sql-parser
28. RIVER https://github.com/andykent/river