Node.js and Cassandra

Jorge Bay explaining his driver Node.js and Cassandra at the Madrid Cassandra Users meetup.

Transcript

1. Node.js and Cassandra For highly concurrent systems
2. Software as a service Most common scenario • I/O Bound o db o other services o file system • Low CPU usage • Peaks and valleys
3. Why Node.js
4. Why Node.js Event-based • Single threaded • Minimum overhead per connection • Predictable amount of memory under load • Apache / IIS vs Nginx: Process-based vs event-loop Everything runs in parallel except your code
5. Why Node.js Event-based: Apache vs Nginx source webfaction.com
6. Why Node.js Async I/O • Uses OS network interfaces + fixed thread pool • Time spent: db connections / files
7. Why Node.js Javascript: Closures • CPS: Continuation passing style • The scope of the outer function -> inner function • Javascript: Functional / Dynamic / Object oriented • ... packet manager / os community / V8 / ubiquitous...
8. Why Node.js Stream everything • Avoid buffering • HTTP: Chunked requests and responses • TCP: Chunks readable in a stream • Stream Piping (UNIX like)
9. The Driver for Cassandra
10. Features • Connection pooling to multiple hosts • Load balancing • Automatic failover / retry • Row and field streaming • Queuing: Concurrent connecting / preparing The Driver for Cassandra
11. The Driver for Cassandra A B C D G E F H App Cassandra nodes
12. Sample: Json Web Api The Driver for Cassandra app.get('/user/:id', function (req, res, next){ var query = 'SELECT * FROM users WHERE id = ?'; cassandra.executeAsPrepared(query, [req.params.id], function (err, result) { if (err) return next(err); var row = result.rows[0]; //Response: expose some properties of the user res.json({id: req.params.id, name: row.get('name')}); }); }); 1 2 3 4 5 6 7 8 9
13. The Driver for Cassandra How row streaming works Socket Protocol Parser Readable stream Transform stream Chunks Heade r and body chunk s Row Client Transform stream
14. Sample: Field streaming The Driver for Cassandra app.get('/user/:id/image', function (req, res, next){ var query = 'SELECT id, profile_image FROM users WHERE id = ?'; cassandra.streamField(query, [req.params.id], function (err, row, image) { if (err) return next(err); //pipe the image stream to the response stream image.pipe(res); }); }); 1 2 3 4 5 6 7 8
15. Sample: Field streaming + image resizing The Driver for Cassandra app.get('/user/:id/image', function (req, res, next){ var query = 'SELECT id, profile_image FROM users WHERE id = ?'; cassandra.streamField(query, [req.params.id], function (err, row, image) { if (err) return next(err); //pipe the image stream to a resizer stream image.pipe(resizer).pipe(res); }); }); 1 2 3 4 5 6 7 8
16. Moving forward Next features • Multiple data centers support. • Cassandra query tracing Contribute! :)
17. Jorge Bay Gondra @jorgebg jorgebaygondra@gmail.com github.com/jorgebay/node-cassandra-cql npm install node-cassandra-cql Thanks!
18. References and further reading A Design Framework for Highly Concurrent Systems by Matt Welsh, Steven D. Gribble, Eric A. Brewer, and David Culler @ UC Berkeley Concurrency is not Parallelism (it's better) by Rob Pike @Google Go lang How the single threaded non blocking IO model works in Node.js