Conquering Asynchronous Context With CLS

Stop the req bloat! Learn how to attach meta-data and information across an asynchronous call-chain in Node.js the smart way, without the clutter, collisions, or even middleware.

Transcript

1. a lightning talk by @FredKSchott Conquering Asynchronous Context with CLS  
2. Imagine a chain of function calls This can be a collection of express/connect middleware, an async.waterfall flow, or something as simple as a series of asynchronous functions and callbacks. a lightning talk by @FredKSchott Annotation Annotation slides added for context ⬡
3. function(req, res, next) { var user = // Wait... uh... shit. res.send(200, 'Hello ' + user.name); } db.getUser(userID, function(err, user) { // Wait what do I do with user? // Oh well.. next(); });
4. At the top of the chain, we fetch the user. But what can we do with it? next() doesn’t take a user argument, and even if it did you wouldn’t want to pass it through each and every function. At the bottom of the chain -- our final middleware -- we want to get that user. Is there a simple way to do this? Annotation ⬡ a lightning talk by @FredKSchott
5. function(req, res, next) { var user = req.user; // Yes! In ‘yo face! res.send(200, 'Hello ' + user.name); } db.getUser(userID, function(err, user) { req.user = user; next(); });
6. req.user If you’re using Express or Connect, a common solution is to leverage the request object that’s passed from middleware-to-middleware. A value or method attached in the first middleware will still exist in the last. Problem Solved! …right? Annotation ⬡ a lightning talk by @FredKSchott
7. req.user = user; next(); }); function(req, res, next) { var user = req.user; var translate = req.translate; // Okay… res.send(200, 'Hello ' + user.name); }
8. next(); } function(req, res, next) { var user = req.user; var translate = req.translate; var sdkClient = req.sdk; // Wait… res.send(200, 'Hello ' + user.name); }
9. function(req, res, next) { var user = req.user; var translate = req.translate; var sdkClient = req.sdk; var traceID = req.traceID; // Please… res.send(200, 'Hello ' + user.name); }
10. function(req, res, next) { var user = req.user; var translate = req.translate; var sdkClient = req.sdk; var traceID = req.traceID; var logger = req.logger; // Why?!? res.send(200, 'Hello ' + user.name); }
11. function(req, res, next) { var user = req.user; var translate = req.translate; var sdkClient = req.sdk; var traceID = req.traceID; var logger = req.logger; // This is the worst. if( req.featureflip['foo'] { req.isThisRediculous = true; } res.send(200, 'Hello ' + user.name); }
12. function(req, res, next) { var user = req.user; var translate = req.translate; var sdkClient = req.sdk; var traceID = req.traceID; var logger = req.logger; // This is the worst. if( req.featureflip['foo'] { req.isThisRediculous = true; } res.send(200, 'Hello ' + user.name); } // I Hate you.
13. While I chose a silly example, it accurately illustrates how cluttered your request object can get if you’re not careful. And eventually you’ll find yourself attaching things that don’t belong, things that aren’t really properties of the request. Annotation ⬡ a lightning talk by @FredKSchott
14. And even if you ARE careful, other modules might not be. Then you have to worry about collisions, unexpected behavior, and you’ll still have that clutter. Annotation ⬡ a lightning talk by @FredKSchott
15. And what if you don’t have Express? Annotation ⬡ a lightning talk by @FredKSchott
16. Forrest L. Norvell @Othiym23 Trevor Norris @trevnorris Tim Caswell @creationix
17. Enter Protagonists About a year ago, Forrest Norvell was creating logging and tracing tools for New Relic. He needed a way to attach meta-data along the call- chain that could be accessed when it was time to log. Annotation ⬡ a lightning talk by @FredKSchott
18. Domains At the time, domains were the only way to do this… and they didn’t do it well. They were originally intended for error handling, and came with a significant performance hit just by including them in your project. Have fun explaining that in your ReadMe… Annotation ⬡ a lightning talk by @FredKSchott
19. CLS Continuation- Local Storage
20. August 3rd, 2013 The first version of what would become ‘continuation-local storage’ is submitted to node.js core. Annotation ⬡ a lightning talk by @FredKSchott
21. CLS was originally intended to be a core module, since it required access to Node that wasn’t available yet. However, the lead contributors were wary to add another core module unnecessarily. Instead, they accepted the improvements to Node.js that would allow CLS to exist in user-land. These improvements became the AsyncListener API. Annotation ⬡ a lightning talk by @FredKSchott
22. So… how does it work? Annotation ⬡ a lightning talk by @FredKSchott
23. function(req, res, next) { var user = // ??? res.send(200, 'Hello ' + user.name); } db.getUser(userID, function(err, user) { // Wait what do I do with user? // Oh well.. next(); });
24. db.getUser(userID, function(err, user) { var session = cls.createNamespace('demo 1'); session.set('user', user); next(); }); function(req, res, next) { var user = // ??? res.send(200, 'Hello ' + user.name); }
25. function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); } db.getUser(userID, function(err, user) { var session = cls.createNamespace('demo 1'); session.set('user', user); next(); });
26. function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); } db.getUser(userID, function(err, user) { var session = cls.createNamespace('demo 1'); session.set('user', user); next(); }); // :boom:!
27. a lightning talk by @FredKSchott Annotation ⬡ CLS creates this namespace in the first middleware, which persists all the way down the call-chain. Data stored within that namespace is unique to that request and namespace. Multiple namespaces can exist for each request, so there’s no limit to how many modules can use CLS without conflict. Namespace are tied to the call-chain
28. Supports Native Functions db.getUser(userID, function(err, user) { var session = cls.getNamespace('demo 1'); session.set('user', user); fs.readdir(‘foo.txt’, next); }); function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); }
29. Supports Native Functions db.getUser(userID, function(err, user) { var session = cls.getNamespace('demo 1'); session.set('user', user); crypto.randomBytes(256, next); }); function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); }
30. Supports process.nextTick() db.getUser(userID, function(err, user) { var session = cls.getNamespace('demo 1'); session.set('user', user); process.nextTick(next); }); function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); }
31. Supports setTimeout() db.getUser(userID, function(err, user) { var session = cls.getNamespace('demo 1'); session.set('user', user); setTimeout(next, 1234); }); function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); }
32. Supports setInterval() db.getUser(userID, function(err, user) { var session = cls.getNamespace('demo 1'); session.set('user', user); setInterval(next, 1234); }); function(req, res, next) { var session = cls.getNamespace('demo 1'); var user = session.get('user'); res.send(200, 'Hello ' + user.name); }
33. Supports Nesting db.getUser(userID, function(err, user) { var session = cls.getNamespace('demo 1'); session.set(’value', 1); session.run(function(outer) { session.set(‘value’, 2); // outer: 2 session.run(function(inner) { session.set(‘value’, 3); // inner: 3, outer: 2 }); }); });
34. Supports so many things!
35. ⬡ othiym23 /node-continuation-local-storage
36. ⬡ Thank You. a lightning talk by @FredKSchott www.FredKSchott.com