Node.js streams

An intro talk about Node.js Streams API for JavaScript Zagreb meetup

Transcript

1. ROW ROW ROW YOUR BOAT by @zladuric
2. We have gathered here today • How to build Node.js Streams? • A talk to introduce the topic of streams in JavaScript, how this works in Node.js and how it didn't work out before. • You're my lab! by @zladuric
3. What are we talking about • The past - what was there before • The present - what are we doing now • The future - how you can get and use this • The client - the browser side of things • The server - Streams API
4. But first - why a stream Regular stuff • Comes as a whole blob • Takes up memory • When action is broken at 95%, redo the whole thing Streamed stuff • Just like regular, but chunked into bits • When broken, we can still reuse parts we got • PIPES!!1
5. The past Let's hope it dies!
6. All right, I'll tell ya On the client • Until a few years ago, we could not do much with streams in JS • Mostly for video/audio transfer with Flash, Java Applets On the server • Comes largely from unix network sockets • stdin, stdout, stderr • Created to streamline (yeah, I know) I/O operations • In web: Flash Media server, Red5 • Encoders/decoders, format converters • Bridges between backend services, more I/O WTF?
7. Node.js past • Streams API was (is?) very volatile • Backwards compatibility • Main problems: – Your stream could ignore stream.pause();) – Stream starts right away – whether you're ready or not
8. The present Node.js “streams2”: Stability: 2 – Unstable • Node.js is unstable ;) • No breaking changes expected for Node.js 0.12
9. Stream in Node.js • All streams are instances of an EventEmmiter (events and listeners/hooks) • var Readable = require('stream').Readable; – Implement your own _read(); • var Writable = require('stream').Writable; – Implement your own _write(); • var Duplex = implement both; • var Transform = convert input then pass to output – Implement _transform(); and flush();
10. How to be streamin' in Node $(#button).on('click', handler); anyone?
11. stream.Readable • Flowing and non-flowing? • Common events and methods • Examples: fs.createReadStream, http.createServer listeners, tcp and net sockets.
12. Translate stream events into english When stream says: It means: • readable “I have stuff to be read.” • data “Here's a chunk.” • end “I'm all out, you've read it.” • close “Bye bye!” (file.close); • error “Pain starts here.”
13. Pushmi-pullyu Flowing mode stream.on('data', function(chunk) { // chunk.length bytes // or if we set encoding, chunk is a string // or if objectMode: true, then values/objs ); Non-flowing: var chunk = stream.read();
14. Borin Simple example • this.push(); → stream.read(); • objectMode is handy
15. Writable stream • To implement a writable stream, implement _write method, drain
16. Transform streams Splits input into lines • _transform() - what to do on input data • _flush – what to do when input ends
17. And now pipes! var stream = getSomeReadStream() .pipe(toConverterStream()) .pipe(toEncoderStream()) .pipe(toTransportStream()) .pipe(toTheConsumerStream());
18. Piping examples Read file line by line Top secret Steal Google logo
19. Do you use it?
20. The client • WebRTC, WebSockets, Workers – they all stream data • XMLHttpRequest • User Media (MediaStream) API, Blob/FileSystem API, Audio, Geolocation etc.
21. Example http://jszgb.com/ - hacked to steal selfies <video id="video" autoplay></video> <script> // simplified navigator.webkitGetUserMedia({video: true}, function(localMediaStream) { console.log('getting some video'); var video = document.getElementById('video'); video.src = window.URL.createObjectURL(localMediaStream); }, console.log); </script>
22. Other useful options, events and methods • options.highWaterMark = 1024; • stream.setEncoding('utf-8'); • pause/resume a stream • wrap(); - for old libs < Node v0.10 • pipe()/unpipe(), unshift() • 'finish', 'close', 'drain' events
23. The future • Node.js is relatively stable • Whatwg is trying to standardize stream APIs from: Domenic Denicola
24. Play time http://thlorenz.github.io/stream-viz/
25. Question time 1. Was my english ok? 2. How boring was I, from 1 – 10? Your turn. Zlatko Đurić