WHATWG Stream ͕ Node.js ʹ΍͖ͬͯͨ 2021/08/30 @ Node ֶԂ37࣌ݶ໨

Twitter: @yosuke_furukawa Github: yosuke-furukawa

Before This Talk...

JSConf JP 2021 ΦϯϥΠϯ։࠵༧ఆʂʂʂ 2021/11/27 ։࠵

https://tinyurl.com/jsconf2021

Stream ࠓੲ෺ޠ

Stream • ੲ͔Β͋ͬͨ • جຊతʹ͢΂ͯͷ΋ͷ͸͜ͷߟ͑ํͰදݱͰ ͖Δ • σʔλͷม׵ݩ͕͋ͬͯม׵͠ͳ͕Βม׵ઌ ʹૹΔ 4PVSDF 5SBOTGPSN %FTUJOBUJPO

Stream • ϓϩάϥϛϯάͷ௨ৗͷϝϯλϧϞσϧ • ҰؾʹಡΈࠐΉɺҰؾʹม׵͢ΔɺҰؾʹॻ͖ग़͢ • Ͱ΋ͦΕͩͱ஗͔ͬͨΓɺϝϞϦΛଟେʹফඅͨ͠Γ͢Δ • ͳͷͰɺಡΈࠐΈͳ͕Βม׵ͯ͠ॻ͖ग़͍ͬͯ͢͏ߟ͑ํ • ྫ: Zip͔ΒಡΈࠐΜͰల։ͯ͠อଘ • 4PVSDF 5SBOTGPSN %FTUJOBUJPO

Stream // Stream Λ࡞Δ࣌ const { Readable } = require('stream'); 
 const controller = new AbortController();
 const read = new Readable({ read(size) { // σʔλΛಡΈࠐΉ࣌ͷຊॲཧ }, signal: controller.signal // cancel ͢Δ࣌ (AbortController͕࢖͑Δ) }); 


Stream // Stream ͔ΒσʔλΛಡΈࠐΉ࣌ readableStream.on("data", (chunk) => { // chunk ΛಡΈࠐΜͰͳΜ͔͢Δ }); readableStream.on("end", () => { // ऴྃॲཧ }); readableStream.on("error", () => { // ྫ֎ॲཧ });

Stream // pipe Ͱ chain ͯ͠ॻ͚Δɻ readableStream.pipe(transformStream).pipe(writableStream); // error ͕͋ͬͨΒ͜͏ॻ͘ɻ readableStream.on("error", (err) => { // ྫ֎ॲཧ }).pipe(transformStream).on("error", (err) => { // ྫ֎ॲཧ }).pipe(writableStream).on("error", (err) => { // ྫ֎ॲཧ });

Stream // error propagate ͕໘౗ͳΒ͜͏ॻ͚ͨΓ͢Δɻ const { pipeline } = require('stream'); pipeline( readableStream, transformStream, writableStream, (err) => { if (err) { console.error('Pipeline failed.', err); } else { console.log('Pipeline succeeded.'); } }, ); // ͜ΕΛ promisify ͢Ε͹ async/await Ͱ΋ॻ͚ͨΓ͢Δɻ

Stream // promisify ͞Εͨ΍ͭ΋͋Δɻ const { pipeline } = require('stream/promises'); await pipeline( readableStream, transformStream, writableStream, ).catch(console.error);

Stream • Node.js ʹੲ͔Β͋Δߟ͑ํ͕ͩɺ Promiseͱ͏·͘౷߹Ͱ͖ͳ͍ɻ • Promise ͷ୆಄ɺٴͼ async/await ʹΑͬͯେ෼࢖ΘΕʹ͘͘ͳ͖ͬͯͨɻ • ੲ͸ͦΕͦ͜ Callback Ͱ EventEmitter ͳߟ͑ํ͔͠ͳ͔ͬͨ࣌୅ͩͬ ͕ͨɺࠓ͸΋͏ΈΜͳ async/await ͬͯ΍ͬͯΔ • Stream Λ AsyncIteratorʹͯ͠࢖ͬͨΓɺ pipeline/promises Λ࢖͑͹ async/await ͱ౷߹ͯ͠࢖͏͜ͱ΋Մೳʹͳ͕ͬͨɺͦΕΑΓ͸ͦ΋ͦ΋ WHATWG Stream ଆͷํࣜʹҠ؅ͯ͠͠·ͬͨ΄͏͕ ecosystem friendly ͱ͍͏ߟ͑ํ

WHATWG Stream • WHATWG ଆͰఏҊ͞Ε͍ͯΔ Stream • Node.js ͷ Stream ͷӨڹ͸ଟগड͚ͯΔͱࢥ͏͕ جຊผ෺ • ओʹॲཧͷ࠷খ୯Ґͷߟ͑ํ͕ҧ͏ɻ Node.js Stream ͸ event ʹొ࿥͞ΕͨλεΫ୯Ґɺ WHATWG ͸ Promise Λجຊ୯Ґʹ͢Δ

WHATWG Stream // WHATWG Stream Λ࡞Δ࣌ 
 import { ReadableStream } from 'node:stream/web'; const stream = new ReadableStream({ async start(controller) { // σʔλΛಡΈࠐΈ࢝ΊΔॲཧ ※ controller ͸ޙड़ } async pull(controller) {
 // ஞ࣍తʹಡΉॲཧ } async cancel() { // Ωϟϯηϧ࣌ͷॲཧ } });

WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // reader ΛऔΓग़ͯ͠ const reader = readableStream.getReader(); // Ұճ͚ͩݺͿͳΒ͜ΕͰΦοέʔ const data = await reader.read(); console.log(data);

WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // ஞ࣮࣍ߦͰऔΔͳΒ const reader = stream.getReader(); let result = ""; let chunk; do { chunk = await reader.read(); if (chunk.value != undefined) result += chunk.value; } while (!chunk.done); // ΊΜͲ͍ɺ do-while ͳΜͯॻ͖ͨ͘ͳ͍

WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // ஞ࣮࣍ߦͰऔΔͳΒ stream ͕ AsyncIterator ͳͷͰɺ for await of ͰऔΕ Δ let result = ""; for await (const chunk of stream) { result += chunk; } console.log(result);

WHATWG Stream // pipeThrough ͰՃ޻͠ͳ͕Β ࠷ऴతʹ writable Stream ʹॻ࣌͘ readableStream.pipeThrough(transformStream) // Stream .pipeTo(writableStream) // Promise .then(() => console.log("done")) .catch(console.error); // Promise ͕ؼͬͯ͘Δͬͯ͜ͱ͸async/awaitͰॻ͚Δɻ try { await readableStream.pipeThrough(transformStream).pipeTo(writableStream); console.log("done"); } catch(e) { console.error(e); }

WHATWG Stream example import { ReadableStream } from 'node:stream/web'; import { setInterval as every } from 'node:timers/promises'; import { performance } from 'node:perf_hooks'; const stream = new ReadableStream({ async start(controller) { for await (const _ of every(1000)) controller.enqueue(performance.now()); } }); for await (const value of stream) console.log(value);

WHATWG Stream • Byteࢦ޲ͷReadable Stream ΋࡞ΕΔ • BYOB (Bring Your Own Buffer) Stream • ReadableStream ࡞੒࣌ʹ type Λ 'bytes' ʹ͢Δ ͱ BYOB ʹͳΔ • ͜Ε͸ byte ୯ҐͰࡉ͔͘ಡΈࠐΉ੍ޚΛߦ͍͍ͨ ࣌ʹ࣮ࢪ͢Δ΋ͷ

WHATWG Stream • Queueing Strategy • ࡉ͔͘ Queueing ͢ΔઓུΛܾΊ੍ͯޚͤ͞Δ͜ͱ͕Ͱ͖Δ • highwatermark ͱݺ͹ΕΔ͖͍͠஋ΛܾΊͯͦΕҎ্ͩͬͨΒ backpressure Λ͔͚ͯಡΈࠐ·ͳ͘͢Δ • 4PVSDF 5SBOTGPSN %FTUJOBUJPO RVFVF RVFVF highwatermark

WHATWG Stream • Queueing Strategy • ByteLengthQueueingStrategy: όΠταΠ ζͷ্ݶΛܾΊͯͦ͜·ͰಡΈࠐΉઓུ • CountQueueingStrategy: ԿճΩϡʔʹॻ͖ ࠐΜ͔ͩͷճ਺੍ޚ

WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • Node.js Writable ʹ͸͋Δ͕ɺ flush ͱ͍ͬͨ Ұؾʹు͖ग़͢ ؔ਺͕ͳ͍ • Θ͟Θ͟ΩϡʔΠϯάͨ͠ΓόοϑΝϦϯάͨ͠Γ͠ͳ͍͍ͯ࣌͘ʹҰؾ ʹు͖ग़͍͚ͨ͠Ͳɺݱঢ়ͩͱͦ͏͍͏ Controller ΋ͳ͍͍ͧͬͯ͏࿩Λ ͍ͯ͠Δɻ •

WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • PipeController ͱ͔Ͳ͏Αʁ https://github.com/whatwg/streams/issues/960

WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • Offset ·Ͱ write ͢Δؔ਺ͱ͔Λݕ౼த? https://github.com/whatwg/streams/issues/1128

WHATWG Stream • ·ͩ·ͩ discussion ͸ଓ͘ • Node.js Ͱ΋·ͩ Stability 1: Experimental

ࠓͳͥ௥Ճͨ͠ͷ͔? • ͍͔ͭ͋͘Δ • Web Compat ΁ͷڧԽ • fetch ͷ࣮૷ͷͨΊͷύʔπ࡞Γ • deno ͳͲͷڝ߹ͷଘࡏ

Node.js ͜Ε͔Βͷ10೥

Node.js ͜Ε͔Βͷ10೥ HTTPɺͲ͛Μ͔͍ͨ͠ɻ

·ͱΊ • Stream ͷࠓੲ • Node.js Stream͔ΒWHATWG Stream΁ • WHATWG Stream ͷࡉ͔͍࿩ • WHATWG Stream Ͱ଍Γͯͳ͍΋ͷ΋·ͩ·ͩ͋Δ • ࠓ͸·ͩ unstable ࢖͏࣌͸ؾΛ͚ͭͯ • ࠓNode.js͸࣍ͷ10೥ʹ޲͚ͯͷมֵظɺ͜ͷ͋ͨΓͷਐԽ΋݁ߏΊ͟·͍͠ • ࠓޙͷNode.jsΛ໘ന͍ͯ͘͜͠͏ɻ

JSConf JP 2021 ΦϯϥΠϯ։࠵༧ఆʂʂʂ 2021/11/27 ։࠵