Upgrade to Pro — share decks privately, control downloads, hide ads and more …

WHATWG Stream の話

WHATWG Stream の話

Node学園37時限目で発表した WHATWG Stream の話です。

Avatar for Yosuke Furukawa

Yosuke Furukawa

August 30, 2021
Tweet

More Decks by Yosuke Furukawa

Other Decks in Programming

Transcript

  1. Stream // Stream Λ࡞Δ࣌ const { Readable } = require('stream');

    
 const controller = new AbortController();
 const read = new Readable({ read(size) { // σʔλΛಡΈࠐΉ࣌ͷຊॲཧ }, signal: controller.signal // cancel ͢Δ࣌ (AbortController͕࢖͑Δ) }); 

  2. Stream // Stream ͔ΒσʔλΛಡΈࠐΉ࣌ readableStream.on("data", (chunk) => { // chunk

    ΛಡΈࠐΜͰͳΜ͔͢Δ }); readableStream.on("end", () => { // ऴྃॲཧ }); readableStream.on("error", () => { // ྫ֎ॲཧ });
  3. Stream // pipe Ͱ chain ͯ͠ॻ͚Δɻ readableStream.pipe(transformStream).pipe(writableStream); // error ͕͋ͬͨΒ͜͏ॻ͘ɻ

    readableStream.on("error", (err) => { // ྫ֎ॲཧ }).pipe(transformStream).on("error", (err) => { // ྫ֎ॲཧ }).pipe(writableStream).on("error", (err) => { // ྫ֎ॲཧ });
  4. Stream // error propagate ͕໘౗ͳΒ͜͏ॻ͚ͨΓ͢Δɻ const { pipeline } =

    require('stream'); pipeline( readableStream, transformStream, writableStream, (err) => { if (err) { console.error('Pipeline failed.', err); } else { console.log('Pipeline succeeded.'); } }, ); // ͜ΕΛ promisify ͢Ε͹ async/await Ͱ΋ॻ͚ͨΓ͢Δɻ
  5. Stream // promisify ͞Εͨ΍ͭ΋͋Δɻ const { pipeline } = require('stream/promises');

    await pipeline( readableStream, transformStream, writableStream, ).catch(console.error);
  6. Stream • Node.js ʹੲ͔Β͋Δߟ͑ํ͕ͩɺ Promiseͱ͏·͘౷߹Ͱ͖ͳ͍ɻ • Promise ͷ୆಄ɺٴͼ async/await ʹΑͬͯେ෼࢖ΘΕʹ͘͘ͳ͖ͬͯͨɻ

    • ੲ͸ͦΕͦ͜ Callback Ͱ EventEmitter ͳߟ͑ํ͔͠ͳ͔ͬͨ࣌୅ͩͬ ͕ͨɺࠓ͸΋͏ΈΜͳ async/await ͬͯ΍ͬͯΔ • Stream Λ AsyncIteratorʹͯ͠࢖ͬͨΓɺ pipeline/promises Λ࢖͑͹ async/await ͱ౷߹ͯ͠࢖͏͜ͱ΋Մೳʹͳ͕ͬͨɺͦΕΑΓ͸ͦ΋ͦ΋ WHATWG Stream ଆͷํࣜʹҠ؅ͯ͠͠·ͬͨ΄͏͕ ecosystem friendly ͱ͍͏ߟ͑ํ
  7. WHATWG Stream • WHATWG ଆͰఏҊ͞Ε͍ͯΔ Stream • Node.js ͷ Stream

    ͷӨڹ͸ଟগड͚ͯΔͱࢥ͏͕ جຊผ෺ • ओʹॲཧͷ࠷খ୯Ґͷߟ͑ํ͕ҧ͏ɻ Node.js Stream ͸ event ʹొ࿥͞ΕͨλεΫ୯Ґɺ WHATWG ͸ Promise Λجຊ୯Ґʹ͢Δ
  8. WHATWG Stream // WHATWG Stream Λ࡞Δ࣌ 
 import { ReadableStream

    } from 'node:stream/web'; const stream = new ReadableStream({ async start(controller) { // σʔλΛಡΈࠐΈ࢝ΊΔॲཧ ※ controller ͸ޙड़ } async pull(controller) {
 // ஞ࣍తʹಡΉॲཧ } async cancel() { // Ωϟϯηϧ࣌ͷॲཧ } });
  9. WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // reader ΛऔΓग़ͯ͠ const

    reader = readableStream.getReader(); // Ұճ͚ͩݺͿͳΒ͜ΕͰΦοέʔ const data = await reader.read(); console.log(data);
  10. WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // ஞ࣮࣍ߦͰऔΔͳΒ const reader

    = stream.getReader(); let result = ""; let chunk; do { chunk = await reader.read(); if (chunk.value != undefined) result += chunk.value; } while (!chunk.done); // ΊΜͲ͍ɺ do-while ͳΜͯॻ͖ͨ͘ͳ͍
  11. WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // ஞ࣮࣍ߦͰऔΔͳΒ stream ͕

    AsyncIterator ͳͷͰɺ for await of ͰऔΕ Δ let result = ""; for await (const chunk of stream) { result += chunk; } console.log(result);
  12. WHATWG Stream // pipeThrough ͰՃ޻͠ͳ͕Β ࠷ऴతʹ writable Stream ʹॻ࣌͘ readableStream.pipeThrough(transformStream)

    // Stream .pipeTo(writableStream) // Promise .then(() => console.log("done")) .catch(console.error); // Promise ͕ؼͬͯ͘Δͬͯ͜ͱ͸async/awaitͰॻ͚Δɻ try { await readableStream.pipeThrough(transformStream).pipeTo(writableStream); console.log("done"); } catch(e) { console.error(e); }
  13. WHATWG Stream example import { ReadableStream } from 'node:stream/web'; import

    { setInterval as every } from 'node:timers/promises'; import { performance } from 'node:perf_hooks'; const stream = new ReadableStream({ async start(controller) { for await (const _ of every(1000)) controller.enqueue(performance.now()); } }); for await (const value of stream) console.log(value);
  14. WHATWG Stream • Byteࢦ޲ͷReadable Stream ΋࡞ΕΔ • BYOB (Bring Your

    Own Buffer) Stream • ReadableStream ࡞੒࣌ʹ type Λ 'bytes' ʹ͢Δ ͱ BYOB ʹͳΔ • ͜Ε͸ byte ୯ҐͰࡉ͔͘ಡΈࠐΉ੍ޚΛߦ͍͍ͨ ࣌ʹ࣮ࢪ͢Δ΋ͷ
  15. WHATWG Stream • Queueing Strategy • ࡉ͔͘ Queueing ͢ΔઓུΛܾΊ੍ͯޚͤ͞Δ͜ͱ͕Ͱ͖Δ •

    highwatermark ͱݺ͹ΕΔ͖͍͠஋ΛܾΊͯͦΕҎ্ͩͬͨΒ backpressure Λ͔͚ͯಡΈࠐ·ͳ͘͢Δ • 4PVSDF 5SBOTGPSN %FTUJOBUJPO RVFVF RVFVF highwatermark
  16. WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • Node.js Writable ʹ͸͋Δ͕ɺ flush ͱ͍ͬͨ

    Ұؾʹు͖ग़͢ ؔ਺͕ͳ͍ • Θ͟Θ͟ΩϡʔΠϯάͨ͠ΓόοϑΝϦϯάͨ͠Γ͠ͳ͍͍ͯ࣌͘ʹҰؾ ʹు͖ग़͍͚ͨ͠Ͳɺݱঢ়ͩͱͦ͏͍͏ Controller ΋ͳ͍͍ͧͬͯ͏࿩Λ ͍ͯ͠Δɻ •
  17. ·ͱΊ • Stream ͷࠓੲ • Node.js Stream͔ΒWHATWG Stream΁ • WHATWG

    Stream ͷࡉ͔͍࿩ • WHATWG Stream Ͱ଍Γͯͳ͍΋ͷ΋·ͩ·ͩ͋Δ • ࠓ͸·ͩ unstable ࢖͏࣌͸ؾΛ͚ͭͯ • ࠓNode.js͸࣍ͷ10೥ʹ޲͚ͯͷมֵظɺ͜ͷ͋ͨΓͷਐԽ΋݁ߏΊ͟·͍͠ • ࠓޙͷNode.jsΛ໘ന͍ͯ͘͜͠͏ɻ