WHATWG Stream の話

Transcript

1. WHATWG Stream ͕ Node.js ʹ΍͖ͬͯͨ 2021/08/30 @ Node ֶԂ37࣌ݶ໨

2. Twitter: @yosuke_furukawa Github: yosuke-furukawa

3. Before This Talk...

4. JSConf JP 2021 ΦϯϥΠϯ։࠵༧ఆʂʂʂ 2021/11/27 ։࠵

5. https://tinyurl.com/jsconf2021

6. Stream ࠓੲ෺ޠ

7. Stream • ੲ͔Β͋ͬͨ • جຊతʹ͢΂ͯͷ΋ͷ͸͜ͷߟ͑ํͰදݱͰ ͖Δ • σʔλͷม׵ݩ͕͋ͬͯม׵͠ͳ͕Βม׵ઌ ʹૹΔ 4PVSDF

   5SBOTGPSN %FTUJOBUJPO

8. Stream • ϓϩάϥϛϯάͷ௨ৗͷϝϯλϧϞσϧ • ҰؾʹಡΈࠐΉɺҰؾʹม׵͢ΔɺҰؾʹॻ͖ग़͢ • Ͱ΋ͦΕͩͱ஗͔ͬͨΓɺϝϞϦΛଟେʹফඅͨ͠Γ͢Δ • ͳͷͰɺಡΈࠐΈͳ͕Βม׵ͯ͠ॻ͖ग़͍ͬͯ͢͏ߟ͑ํ •

   ྫ: Zip͔ΒಡΈࠐΜͰల։ͯ͠อଘ • 4PVSDF 5SBOTGPSN %FTUJOBUJPO

9. Stream // Stream Λ࡞Δ࣌ const { Readable } = require('stream');

   
 const controller = new AbortController();
 const read = new Readable({ read(size) { // σʔλΛಡΈࠐΉ࣌ͷຊॲཧ }, signal: controller.signal // cancel ͢Δ࣌ (AbortController͕࢖͑Δ) }); 


10. Stream // Stream ͔ΒσʔλΛಡΈࠐΉ࣌ readableStream.on("data", (chunk) => { // chunk

    ΛಡΈࠐΜͰͳΜ͔͢Δ }); readableStream.on("end", () => { // ऴྃॲཧ }); readableStream.on("error", () => { // ྫ֎ॲཧ });

11. Stream // pipe Ͱ chain ͯ͠ॻ͚Δɻ readableStream.pipe(transformStream).pipe(writableStream); // error ͕͋ͬͨΒ͜͏ॻ͘ɻ

    readableStream.on("error", (err) => { // ྫ֎ॲཧ }).pipe(transformStream).on("error", (err) => { // ྫ֎ॲཧ }).pipe(writableStream).on("error", (err) => { // ྫ֎ॲཧ });

12. Stream // error propagate ͕໘౗ͳΒ͜͏ॻ͚ͨΓ͢Δɻ const { pipeline } =

    require('stream'); pipeline( readableStream, transformStream, writableStream, (err) => { if (err) { console.error('Pipeline failed.', err); } else { console.log('Pipeline succeeded.'); } }, ); // ͜ΕΛ promisify ͢Ε͹ async/await Ͱ΋ॻ͚ͨΓ͢Δɻ

13. Stream // promisify ͞Εͨ΍ͭ΋͋Δɻ const { pipeline } = require('stream/promises');

    await pipeline( readableStream, transformStream, writableStream, ).catch(console.error);

14. Stream • Node.js ʹੲ͔Β͋Δߟ͑ํ͕ͩɺ Promiseͱ͏·͘౷߹Ͱ͖ͳ͍ɻ • Promise ͷ୆಄ɺٴͼ async/await ʹΑͬͯେ෼࢖ΘΕʹ͘͘ͳ͖ͬͯͨɻ

    • ੲ͸ͦΕͦ͜ Callback Ͱ EventEmitter ͳߟ͑ํ͔͠ͳ͔ͬͨ࣌୅ͩͬ ͕ͨɺࠓ͸΋͏ΈΜͳ async/await ͬͯ΍ͬͯΔ • Stream Λ AsyncIteratorʹͯ͠࢖ͬͨΓɺ pipeline/promises Λ࢖͑͹ async/await ͱ౷߹ͯ͠࢖͏͜ͱ΋Մೳʹͳ͕ͬͨɺͦΕΑΓ͸ͦ΋ͦ΋ WHATWG Stream ଆͷํࣜʹҠ؅ͯ͠͠·ͬͨ΄͏͕ ecosystem friendly ͱ͍͏ߟ͑ํ

15. WHATWG Stream • WHATWG ଆͰఏҊ͞Ε͍ͯΔ Stream • Node.js ͷ Stream

    ͷӨڹ͸ଟগड͚ͯΔͱࢥ͏͕ جຊผ෺ • ओʹॲཧͷ࠷খ୯Ґͷߟ͑ํ͕ҧ͏ɻ Node.js Stream ͸ event ʹొ࿥͞ΕͨλεΫ୯Ґɺ WHATWG ͸ Promise Λجຊ୯Ґʹ͢Δ

16. WHATWG Stream // WHATWG Stream Λ࡞Δ࣌ 
 import { ReadableStream

    } from 'node:stream/web'; const stream = new ReadableStream({ async start(controller) { // σʔλΛಡΈࠐΈ࢝ΊΔॲཧ ※ controller ͸ޙड़ } async pull(controller) {
 // ஞ࣍తʹಡΉॲཧ } async cancel() { // Ωϟϯηϧ࣌ͷॲཧ } });

17. WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // reader ΛऔΓग़ͯ͠ const

    reader = readableStream.getReader(); // Ұճ͚ͩݺͿͳΒ͜ΕͰΦοέʔ const data = await reader.read(); console.log(data);

18. WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // ஞ࣮࣍ߦͰऔΔͳΒ const reader

    = stream.getReader(); let result = ""; let chunk; do { chunk = await reader.read(); if (chunk.value != undefined) result += chunk.value; } while (!chunk.done); // ΊΜͲ͍ɺ do-while ͳΜͯॻ͖ͨ͘ͳ͍

19. WHATWG Stream // WHATWG Stream Λσʔλ͔ΒಡΈࠐΉ࣌ // ஞ࣮࣍ߦͰऔΔͳΒ stream ͕

    AsyncIterator ͳͷͰɺ for await of ͰऔΕ Δ let result = ""; for await (const chunk of stream) { result += chunk; } console.log(result);

20. WHATWG Stream // pipeThrough ͰՃ޻͠ͳ͕Β ࠷ऴతʹ writable Stream ʹॻ࣌͘ readableStream.pipeThrough(transformStream)

    // Stream .pipeTo(writableStream) // Promise .then(() => console.log("done")) .catch(console.error); // Promise ͕ؼͬͯ͘Δͬͯ͜ͱ͸async/awaitͰॻ͚Δɻ try { await readableStream.pipeThrough(transformStream).pipeTo(writableStream); console.log("done"); } catch(e) { console.error(e); }

21. WHATWG Stream example import { ReadableStream } from 'node:stream/web'; import

    { setInterval as every } from 'node:timers/promises'; import { performance } from 'node:perf_hooks'; const stream = new ReadableStream({ async start(controller) { for await (const _ of every(1000)) controller.enqueue(performance.now()); } }); for await (const value of stream) console.log(value);

22. WHATWG Stream • Byteࢦ޲ͷReadable Stream ΋࡞ΕΔ • BYOB (Bring Your

    Own Buffer) Stream • ReadableStream ࡞੒࣌ʹ type Λ 'bytes' ʹ͢Δ ͱ BYOB ʹͳΔ • ͜Ε͸ byte ୯ҐͰࡉ͔͘ಡΈࠐΉ੍ޚΛߦ͍͍ͨ ࣌ʹ࣮ࢪ͢Δ΋ͷ

23. WHATWG Stream • Queueing Strategy • ࡉ͔͘ Queueing ͢ΔઓུΛܾΊ੍ͯޚͤ͞Δ͜ͱ͕Ͱ͖Δ •

    highwatermark ͱݺ͹ΕΔ͖͍͠஋ΛܾΊͯͦΕҎ্ͩͬͨΒ backpressure Λ͔͚ͯಡΈࠐ·ͳ͘͢Δ • 4PVSDF 5SBOTGPSN %FTUJOBUJPO RVFVF RVFVF highwatermark

24. WHATWG Stream • Queueing Strategy • ByteLengthQueueingStrategy: όΠταΠ ζͷ্ݶΛܾΊͯͦ͜·ͰಡΈࠐΉઓུ •

    CountQueueingStrategy: ԿճΩϡʔʹॻ͖ ࠐΜ͔ͩͷճ਺੍ޚ

25. WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • Node.js Writable ʹ͸͋Δ͕ɺ flush ͱ͍ͬͨ

    Ұؾʹు͖ग़͢ ؔ਺͕ͳ͍ • Θ͟Θ͟ΩϡʔΠϯάͨ͠ΓόοϑΝϦϯάͨ͠Γ͠ͳ͍͍ͯ࣌͘ʹҰؾ ʹు͖ग़͍͚ͨ͠Ͳɺݱঢ়ͩͱͦ͏͍͏ Controller ΋ͳ͍͍ͧͬͯ͏࿩Λ ͍ͯ͠Δɻ •

26. WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • PipeController ͱ͔Ͳ͏Αʁ https://github.com/whatwg/streams/issues/960

27. WHATWG Stream ͷ଍Γ͍ͯ ͳ͍΋ͷ • Offset ·Ͱ write ͢Δؔ਺ͱ͔Λݕ౼த? https://github.com/whatwg/streams/issues/1128

28. WHATWG Stream • ·ͩ·ͩ discussion ͸ଓ͘ • Node.js Ͱ΋·ͩ Stability

    1: Experimental

29. ࠓͳͥ௥Ճͨ͠ͷ͔? • ͍͔ͭ͋͘Δ • Web Compat ΁ͷڧԽ • fetch ͷ࣮૷ͷͨΊͷύʔπ࡞Γ

    • deno ͳͲͷڝ߹ͷଘࡏ

30. Node.js ͜Ε͔Βͷ10೥

31. Node.js ͜Ε͔Βͷ10೥ HTTPɺͲ͛Μ͔͍ͨ͠ɻ

32. ·ͱΊ • Stream ͷࠓੲ • Node.js Stream͔ΒWHATWG Stream΁ • WHATWG

    Stream ͷࡉ͔͍࿩ • WHATWG Stream Ͱ଍Γͯͳ͍΋ͷ΋·ͩ·ͩ͋Δ • ࠓ͸·ͩ unstable ࢖͏࣌͸ؾΛ͚ͭͯ • ࠓNode.js͸࣍ͷ10೥ʹ޲͚ͯͷมֵظɺ͜ͷ͋ͨΓͷਐԽ΋݁ߏΊ͟·͍͠ • ࠓޙͷNode.jsΛ໘ന͍ͯ͘͜͠͏ɻ

33. JSConf JP 2021 ΦϯϥΠϯ։࠵༧ఆʂʂʂ 2021/11/27 ։࠵