2度もゼロから書き直して、やっとブラウザでぬるぬる動くAIに辿り着いた話

Transcript

1. 2౓΋θϩ͔Βॻ͖௚ͯ͠ɺ
 ΍ͬͱϒϥ΢βͰ͵Δ͵Δಈ͘AIʹḷΓண͍ͨ࿩ Χϛφγ StatHackΧϯύχʔ Ҫ্ஐ༟ TechBowl x Χϛφγ ߹ಉษڧձ @20250530
2. ΧϛφγͰ͸ࣗࣾͰAI։ൃΛߦ͍ͬͯ·͢ɻ ࠓ೔͸AIʹΑΔϦΞϧλΠϜը૾ॲཧΛ
 ϒϥ΢βͰಈ͔͢ํ๏ʹ͍ͭͯ
 ࣦഊஊΛަ͓͑ͭͭ࿩͠·͢ɻ
3. ԿΛ࡞͔ͬͨʁ ৯඼ϥϕϧͷϦΞϧλΠϜݕࠪΞϓϦ
4. ۤ࿑࿩ 2౓΋θϩ͔ΒίʔυΛॻ͖௚͢͸Ίʹ 1ճ໨͸ҙਤత͕ͩͬͨɺ2ճ໨͸૝ఆ͍ͯ͠ͳ͔ͬͨ 12݄ AI Ҏ֎ͷ Frontend & Backend AI ʢPythonʣ AI ʢTypeScriptʣ AI ʢC++ʣ 1݄
5. AI࣮૷1ճ໨ in Python Python ͰϓϩτλΠϓ࡞੒ ਫ਼౓ݕূͷ໨తͰ։ൃɻϥΠϒϥϦ͕ॆ࣮͍ͯ͠ΔͷͰߴ଎ʹݕূՄೳɻ ։ൃظؒɿ2िؒ
6. ͳͥɺPython ίʔυΛࣺ͔ͯͨʁ Server ଆͰ͸ͳ͘ɺClient ଆͰॲཧ͔ͨͬͨ͠ ϦΞϧλΠϜͰॲཧ͔ͨͬͨ͠ʢUX໘Ͱ΋ਫ਼౓໘Ͱ΋ʣ ௨৴ʹ͕͔͔࣌ؒΔʂ ɹॲཧ͕ߴ଎ͳͷͰ
 ܁Γฦ͠ݕࠪͰ͖Δ Server ʹ AI ͕͋Δ৔߹ Client ʹ AI ͕͋Δ৔߹
7. AI࣮૷2ճ໨ in TypeScript TypeScript ͰΞϓϦʹ௚઀࣮૷ OpenCV.js ͱ͍͏ը૾ॲཧϥΠϒϥϦΛ࢖༻ <script src="opencv.js" type="text/javascript"></script> // cv.Mat ͱ͍͏ܗࣜͰը૾Λѻ͏ function processImage(image: cv.Mat): cv.Mat { // RGB -> നࠇը૾ʹม׵ let grayImage = new cv.Mat(); cv.cvtColor(image, grayImage, cv.COLOR_RGBA2GRAY); // 2ഒʹ֦େ let largeImage = new cv.Mat(); const largeImageSize = new cv.Size(grayImage.cols * 2, grayImage.rows * 2); cv.resize(grayImage, largeImage, largeImageSize, 0, 0, cv.INTER_LINEAR); return largeImage; }
8. OpenCV.js Ͳ͏ͩͬͨʁ ΊͪΌͪ͘ΌϝϞϦϦʔΫͨ͠ OpenCV.js ͸ϝϞϦΛखಈͰղ์͠ͳ͍ͱ͍͚ͳ͍... // cv.Mat ͱ͍͏ܗࣜͰը૾Λѻ͏ function processImage(image: cv.Mat): cv.Mat { // RGB -> നࠇը૾ʹม׵ let grayImage = new cv.Mat(); cv.cvtColor(image, grayImage, cv.COLOR_RGBA2GRAY); // 2ഒʹ֦େ let largeImage = new cv.Mat(); const largeImageSize = new cv.Size(grayImage.cols * 2, grayImage.rows * 2); cv.resize(grayImage, largeImage, largeImageSize, 0, 0, cv.INTER_LINEAR); grayImage.delete(); return largeImage; } Delete Λ๨ΕΔͱϝϞϦʹ࢒Γଓ͚Δ
9. OpenCV.jsɺͳΜͰͦΜͳ࢓༷ͳͷ...ʁʢ1/3ʣ OpenCV.js ͕ WebAssembly (Wasm) Λ࢖͍ͬͯΔ͔Β Wasm: 
 C++΍RustͷίʔυΛίϯύΠϧͨ͠όΠτίʔυɻ
 ϒϥ΢β্ͰωΠςΟϒʹ͍ۙߴ଎ͳॲཧ͕࣮ݱՄೳɻ
10. OpenCV.jsɺͳΜͰͦΜͳ࢓༷ͳͷ...ʁʢ2/3ʣ OpenCV.js ͕ WebAssembly (Wasm) Λ࢖͍ͬͯΔ͔Β Wasm Ͱ֬อ͞ΕͨϝϞϦ͸ TS ͷ GC ͷର৅֎ let image = new cv.Mat(); cv::Mat* image = new cv::Mat(); TypeScript ଆͷॲཧ Wasm ଆͷॲཧ TypeScript ͷϝϞϦ Wasm ͷϝϞϦ image ͷϙΠϯλ image σʔλ delete Λݺͼ๨Εͨ৔߹
11. OpenCV.jsɺͳΜͰͦΜͳ࢓༷ͳͷ...ʁʢ3/3ʣ OpenCV.js ͕ WebAssembly (Wasm) Λ࢖͍ͬͯΔ͔Β Wasm Ͱ֬อ͞ΕͨϝϞϦ͸ TS ͷ GC ͷର৅֎ let image = new cv.Mat(); image.delete(); cv::Mat* image = new cv::Mat(); delete image; TypeScript ଆͷॲཧ Wasm ଆͷॲཧ TypeScript ͷϝϞϦ Wasm ͷϝϞϦ image ͷϙΠϯλ image σʔλ delete ΛݺΜͩ৔߹
12. AI࣮૷3ճ໨ɺͲ͏͢Δ͔ʁ Wasm ʹશ෦ͷॲཧΛด͡ࠐΊͪΌ͓͏ C++ ͰAIݕࠪϞδϡʔϧؙ͝ͱ Wasm ʹ͢Δ͜ͱʹ • Wasm ʹΑΔߴ଎ͳॲཧΛҡ࣋ • खಈ delete ΛۃྗݮΒ͢
13. AI࣮૷3ճ໨ in C++ ͷ݁຤ Wasm ࡞ͬͯΈͯͲ͏ͩͬͨʁ • ϝϞϦϦʔΫ͕ى͜Γʹ͘͘ͳͬͨ
 
 
 • େن໛σʔλͰϝτϦΫεऔΕΔΑ͏ʹͳͬͨͷͰAIਫ਼౓վળ΋ޮ཰Խͨ͠ const labelChecker = new LabelChecker(); const result = labelChecker.check(image); result.delete(); labelChecker.delete();
14. ͍͞͝ʹ ·ͱΊ • Wasm Λ࢖͏࣌͸ delete ͕ඞཁ͔Ͳ͏͔͔֬ΊΑ͏ • ϒϥ΢βͰಈ͘ AI Λࣗࣾ։ൃ͢ΔͳΒ Wasm ʹ͢Δͷ͸݁ߏΞϦͳબ୒ࢶ ແࣄ׬੒ͯ͠νʔϜͰϩϒελʔΛ৯΂ͨ࣌ͷҰຕ
15. Appendix: ·ͩ΍Ε͍ͯͳ͍͜ͱ ϚϧνεϨουʹΑΔߴ଎Խ wasm ଆͰ͸ϚϧνεϨουͷରԠ͕ൺֱత؆୯͕ͩͬͨɺ
 ΞϓϦଆΛ Cross-origin isolated ͱ͍͏ঢ়ଶʹ͢Δඞཁ͕͋Γɺݱঢ়Ͱ͖͍ͯͳ͍ JSɿϝΠϯεϨου wasm SharedArrayBuffer JSɿWeb Worker wasm JSɿWeb Worker wasm worker ݺͼग़͠