Vim compiled to WebAssembly

- Emscripten & WebAssembly night !! #8
- 30min
- Demo: https://rhysd.github.io/vim.wasm/
- Repo: https://github.com/rhysd/vim.wasm
- Event: https://emsn.connpass.com/event/136346/

Transcript

1. Vim compiled to WebAssembly
   Emscripten & WebAssembly night !! #8
   2019/07/24
   

   View Slide

2. • ݴޠॲཧܥ΍ίʔυΤσΟλͳͲͷϓϩ
   άϥϛϯάπʔϧ͕޷͖
   • 70+ Vim plugins
   • NyaoVim, vim.wasm ͳͲͷ Vim/Neovim
   ΤσΟλͷ Web ϑϩϯτΤϯυ
   • LLVM Λ࢖ͬͨݴޠࣗ࡞ (Dachs, gocaml)
   @Linda_pp
   @rhysd
   
   
   

   View Slide

3. 1. vim.wasm ͱ͸
   2. vim.wasm ͷ࣮૷ৄࡉ
   3. ࠓޙ
   

   View Slide

4. 1. vim.wasm ͱ͸
   2. vim.wasm ͷ࣮૷ৄࡉ
   3. ࠓޙ
   

   View Slide

5. Vim ΤσΟλͱ͸
   • vi Ϋϩʔϯͱͯͭ͘͠ΒΕɼ޿͘࢖༻͞Ε͍ͯΔςΩετΤσΟλɽ
   28೥ܦͬͨࠓ΋ਐԽ͠ଓ͚͍ͯΔ
   • CUI ൛ͱ GUI ൛ͷ྆ํ͕͋Γɼଟ͘ͷ OS ʹҠ২͞Ε͍ͯΔʢ࣮૷
   ͸ C99ʣ
   • Vim script ͱ͍͏ಠࣗݴޠͰϋΠϥΠτɾϓϥάΠϯͳͲ֦ுͰ͖Δ
   > wc -l vim/src/**/*.{c,h} | grep total
   465419 total
   > wc -l vim/runtime/**/*.vim | grep total
   217978 total
   

   View Slide

6. vim.wasm ͱ͸
   Vim ΤσΟλΛ fork ͯ͠ WebAssembly ʹί
   ϯύΠϧͰ͖ΔΑ͏ʹ͠ɼϒϥ΢β͔Β࢖͑
   ΔΑ͏ʹ͢ΔϓϩδΣΫτ
   https://github.com/rhysd/vim.wasm
   

   View Slide

7. ϒϥ΢βͰࢼͤ·͢
   https://rhysd.github.io/vim.wasm/
   (Chrome ͰΞΫηε͍ͯͩ͘͠͞ʣ
   

   View Slide

8. Ԡ༻ྫ
   • ֎෦ϓϥάΠϯΛࢼ͢
   • vim-buffer: https://tinyurl.com/y48abmbn
   • https://github.com/rhysd/react-vim-wasm
   • vim.wasm ͷ React ίϯϙʔωϯτ
   • https://github.com/rhysd/vimwasm-try-plugin
   • vim.wasm Ͱ֎෦ϓϥάΠϯΛࢼͨ͢Ίͷ
   URL ੜ੒
   • https://github.com/nat-chan/vim.wasm.ipynb
   • Jupyter Notebook ͷΤσΟλͰ Vim Λ࢖
   ͏
   
   
   

   View Slide

9. vim.wasm ͷػೳ
   • ΄΅શͯͷ Vim ͷػೳʢߏจϋΠϥΠτ΍ϓϥάΠϯͳ
   Ͳʣ͕ϒϥ΢βͰಈ͘
   • navigator.clipboard Λ࢖ͬͨΫϦοϓϘʔυͷαϙʔτ
   ΍ɼϑΝΠϧͷ΍ΓͱΓʢD&DɼVim ͰฤूதͷϑΝΠ
   ϧͷμ΢ϯϩʔυʣ
   • Indexed DB Λ࢖ͬͯ .vimrc ΛӬଓԽ
   • ϦϞʔτͷϑΝΠϧΛ
   
   
   

   View Slide

10. vim.wasm ͷϞνϕʔγϣϯ
    • Vim ͸޿͘࢖ΘΕ͍ͯΔ͕ɼ·ͩ΢ΣϒͰ͸࢖ΘΕ
    ͍ͯͳ͍
    • Wasm ʹطଘͷΤσΟλΛϙʔτͯ͠ಈ͔͢ํࣜͷ
    Web ίʔυΤσΟλ͸ʢ୳ͨ͠ݶΓͰ͸ʣͳ͔ͬͨ
    ͷͰɼͲΕ͘Β͍͏·͘ಈ͔͘ڵຯ͕͋ͬͨ
    • WebAssembly, Web Worker, ES Modules ͳͲͷ࠷
    ৽࢓༷Λ࣮ફతʹ࢖ͬͯΈ͍ͨ
    
    
    

    View Slide

11. ͳͥ Vim Λϙʔτ͢Δͷ͔ʁ
    • ଟ͘ͷ؀ڥʹҠ২͞Ε͍ͯΔͨΊɼ͔ͳΓϙʔλϒϧ
    ͳ C Ͱॻ͔Ε͍ͯΔ
    • feature ͱ͍͏֓೦͕͋Γɼtiny, small, normal, big,
    huge ͷ5ஈ֊ͷػೳηοτΛఏڙ͍ͯ͠Δʢhuge ͕
    શ෦ೖΓʣɽ࠷খηοτͷ tiny ͔ΒҠ২Λ࢝ΊΒΕ
    Δ
    • Vim ޷͖
    
    
    

    View Slide

12. npm ύοέʔδ vim-wasm
    https://www.npmjs.com/package/vim-wasm
    ES ModuleɼWeb Worker εΫϦϓτɼwasm ό
    ΠφϦΛ1ͭͷ npm ύοέʔδʹɽ؆୯ʹ
    vim.wasm Λϖʔδʹ૊ΈࠐΊΔ
    
    
    

    View Slide

13. npm ύοέʔδ vim-wasm
    
    
    hello vim.wasm
    
    
    
    
    
    
    
    
    
    import { VimWasm } from '/path/to/vim-wasm/vimwasm.js';
    const vim = new VimWasm({
    workerScriptPath: '/path/to/vim-wasm/vim.js',
    canvas: document.getElementById('vim-screen'),
    input: document.getElementById('vim-input'),
    });
    vim.start();
    JOEFYIUNM
    JOEFYKT
    
    
    

    View Slide

14. 1. vim.wasm ͱ͸
    2. vim.wasm ͷ࣮૷ৄࡉ
    3. ࠓޙ
    

    View Slide

15. vim.wasm ͷߏ੒
    WJNXBTNKT
    ΤϯτϦʔϙΠϯτɽ8FC8PSLFSͷϥΠϑαΠΫϧΛ؅ཧ͢Δɽ
    %0.ʹΞΫηεͰ͖Δɽ8FC8PSLFS͔Βඳը৘ใΛड͚औΓɼ
    DBOWBT΁ͷඳըΛߦͬͨΓɼ,FZCPBSE&WFOUΛड͚ͨΓ͢Δɽ
    WJNKT
    8FC8PSLFS಺Ͱ
    ૸ΔεΫϦϓτɽ
    8BTNϑΝΠϧΛ
    ಡΈࠐΈɼ7JNͱ΍
    ΓऔΓ͢Δ
    WJNXBTN
    7JNͷ$ίʔυ͔ΒίϯύΠϧ
    ͨ͠8BTNόΠφϦɽ
    ΤσΟλͷίΞϩδοΫʢςΩε
    τόοϑΝͷ؅ཧ΍ೖྗͷॲཧɼ
    εΫϦʔϯͷඳըܭࢉͳͲʣ
    new Worker("vim.js");
    wasm_main();
    postMessage(); Atomics.notify();
    gui_wasm_*();
    vimwasm_*();
    ϝΠϯεϨου
    ϫʔΧʔεϨου
    
    
    

    View Slide

16. $BMDVMBUF
    *OQVU
    4FRVFODF
    ,FZEPXO
    &WFOU
    *OQVUUP
    #V⒎FS
    &WFOU
    -PPQ
    6TFS
    ,FZ*OQVU
    %SBX5FYU
    %SBX3FDU
    4DSPMM3FHJPO
    ʜ
    %SBX&WFOU
    5ZQF4DSJQU
    7JN 8BTN
    
    %JTQMBZ $BMDVMBUF
    3FOEFSJOH
    $PSF
    &EJUPS
    -PHJD
    5ZQF4DSJQU
    $BMDVMBUF
    DBOWBT
    SFOEFSJOH
    3FOEFSUP
    DBOWBT
    POBOJNBUJPOGSBNF
    4IBSFE#V⒎FS
    ,FZ%BUB
    LFZOBNF
    DIBSDPEF
    NPEJpFST
    
    8BJU-PBE
    1PTU
    .FTTBHF
    .BJO5ISFBE 8PSLFS5ISFBE
    WJNXBTNKT
    WJNKT WJNXBTN
    4UPSF
    
    
    

    View Slide

17. ϝΠϯεϨου
    • ී௨ͷ JavaScript ϥΠϒϥϦʢES Moduleʣ
    • Worker ͷىಈͱऴྃ
    • ϫʔΧʔεϨουͱͷ΍ΓऔΓ
    • ඳըΠϕϯτΛ animation frame Ͱ ʹඳըʢcustom drawer Λ࣮૷͢Δ͜ͱ
    ΋Ͱ͖Δʣ
    • Ωʔೖྗ
    • ΢Οϯυ΢ͷϦαΠζ
    • ...
    • ϫʔΧʔεϨου͔Βͷ௨஌͸ onmessage Ͱड͚ɼϫʔΧʔεϨου΁ͷ௨஌͸
    SharedMemoryBuffer ͱ Atomics API Λ࢖͏
    import { VimWasm } from
    '/path/to/vim-wasm/vimwasm.js';
    const vim = new VimWasm({ ... });
    vim.start();
    IUUQTHJUIVCDPNSIZTEWJNXBTNCMPCXBTNXBTNWJNXBTNUT
    
    
    

    View Slide

18. ϫʔΧʔεϨου
    • emcc -o vim.js ͰϏϧυͨ͠ Vim
    • ϝΠϯεϨουͱͷ΍ΓऔΓ͸ --js-library Ͱ૊ΈࠐΜͩ
    JavaScript Ͱߦ͏
    • Vim (vim.wasm) ͱ JS ϥϯλΠϜ (runtime.ts) ͱ͸ؔ਺ݺͼ
    ग़͠Ͱ΍ΓऔΓɽemscripten ͕ͭͳ͍Ͱ͘ΕΔ
    • Atomics API Λ࢖͍ɼϝΠϯεϨου͔Βͷ௨஌Λ
    SharedArrayBuffer Ͱಉظతʹड͚औΔ
    IUUQTHJUIVCDPNSIZTEWJNXBTNCMPCXBTNXBTNSVOUJNFUT
    
    
    

    View Slide

19. ϫʔΧʔεΫϦϓτͷϏϧυ
    $MBOH
    NBJOD
    HVJD
    UFSND
    PT@VOJYD
    HVJ@XBTN
    ɾ
    ɾ
    ɾ
    NBJOP
    HVJP
    UFSNP
    PT@VOJYP
    HVJ@XBTNP
    ɾ
    ɾ
    ɾ
    $4PVSDFT
    --7.
    CJUDPEF
    WJNCD
    --7.
    CJUDPEF
    QSFUT
    NBJOUT
    WJNXBTN
    WJNEBUB
    5ZQF4DSJQU
    3VOUJNF
    SVOUJNFUT
    FYFDVUBCMF
    MMWNMJOL
    #JOBSZFO
    NBJOKT
    WJNKT
    JOEFYIUNM
    TUZMFDTT
    VTSMPDBMTIBSFWJN
    WJNUVUPS
    
    
    

    View Slide

20. Vim ͷ C ଆͷ࣮૷
    Vim ͸ Makefile ͰϦϯΫ͢ΔϑΝΠϧΛ੾Γସ
    ͑Δ͜ͱͰɼGUI ࣮૷Λ੾Γସ͑Δ͜ͱ͕Ͱ͖Δ
    → ৽͍͠ GUI ࣮૷ͱͯ͠ gui_wasm Λ௥Ճ͠ɼ
    ͦͷதͰ JavaScript ଆͷؔ਺ΛݺͿ
    ΤσΟλͷίΞ෦෼ʹखΛೖΕͳͯ͘ྑ͍
    
    
    

    View Slide

21. Vim ͷ C ଆͷ࣮૷
    SVOUJNFKT
    ϝΠϯεϨου͔Βೖ
    ྗΛड͚औΔ
    HVJ@XBTND
    (6*ॲཧʹԠͯ͡ɼదٓ
    +BWB4DSJQUଆʹॲཧΛ౤͛
    Δ
    (6*࣮૷ؔ਺Λݺͼ
    ग़ͯ͠(6*ॲཧ
    7JNͷίΞϩδοΫ
    ೖྗΛॲཧͯ͠ΤσΟλͷঢ়
    ଶΛΞοϓσʔτ
    ʢςΩετόοϑΝɼεΫϦʔ
    ϯඳըͳͲʣ
    $ͷؔ਺ݺ
    ͼग़͠
    HVJ@XBTN@
    
    SVOUJNFKT
    w ඳըΠϕϯτΛϝΠϯ
    εϨουʹ౤͛Δ
    w "UPNJDTͰಉظXBJU
    +4ͷؔ਺ݺ
    ͼग़͠
    WJNXBTN@
    
    
    
    

    View Slide

22. Vim ͷϝΠϯϧʔϓ
    ΤσΟλͷίΞϩδοΫ
    w ςΩετόοϑΝͷߋ৽
    w εΫϦʔϯඳը
    w 7JNTDSJQUͷධՁ
    w
    (6*ॲཧ
    w ը໘ඳը
    w
    Ϣʔβͷೖྗ଴ͪ
    w Ωʔೖྗ
    w ը໘ͷϦαΠζ
    w
    ಉظతͳೖྗ଴͕ͪඞཁ
    
    
    

    View Slide

23. ಉظతͳೖྗ଴͕ͪඞཁ
    • Vim ͸ GUI ࣮૷Λ௨ͯ͠ϢʔβͷೖྗΛಉظతʹ଴ͭɽ
    ଟ͘ͷ GUI ϑϨʔϜϫʔΫ࣮૷Ͱ͸…
    • ϑϨʔϜϫʔΫ͕࣋ͭϝΠϯϧʔϓ͕؅ཧ
    • ୯ʹ sleep() Ͱ polling ͯ͠଴ͭ
    • WebAssembly ΍ JavaScript ͸ϒϩοΩϯάૢ࡞ʹ͸͔
    ͳΓݫ͘͠ɼجຊతʹඇಉظॲཧ͔͠Ͱ͖ͣϒϩοΫͰ
    ͖ͳ͍
    
    
    

    View Slide

24. ಉظతͳೖྗ଴͕ͪඞཁ
    • vim.wasm Ͱ͸࠷ॳ Emterpreter Λ࢖͍ɼ
    emscripten_sleep Λ࢖͍ͬͯͨ
    • ࣮ߦ͕஗͍
    • όΠφϦαΠζ͕๲ΕΔ
    • Emterpreter Ͱ૸ΒͤΔؔ਺ͷڊେͳϦετΛखͰ؅ཧ͢Δඞཁ͕͋Δ
    • ࣮૷͕ buggy
    • https://speakerdeck.com/rhysd/vim-ported-to-webassembly-vimconf-2018
    • Web Worker ಺Ͱ JS ͷ Atomics API Λ࢖ͬͯಉظ
    తʹ଴ͭ
    
    
    

    View Slide

25. Atomics API
    • JavaScript ͰεϨουؒͷ
    ಉظॲཧΛߦ͏ͨΊͷ௿Ϩ
    ϕϧ API
    • εϨουؒͷڞ༗ϝϞϦͱ
    ͯ͠ݻఆ௕ͷ int32 ഑ྻͷ
    SharedArrayBuffer Λ࢖͏
    • Web Worker ݶఆͰɼόο
    ϑΝ্ͷ஋͕มΘΔͷΛ
    Atomics.wait Ͱಉظతʹ଴
    ͯΔ (futex(7))
    const worker = new Worker('worker.js');
    const buffer = new Int32Array(
    new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT * 1)
    );
    new Worker('worker.js').postMessage(buffer);
    setTimeout(() => {
    // ࠷ॳͷཁૉΛ 0 ! 1 ʹॻ͖׵͑Δ
    Atomics.store(buffer, 0, 1);
    // ཁૉ͕ॻ͖׵Θͬͨ͜ͱΛ௨஌ͯ͠ϫʔΧʔΛى͜͢
    Atomics.notify(buffer, 0, 1);
    }, 1000);
    onmessage = e => {
    const buffer = e.data;
    // 1ཁૉ໨Λ0ΫϦΞ
    Atomics.store(buffer, 0, 0);
    // ͜͜Ͱ buffer ͕ॻ͖׵ΘΔ·Ͱ࣮ߦ͕ࢭ·Δ
    Atomics.wait(buffer, 0, 0);
    };
    ϝΠϯεϨου
    ϫʔΧʔεϨου
    
    
    

    View Slide

26. Atomics API
    • όοϑΝ௕͸ݻఆͳͷͰɼϝΠϯɾϫʔΧʔؒͷՄม௕σʔλ
    ʢe.g. ςΩετʣ͸ผ్όοϑΝΛ༻ҙͯ͠΍Δඞཁ͕͋Δ
    ϝΠϯεϨου ϫʔΧʔεϨου
    BCDEFΛૹΓ͍ͨ
    CZUFTόοϑΝ΄͍͠
    ੜ੒
    ڞ༗όοϑΝ
    όοϑΝͷࢀরΛ౉͢
    ॻ͖ࠐΈ
    όοϑΝʹॻ͍ͨΑ
    ($
    
    ಡΈऔΓ
    BCDEFΛड͚औͬͨ
    

    View Slide

27. Atomics API Λ࢖͏ܽ఺
    • Spectre ੬ऑੑͷͨΊɼݱঢ়Ͱ͸΄΅ Chromeʢͱ Chromium
    ϕʔεϒϥ΢βʣݶఆ
    • Ϣʔβ͔ΒͷೖྗΛಉظతʹ଴ͭͨΊɼίʔϧόοΫ͕ൃՐ͢Δ
    λΠϛϯά͕ͳ͘ͳΔ
    • requestAnimationFrame() ൃՐ͠ͳ͍
    • onmessage ίʔϧόοΫൃՐ͠ͳ͍
    • Մม௕σʔλͷ΍ΓऔΓ͕൥ࡶ
    • ϒϥ΢βͷόάΛ౿Ή
    • @nhiroki_ ͞Μʹ৭ʑରԠ͍͖ͯͨͩ͠·ͨ͠
    
    
    

    View Slide

28. 1. vim.wasm ͱ͸
    2. vim.wasm ͷ࣮૷ৄࡉ
    3. ࠓޙ
    

    View Slide

29. ࠓޙ
    • emscripten LLVM όοΫΤϯυʹ৐Γ׵͑
    • ৽͍͠ Asyncify
    • ύϑΥʔϚϯεվળʢC ଆͷ tracing ΛͲ͏͢
    Δ͔͕՝୊ʣ
    • Web Components ͷΧελϜཁૉԽ
    
    
    

    View Slide

30. emscripten LLVM Wasm όοΫ
    Τϯυ
    • https://v8.dev/blog/emscripten-llvm-wasm
    • Ҏલ͸ LLVM bitcode ͔ΒҰ୴ asm.js Λܦ༝ͯ͠
    Wasm ʹม׵͍ͯͨ͠ʢfastcompʣ͕ɼ௚઀
    Wasm ʹม׵Ͱ͖ΔΑ͏ʹͳΔ
    • Կ΋͠ͳͯ͘΋όΠφϦαΠζͱϦϯΫ͕࣌ؒվળɽ
    ຤ඌݺͼग़͠࠷దԽͳͲͷ Wasm ͷ৽ػೳ͕͙͢
    ࢖͑Δ
    
    
    

    View Slide

31. • Ϗϧυ࣌ؒ
    • όΠφϦαΠζ
    emscripten LLVM Wasm όοΫ
    Τϯυ
    
    NBLFDQVTUPUBM
    FNDDDQVTUPUBM
    
    NBLFDQVTUPUBM
    FNDDDQVTUPUBM
    ˠ,J#
    ˠ,J#
    
    
    

    View Slide

32. ৽͍͠ Asyncify
    • چ Asyncify, Emterpreter ʹ୅ΘΔɼC ͰඇಉظॲཧΛ࣮૷͢Δͨ
    Ίͷ࢓૊ΈɽίϧʔνϯͷΑ͏ʹॲཧΛதஅɾ࠶։Ͱ͖Δ
    • https://kripken.github.io/blog/wasm/2019/07/16/asyncify.html
    • େن໛ͳม׵Λڬ·ͳ͍ͷͰɼ࣮ߦͷΦʔόʔϔου͕͔ͳΓখ
    ͍͞
    • Wasm ʹΤΫεϙʔτ͞Εͨ intrinsic ؔ਺Ͱ rewind/unwind ͢Δ
    • Binaryen ͷ࠷దԽύεͰ rewind/unwind ͞ΕΔ͔΋͠Εͳ͍ؔ
    ਺ͷ caller&callee ͰελοΫΛอଘ͢ΔॲཧΛίʔυʹࠩ͠ࠐΉ
    
    
    

    View Slide

33. ৽͍͠ Asyncify
    • vim.wasm Ͱ΋ Atomics ʹΑΔಉظతͳ wait ͷ୅ΘΓʹ
    ಋೖΛݕ౼த
    •
    requestAnimationFrame ͕࢖͑ΔΑ͏ʹͳΔͷͰɼ
    OffscreenCanvas ʹΑΔը໘ඳը͕ߦ͑ΔΑ͏ʹͳΔ
    •
    Atomics API Λ࢖Θͳ͍ͷͰɼChrome Ҏ֎ͷϒϥ΢
    βͰ΋ಈ͘Α͏ʹͳΔ
    •
    όΠφϦαΠζ͕݁ߏେ͖͘ͳΓͦ͏ʁʢཁௐࠪʣ
    
    
    

    View Slide