Assembling the Future: crafting the missing pieces of the Ruby on Wasm puzzle

Dive into the frontier of web innovation with a mission to bring Bundler, Rack, Rake, and more Ruby ecosystem tools into the browser! This session unveils the potential of Ruby.wasm, pushing past the limits of WASI to reshape how we think about web development. Explore the possibilities of running essential Ruby development tools directly in your browser, turning the dream of a fully interactive Ruby development environment on the web into reality.

Discover strategies for integrating Bundler for gem management, enabling Rack for web server interfacing, and leveraging Rake for automated tasks—all within the browser. We’ll navigate the challenges of virtual file systems, dynamic gem loading, and network interactions through Ruby.wasm, offering insights into overcoming these hurdles.

We’ll shine a light on the critical gaps in the ecosystem and the essential components still needed to make running comprehensive Ruby tools in the browser—from a theoretical dream to a practical, everyday reality. Get ready for a journey that promises to expand your understanding of what’s possible with Ruby.wasm, setting the stage for a future where the web is powered by Ruby’s elegance and versatility. Buckle up!

Transcript

2. I’ve seen things you people wouldn’t believe…
3. Node.js running in browsers, powered by WebAssembly…
4. I watched bug reproductions, quick experiments, live documentation examples…
5. All those moments, available for JavaScript, but missing to Ruby…
6. like tears in rain…
7. But it’s not time to die. It’s time to change.
8. Assembling the Future: crafting the missing pieces of the Ruby on Wasm puzzle Svyatoslav Kryukov, Evil Martians EuRuKo, 2024
9. Early in the 21st Century…
10. the Ruby Association advanced programming evolution into the Wasm phase—a runtime virtually identical to native code —known as ruby.wasm
11. RunRuby.dev
12. Agenda Wasm & WASI What ruby.wasm is and why it has it’s limitations How to run ruby.wasm in the Browser How to overcome some of those limitations
13. WebAssembly (Wasm) A crimson nothingness began to compile. Runtime. What does it feel like to be part of the runtime? Runtime. > ...
14. WebAssembly (Wasm) Binary instruction format for a stack-based virtual machine Portable, Fast, Sandboxed Still in development
15. Wasm as LLVM Target
16. Wasm Runtimes Browsers Wasmtime Wasmer and more…
17. Wasm module https://wasdk.github.io/wasmcodeexplorer/
18. Wasm in the Browser let memory = null; const print = (ptr) => { const view = new Uint8Array(memory.buffer); console.log( new TextDecoder().decode(view.subarray(ptr, view.indexOf(0, ptr))) ); } WebAssembly.instantiateStreaming( fetch("hello.wasm"), {sys: {print}} ).then((mod) => { memory = mod.instance.exports.memory; mod.instance.exports.main(); });
19. WASI Feel that in your runtime. WASI. What does it feel like to be part of the sandbox? WASI. > ...
20. WASI Platform independent, secure, portable interface WASI defines a limited list of standard syscalls Still in development Other ABIs: Emscripten, GoJS
21. WASI Preview 1 Example: fd_write(fd: fd, iovs: ciovec_array) -> Result<size, errno> Write to a file descriptor. Note: This is similar to writev in POSIX. Like POSIX, any calls of write (and other functions to read or write) for a regular file by other threads in the WASI process should not be interleaved while write is executed.
22. WASI Architecture module-side: wasi-sdk injects WASI functions into the wasm module host-side: native implementation or shims
23. WASI Shims https://github.com/bjorn3/browser_wasi_shim fd_write(data: Uint8Array): { ret: number; nwritten: number } { if (this.file.readonly) return { ret: wasi.ERRNO_BADF, nwritten: 0 }; if (this.file_pos + BigInt(data.byteLength) > this.file.size) { const old = this.file.data; this.file.data = new Uint8Array( Number(this.file_pos + BigInt(data.byteLength)), ); this.file.data.set(old); } this.file.data.set(data, Number(this.file_pos)); this.file_pos += BigInt(data.byteLength); return { ret: 0, nwritten: data.byteLength }; }
24. WASI Shims https://github.com/bjorn3/browser_wasi_shim let args = ["bin", "arg1", "arg2"]; let env = ["FOO=bar"]; let fds = [stdin, stdout, stderr, new PreopenDirectory(".", [ ["main.rb", mainRb], ])]; let wasi = new WASI(args, env, fds); let wasm = await WebAssembly.compile(fetch("bin.wasm")); let inst = await WebAssembly.instantiateStreaming(wasm, { "wasi_snapshot_preview1": wasi.wasiImport, }); wasi.start(inst);
25. WASI Preview 2 wasi-io wasi-clocks wasi-random wasi-filesystem wasi-sockets wasi-cli wasi-http Shim: https://github.com/WebAssembly/WASI/tree/main/wasip2 WASI Preview 2 contains the following APIs:
26. I can fix that
27. What is ruby.wasm? CRuby compiled to WebAssembly Number of patches in the CRuby sourcecode GC, Fibers, Exceptions catch them all: #if defined(__wasm__) WASI: no threads, networking, dynamic linking, etc RubyKaigi 2022 talk by Yuta Saito
28. ruby.wasm & dependencies Buildtime: Packaging system (see EuRuKo 2024 talk by Yuta Saito) Runtime: dependencies installation via network
29. But how to do it without networking?
30. ruby.wasm & JS interop Export Ruby functions to JavaScript ( @ruby/wasm-wasi npm package) Import JavaScript functions into Ruby (gem js ) Bidirectional data conversion between Ruby and JavaScript (ABI via wit-bindgen )
31. ruby.wasm & JS interop window.MyValue = "🐐" const value = window.rubyVM.eval(` require "js" name = JS.global["window"]["MyValue"] "Hello #{name}" `); console.log(value.toString()) // => "Hello 🐐"
32. Playground implementation Within modules interlinked. Do you dream of JavaScript and Ruby coexisting? Interlinked. > ...
33. Playground implementation Run ruby.wasm in browser Share FS between JS & Ruby Install gems via Bundler Bonus: Spin up a server
34. Running ruby.wasm in the Browser <html> <!-- dist/browser.script.iife.js --> <script src="https://cdn.jsdelivr.net/npm/@ruby/3.3-wasm-wasi@2.6.2/dist/browser.script.iife.js"></sc <script type="text/ruby"> require "js" puts RUBY_VERSION # (Printed to the Web browser console) JS.global[:document].write "Hello, world!" </script> </html>
35. Enhance 2:16 (browser.script.iife.js) <script type="module"> import { DefaultRubyVM } from "@ruby/wasm-wasi/browser" const response = fetch(`<...>/ruby+stdlib.wasm`); const module = await WebAssembly.compileStreaming(response); const { vm } = await DefaultRubyVM(module, {}); vm.eval(` require "js" puts RUBY_VERSION JS.global[:document].write "Hello, world!" `) </script>
36. Enhance 5:26 (DefaultRubyVM) <script type="module"> import { WASI, File, OpenFile } from "@bjorn3/browser_wasi_shim"; import { RubyVM } from "@ruby/wasm-wasi"; // ... const wasi = new WASI(args, env, fds); const imports = { wasi_snapshot_preview1: wasi.wasiImport, }; const vm = new RubyVM(); vm.addToImports(imports); const instance = await WebAssembly.instantiate(module, imports); // ... </script>
37. Playground implementation Run ruby.wasm in browser Share FS between JS & Ruby Install gems via Bundler Bonus: Spin up a server
38. FS sharing <script type="module"> // ... const encoder = new TextEncoder("utf-8") const mainRb = new File(encoder.encode('puts "Hello"')) const fds = [ new OpenFile(new File([])), // stdin new OpenFile(new File([])), // stdout new OpenFile(new File([])), // stderr new PreopenDirectory("/", new Map( ["main.rb", mainRb], )), ]; </script>
39. FS sharing <script type="module"> // ... vm.eval('puts File.read("/main.rb")') // -> prints the file content </script>
40. FS sharing <script> // ... const editor = CodeMirror.fromTextArea(<...>) editor.on("change", function (event) { encoder = new TextEncoder("utf-8") currentFile.data = encoder.encode(event.doc.getValue()); }); </script>
41. FS sharing <script> // ... runButton.addEventListener("click", () => { vm.evalAsync('eval(File.read("/main.rb"))').then( (result) => (resultBlock.innerText += `=> ${result}\n`) ); }); </script>
42. Files main.rb Run Code => Hello EuRuKo 2024 str = "EuRuKo 2024" "Hello #{str}" 1 2 3
43. FS sharing FS is just a in-memory JavaScript map It’s possible to manipulate FS from both JS and Ruby
44. Playground implementation Run ruby.wasm in browser Share FS between JS & Ruby Install gems via Bundler Bonus: Spin up a server
45. Bundler support bundle install require "bundler" require "bundler/cli" require "bundler/cli/install" Bundler::CLI::Install.new({path: "./gems"}).run
46. Files main.rb Gemfile Run Code => cannot load such file -- socket <internal:/usr/local/lib/ruby/3.3.0/rubygems/c ore_ext/kernel_require.rb>:136:in `require' <internal:/usr/local/lib/ruby/3.3.0/rubygems/c ore_ext/kernel_require.rb>:136:in `require' /usr/local/lib/ruby/3.3.0/net/protocol.rb:22:i n `<top (required)>' <internal:/usr/local/lib/ruby/3.3.0/rubygems/c ore_ext/kernel_require.rb>:136:in `require' <internal:/usr/local/lib/ruby/3.3.0/rubygems/c ore_ext/kernel_require.rb>:136:in `require' /usr/local/lib/ruby/3.3.0/net/http.rb:23:in `< top (required)>' <internal:/usr/local/lib/ruby/3.3.0/rubygems/c ore_ext/kernel_require.rb>:136:in `require' <internal:/usr/local/lib/ruby/3.3.0/rubygems/c ore_ext/kernel_require.rb>:136:in `require' /usr/local/lib/ruby/3.3.0/bundler/vendored_net _http.rb:9:in `rescue in rescue in <top (requi red)>' /usr/local/lib/ruby/3.3.0/bundler/vendored_net _http.rb:6:in `rescue in <top (required)>' /usr/local/lib/ruby/3.3.0/bundler/vendored_net _http.rb:3:in `<top (required)>' /usr/local/lib/ruby/3.3.0/bundler/vendor/net-h ttp-persistent/lib/net/http/persistent.rb:1:in `require_relative' /usr/local/lib/ruby/3.3.0/bundler/vendor/net-h ttp-persistent/lib/net/http/persistent.rb:1:in `<top (required)>' /usr/local/lib/ruby/3.3.0/bundler/vendored_per sistent.rb:11:in `require_relative' /usr/local/lib/ruby/3.3.0/bundler/vendored_per sistent.rb:11:in `<top (required)>' /usr/local/lib/ruby/3.3.0/bundler/fetcher.rb:3 require "bundler" require "bundler/cli" require "bundler/cli/install" Bundler::CLI::Install.new({path: "./gems"}).run 1 2 3 4 5 6
47. ruby.wasm tool #1 Reimplement missing core classes enough `require': cannot load such file -- socket (LoadError)
48. class Socket def initialize(...) raise NotImplementedError, "Socket is not supported" end def self.do_not_reverse_lookup=(value) value end end class SocketError < StandardError end
49. <script> socketRb = new File(new TextEncoder("utf-8").encode(`class Socket<...>`)) fs = [ //... new PreopenDirectory("/stubs", [ ["socket.rb", socketRb], ["io", new Directory([ ["wait.rb", new File([])] ])], // ... ]) ] </script>
50. $LOAD_PATH.unshift "/stubs" require "bundler" require "bundler/cli" require "bundler/cli/install" Bundler::CLI::Install.new({path: "./gems"}).run
51. Files main.rb Gemfile Run Code => Invalid argument @ rb_file_flock - /gems/ru by/3.3.0/bundler.lock /usr/local/lib/ruby/3.3.0/bundler/process_lock .rb:10:in `flock' /usr/local/lib/ruby/3.3.0/bundler/process_lock .rb:10:in `block in lock' /usr/local/lib/ruby/3.3.0/bundler/process_lock .rb:9:in `open' /usr/local/lib/ruby/3.3.0/bundler/process_lock .rb:9:in `lock' /usr/local/lib/ruby/3.3.0/bundler/installer.rb :71:in `run' /usr/local/lib/ruby/3.3.0/bundler/installer.rb :23:in `install' /usr/local/lib/ruby/3.3.0/bundler/cli/install. rb:63:in `run' (eval at eval_async:3):7:in `<main>' eval_async:3:in `eval' eval_async:3:in `<main>' /bundle/gems/js-2.6.2/lib/js.rb:109:in `eval' /bundle/gems/js-2.6.2/lib/js.rb:109:in `block in __eval_async_rb' /bundle/gems/js-2.6.2/lib/js.rb:120:in `block in __async' $LOAD_PATH.unshift "/stubs" require "bundler" require "bundler/cli" require "bundler/cli/install" Bundler::CLI::Install.new({path: "./gems"}).run 1 2 3 4 5 6 7 8
52. ruby.wasm tool #2 Yes, monkey patch it Invalid argument @ rb_file_flock - /gems/ruby/3.3.0/bundler.lock /usr/local/lib/ruby/3.3.0/bundler/process_lock.rb:10:in `flock'
53. # flock is not implemented File.prepend(Module.new do def flock(*);end end) # shim marks all dirs/files as readonly File.singleton_class.prepend(Module.new do def writable?(*) = true end)
54. Files main.rb Gemfile Run Code => uninitialized constant OpenSSL::SSL::VERIFY _PEER /usr/local/lib/ruby/3.3.0/bundler/vendor/net-h ttp-persistent/lib/net/http/persistent.rb:516: in `initialize' /usr/local/lib/ruby/3.3.0/bundler/fetcher.rb:2 56:in `new' /usr/local/lib/ruby/3.3.0/bundler/fetcher.rb:2 56:in `connection' /usr/local/lib/ruby/3.3.0/bundler/fetcher.rb:1 02:in `initialize' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:262:in `new' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:262:in `block in remote_fetchers' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:260:in `to_h' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:260:in `remote_fetchers' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:267:in `fetchers' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:396:in `block in remote_specs' /usr/local/lib/ruby/3.3.0/bundler/index.rb:9:i n `build' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:395:in `remote_specs' /usr/local/lib/ruby/3.3.0/bundler/source/rubyg ems.rb:136:in `specs' /usr/local/lib/ruby/3.3.0/bundler/resolver.rb: 41:in `block in setup_solver' /usr/local/lib/ruby/3.3.0/bundler/resolver.rb: 246:in `all_versions_for' /usr/local/lib/ruby/3.3.0/bundler/resolver.rb: 54:in `block in setup solver' $LOAD_PATH.unshift "/stubs" require "bundler_patches" require "bundler" require "bundler/cli" require "bundler/cli/install" Bundler::CLI::Install.new({path: "./gems"}).run 1 2 3 4 5 6 7 8
55. ruby.wasm tool #3 Use JS to overcome limitations uninitialized constant OpenSSL::SSL::VERIFY_PEER /usr/local/lib/ruby/3.3.0/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb:516:in `initial /usr/local/lib/ruby/3.3.0/bundler/fetcher.rb:256:in `new' /usr/local/lib/ruby/3.3.0/bundler/fetcher.rb:256:in `connection'
56. def request(uri, req) js_response = JS.eval(<<~JS).await return fetch('#{uri}', { method: '#{req.method}', headers: #{req.to_hash.transform_values { _1.join(';') }.to_json}, body: #{req.body ? req.body.to_json : 'undefined'} }) .then(response => { return response.text().then(text => ( {status: response.status, headers: response.headers, text} )) } }) JS serialize_js_response(js_response) end
57. require "js_connection" require "rubygems" require "rubygems/commands/install_command" Gem::Request.prepend(Module.new do def perform_request(request) JS::Connection.new.request(Gem::Uri.redact(@uri).to_s, request) end end) Bundler::Fetcher.prepend(Module.new do def connection JS::Connection.new end end)
58. Files main.rb Gemfile Run Code Fetching gem metadata from https://rubygems.or g/ Resolving dependencies... Fetching uri-idna 0.2.2 Installing uri-idna 0.2.2 Bundle complete! 1 Gemfile dependency, 2 gems now installed. Bundled gems are installed into `./gems` => xn--gdkl8fhk5egc.jp $LOAD_PATH.unshift "/stubs" require "bundler_patches" require "bundler_http_patches" Bundler::CLI::Install.new({path: "./gems"}).run require "uri-idna" URI::IDNA.whatwg_to_ascii("ハロー・ワールド.jp") # Should return: # => xn--gdkl8fhk5egc.jp 1 2 3 4 5 6 7 8 9 10 11 12 13
59. ruby.wasm tool #4 Be creative CORS issues: CORS extension or a lambda-proxy Slow startup times: Resolve gems in cloud, use localStorage and counting…
60. Playground implementation Run ruby.wasm in browser Share FS between JS & Ruby Install gems via Bundler Bonus: Spin up a server
61. # app.rb require "sinatra/base" class App < Sinatra::Base get "/goat" do "I don't know how I goat here 🐐 <a href='/'>Baaa-ack</a>" end get "/" do "Hello, take a look at <br><a href='/goat'>another page</a>" end end # config.ru require_relative 'app' run App
62. Enabling web server
63. Prepare data for Rack in JS const headers = { method: request.method }; request.headers.entries().forEach(([key, value]) => { headers[`HTTP_${key.toUpperCase().replaceAll("-", "_")}`] = value; }) const command = ` request = Rack::MockRequest.env_for( "${request.url}", JSON.parse(%q[${JSON.stringify(headers)}]) ) # ... `;
64. request = Rack::MockRequest.env_for(<...>) app = Rack::Builder.load_file('config.ru') response = Rack::Response[*app.call(request)] status, headers, body_chunks = *response.finish body = body_chunks.reduce(:+) { status:, headers:, body: }
65. Making requests in the browser ServiceWorker addEventListener("fetch", ...) WebRTC sinclairzx81/smoke Custom networking layer
66. ServiceWorker // sw.js self.addEventListener("fetch", (event) => { event.respondWith((async () => { if (!self.RubyVM) { await installRubyWasm(); } command = prepareCommand(event.request); const res = await self.RubyVM.evalAsync(command) const { code, headers, body } = res.toJS(); return new Response(body, { headers, status: code }); })()) });
67. iframe Hello, take a look at another page
68. Can we run Rails? Nokogiri, SQLite3, etc. Database adapter and more… Yes, but that requires even more hacks
70. Thank you!