Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Language Summit @ PyCon US 2022 WebAssembly: Python in the browser & beyond Christian Heimes Principal Software Engineer [email protected] [email protected] 1 @ChristianHeimes 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 caniuse.com/wasm ? 2 Modern browsers support two programming languages Wasmer wasmtime Node 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 3 Compiling to WASM human code to native machine code C C++ Go Rust frontend frontend frontend frontend intermediate representation backend backend backend backend x86_64 i686 aarch64 armv7 backend wasm32 Emscripten SDK 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 4 def line(a, x, b): return a * x + b double line(double a, double x, double b) { return a * x + b; } What is WebAssembly? C/C++ 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 5 # WASM text (WAT) (module (export "line" (func $line)) (func $line (param $a f64) (param $x f64) (param $b f64) (result f64) get_local $a get_local $x f64.mul get_local $b f64.add ) ) Stack machine VM >>> import dis >>> def line(a, x, b): ... return a * x + b ... >>> dis.dis(line) 2 0 LOAD_FAST 0 (a) 2 LOAD_FAST 1 (x) 4 BINARY_MULTIPLY 6 LOAD_FAST 2 (b) 8 BINARY_ADD 10 RETURN_VALUE 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Opening a file on disk 6 open("file.txt") _io.TextIOWrapper _io.BufferedReader _io.FileIO // _io/fileio.c #include int fd = open( path, flags, ... ); glibc libc.so.6 open(path, ...); openat( dirfd, path, ... ); syscall( SYS_openat, dirfd, path, ... ); open Kernel sys_call_table ... VFS ext4fs block device SATA ... syscall 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 7 Problems ▸ Linux, macOS, Windows, … have different APIs ▸ Linux syscall numbers are arch-specific x86_64: 257, i686: 295, ARM: 332, aarch64: 56, ... ▸ Browsers are sandboxed How to open a file safely on all platforms? (import "env" "__syscall_openat" (func $fimport$1 (param i32 i32 i32 i32) (result i32))) (import "wasi_snapshot_preview1" "fd_read" (func $fimport$2 (param i32 i32 i32 i32) (result i32))) (import "wasi_snapshot_preview1" "fd_close" (func $fimport$0 (param i32) (result i32))) 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Opening a file on Emscripten / WASI 8 open("file.txt") _io.TextIOWrapper _io.BufferedReader _io.FileIO // _io/fileio.c #include int fd = open( path, flags, ... ); musl static libc.a open(path, ...); openat( dirfd, path, ... ); syscall( SYS_openat, dirfd, path, ... ); open Browser NodeJS WASM import / export WASI runtime MEMFS NODE raw FS 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 9 ▸ x86_64-unknown-linux-gnu ▸ x86_64-unknown-linux-musl ▸ armv7l-unknown-linux-gnueabihf ▸ aarch64-unknown-linux-android ▸ aarch64-apple-darwin ▸ i686-pc-windows-msvc ▸ sparcv9-sun-solaris ▸ wasm32-emscripten ▸ wasm32-wasi Target Triple(t) machine vendor os (abi) "CPU" "OS" 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 10 What is WebAssembly ▸ "machine-independent machine code" ▸ binary (.wasm), text (.wat) with metadata ▸ designed to be portable and fast ▸ 4 value types (integer i32, i64; float f32, f64) ▸ ~200 instructions ▸ optional extension (128 bits vector ops, threading) ▸ wasm modules have ･ imports ･ exports ･ memory 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Cross-compile with Emscripten 11 EMSDK ▸ build system: x86_64-pc-linux-gnu ▸ target host: wasm32-unknown-emscripten ▸ cross compiler: emcc (Python script, Clang, LLVM, Binaryen, NodeJS) ▸ sysroot (headers, libraries) ▸ configure wrapper: emconfigure ▸ make wrapper: emmake 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Cross compiling improvements 12 ▸ new wasm32 targets ▸ new --with-build-python ▸ re-implement _freeze_module in pure Python ▸ detect module deps in configure ▸ Modules/Setup.stdlib ▸ improved config.cache ▸ VPATH / out-of-tree build fixes ▸ new --with-emscripten-target # Makefile MODULE_ZLIB=yes MODULE_ZLIB_CFLAGS= MODULE_ZLIB_LDFLAGS=-lz MODULE__SCPROXY=n/a # Setup.stdlib.in @MODULE__SCPROXY_TRUE@_scproxy _scproxy.c @MODULE_ZLIB_TRUE@zlib zlibmodule.c # Setup.stdlib #_scproxy _scproxy.c zlib zlibmodule.c 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 13 Build Python for browsers $ git clone https://github.com/python/cpython.git $ cd cpython $ mkdir -p builddir/build $ mkdir -p builddir/emscripten-browser $ cd builddir/emscripten-browser $ CONFIG_SITE=config.site-wasm32-emscripten \ emconfigure ../../configure \ --host=wasm32-unknown-emscripten \ --build=x86_64-pc-linux-gnu \ --with-emscripten-target=browser \ --with-build-python=$(pwd)/../build/python $ emmake make $ cd builddir/build $ ../../configure $ make $ ls python python $ cd ../.. $ ls python* python.data python.html python.js python.wasm python.worker.js # config site ac_cv_func_eventfd=no ac_cv_func_memfd_create=no 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 14 How it's going 2022-04-05 / 2022-03-21 $ node --experimental-wasm-threads \ --experimental-wasm-bulk-memory \ python.js -m test -w == CPython 3.11.0a7+ (heads/bpo-40280-smaller-build-dirty:eb79f02293d, Apr 8 2022, 10:58:59) [Clang 15.0.0 (https://github.com/llvm/llvm-project 80ec0ebfdc5692a58e0832125f2c == Emscripten-1.0-wasm32-32bit little-endian == cwd: /python-wasm/cpython/builddir/emscripten-node/build/test_pyt hon_42æ == CPU count: 8 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 0.10 Run tests sequentially 0:00:00 load avg: 0.10 [ 1/435] test_grammar 0:00:00 load avg: 0.10 [ 2/435] test_opcodes ... == Tests result: SUCCESS == 342 tests OK. 92 tests skipped. Total duration: 12 min 22 sec Tests result: SUCCESS 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 15 Platform >>> sys.platform 'emscripten' >>> os.name 'posix' >>> sys._emscripten_info sys._emscripten_info( emscripten_version=(3, 1, 8), runtime='Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0', pthreads=False, shared_memory=False) >>> os.uname() posix.uname_result(sysname='Emscripten', nodename='emscripten', release='1.0', version='#1', machine='wasm32') EM_JS(char *, _Py_emscripten_runtime, (void), { var info; if (typeof navigator == 'object') { info = navigator.userAgent; } else if (typeof process == 'object') { info = "Node.js ".concat(process.version) } else { info = "UNKNOWN" } var len = lengthBytesUTF8(info) + 1; var res = _malloc(len); stringToUTF8(info, res, len); return res; }); 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 > 60 CPython pull requests 16 https://bugs.python.org/issue40280 $ ./run-python-node.sh -c 'import _zoneinfo' builddir/emscripten-node/python.js:147 throw ex; ^ Error [RuntimeError]: function signature mismatch at module_dealloc (wasm-function[1908]) at _Py_Dealloc (wasm-function[1986]) at insertdict (wasm-function[1652]) at _PyDict_SetItem_Take2 (wasm-function[1641]) at dict_ass_sub (wasm-function[1733]) at PyObject_SetItem (wasm-function[495]) at finalize_modules (wasm-function[3677]) at Py_FinalizeEx (wasm-function[3674]) at Py_RunMain (wasm-function[4057]) at pymain_main (wasm-function[4060]) Emitted 'error' event on process instance at: at emitUnhandledRejectionOrErr at MessagePort.[nodejs.internal.kHybridDispatch] at MessagePort.exports.emitMessage --- a/Modules/_zoneinfo.c +++ b/Modules/_zoneinfo.c @@ -2612,7 +2612,7 @@ static PyTypeObject PyZoneInfo_ZoneInfoType = { // Specify the _zoneinfo module static PyMethodDef module_methods[] = {{NULL, NULL}}; static void -module_free(void) +module_free(void *m) { 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Emscripten bug fixes 17 https://github.com/ethanhs/python-wasm/issues/43 ▸ Segfault in freeaddrinfo, memory leak in getaddrinfo ▸ clock_nanosleep() sleeps forever ▸ strftime() mishandles quoted percent, e.g. %%z ▸ week number of strftime("%U") and %W ▸ Fix edge case in strftime %V (ISO 8601 week number) ▸ opendir() leak file descriptor on error ▸ msync() on anon mapping fails with JS TypeError ▸ utimensat() sets wrong atime and mtime ▸ Blocking socket accept() fails with JS TypeError 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 JupyterLite 18 SDK CPython REPL ⁉ Browser Server 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 19 Browser ❌ subprocess (fork, exec) ❌ threads ❌ file system (only MEMFS) 🚧 shared extension modules ❌ PyPI packages ❓ sockets ❌ urllib, asyncio ❌ signals Node ❌ subprocess (fork, exec) ✅ threads ✅ file system (Node raw FS) 🚧 shared extension modules ❌ PyPI packages ❓ sockets ❌ urllib, asyncio 🚧 signals Supported features CPython 3.11 alpha 7 Pyodide ❌ subprocess (fork, exec) 🚧 threads ✅ file system (IDB, Node, …) ✅ shared extension modules ✅ PyPI packages ✅ WebAPI fetch / WebSocket ✅ signals 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 What has WASM ever done for us? 20 ▸ Run Python anywhere (including Android, iOS) ･ great for education (no installation required) ･ easy sharing of programs (platform-independent code) ･ secure (browser are sandboxes) ▸ App development ▸ WASI (WASM on the server) for edge computing ･ small runtime with JIT / AOT optimization ･ capability-based sandbox ▸ platform- and OS-agnostic binary wheels? 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 21 Executable docs and examples Trey Hunner: www.pythonmorsels.com/paste/ 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Paul m. p. P. https://pmp-p.github.io/pygame-wasm/ 22 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 JupyterLite Jupyter ❤ WebAssembly ❤ Python jupyterlite.github.io/demo/lab/ 23 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 24 ▸ numpy-1.22.3-cp310-cp310-win_amd64.whl ▸ numpy-1.22.3-cp310-cp310-win32.whl ▸ numpy-1.22.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ▸ numpy-1.22.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl ▸ numpy-1.22.3-cp310-cp310-macosx_11_0_arm64.whl ▸ numpy-1.22.3-cp310-cp310-macosx_10_14_x86_64.whl ▸ numpy-1.22.3-cp39-cp39-win_amd64.whl ▸ numpy-1.22.3-cp39-cp39-win32.whl ▸ numpy-1.22.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ▸ numpy-1.22.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl ▸ numpy-1.22.3-cp39-cp39-macosx_11_0_arm64.whl ▸ numpy-1.22.3-cp39-cp39-macosx_10_14_x86_64.whl ▸ numpy-1.22.3-cp38-cp38-win_amd64.whl ▸ numpy-1.22.3-cp38-cp38-win32.whl ▸ numpy-1.22.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ▸ numpy-1.22.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl ▸ numpy-1.22.3-cp38-cp38-macosx_11_0_arm64.whl ▸ numpy-1.22.3-cp38-cp38-macosx_10_14_x86_64.whl numpy-1.22.3-cp311-abi3-wasm32.whl ▸ Python >= 3.11, <4 ▸ Any OS ▸ Any CPU architecture Python binary extension hell 3 Python versions, 3 CPU archs, 3 OSes: 18 wheels 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 25 wasmtime for Python github.com/bytecodealliance/wasmtime-py from functools import partial from wasmtime import Store, Module, Instance store = Store() module = Module(store.engine, """ (module (export "line" (func $line)) (func $line (param $a f64) (param $x f64) (param $b f64) (result f64) get_local $a get_local $x f64.mul get_local $b f64.add ) )""") instance = Instance(store, module, []) line = instance.exports(store)["line"] line = partial(line, store) print(line(3., 5., 2.)) # 17.0 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Discussion items 26 ▸ Tier 2 platform support (buildbot) ▸ Should we publicly pronounce official WASM support? ▸ Ship / provide official WASM binaries (CDN)? ▸ Use WASM to have runnable examples in documentation? ▸ Add browser API and Emscripten API modules? https://github.com/pmp-p/python-wasm-plus/tree/main/support/__EMSCRIPTEN__.embed ▸ Should we add a very high-level HTTPS API? sockets / networking in Emscripten is not compatible with urllib ▸ asyncio / async code? ▸ Reduce stdlib and binary size? ▸ Wizer support (WASM pre-initializer / memory freezer) 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 Python on WebAssembly … a lot of opportunities! … a lot of work left to do! 27 ❤ 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 28 ▸ https://bugs.python.org/issue40280 ▸ https://github.com/ethanhs/python-wasm ▸ https://pyodide.org/ ▸ https://pmp-p.github.io/pygame-wasm ▸ https://jupyterlite.github.io/demo/lab/ ▸ https://assets.metacade.com/emulators/wi n311vr.html ▸ https://pypi.org/project/wasmtime/ ▸ https://mbebenita.github.io/WasmExplorer/ ▸ https://wasdk.github.io/WasmFiddle/ ▸ David Beazley Keynote PyCon India 2019 https://youtu.be/VUT386_GKI8 ▸ Lin Clark: A Cartoon Intro to WebAssembly https://youtu.be/HktWin_LPf4 Links & Resources https://repl.ethanhs.me https://cheimes.fedorapeople.org/python-wasm https://speakerdeck.com/tiran/ 

Language Summit 2022: WASM Python, @ChristianHeimes, CC BY-SA 4.0 linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 29 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you