COMPUTER VISION AT THE EDGE WITH WEBASSEMBLY Miguel Angel Cabrera @miguelaeh_ Angel M De Miguel @_angelmm Pipeless 

COMPUTER VISION (CV) Pipeless 

A SIMPLE CV APP Model Inference Runtime GPU Output Pipeline Input Pipeline Post-processing Pre-processing CPU video streams video streams Pipeless 60 FPS / 16ms per frame Raw frame Video 

CHALLENGES OF COMPUTER VISION ● Parallelize frame processing? ● Latency? Pipeless Performance and stability ● How do you start with computer vision? ● What languages and frameworks can you use? Development experience ● How to distribute a computer vision platform? ● How to share processing logic? Distribution 

PIPELESS.AI Pipeless is an open-source framework that takes care of everything you need to develop and deploy computer vision applications in just minutes https://github.com/pipeless-ai/pipeless Pipeless 

ARCHITECTURE webcam, RTSP, WebRTC, … f f f f actionable data streams (JSON payloads) Pipeless pre-process post-process process (inference) frame data OpenVINO CoreML ONNX … Raw RGB structured JSON 

Pipeless pre-process.py process.json post-process.py PIPELESS HELLO WORLD 

Pipeless 

Pipeless Pipeless Original: ● FPS: 24 ● Duration: 9s Processing: ● FPS: 31 ● Duration: 7s 

WASM? Pipeless 

PIPELESS 6 MONTHS AGO… Pipeless ● A Python project ● Filters (user-defined) were written in Python ● Performance issues (processing frames in parallel) ● Big size artifacts Pipeless 

● Provide tools to simplify filters' development. ● Develop filters in multiple languages. CHALLENGES OF COMPUTER VISION Pipeless Performance and stability Developer Experience ● Ensure filters perform properly and you can parallelize them. ● Filters run in any environment / device. ● Simplify filter distribution and sharing them. Distribution 

Pipeless 

PIPELESS 🖤 WASM Pipeless pre-process post-process process (inference) frame data OpenVINO CoreML ONNX … Raw RGB structured JSON ● Define filters as Wasm components, so you can use different languages. ● Instantiate multiple filters in parallel and isolate them. ● Filters run in any environment where Pipeless run. ● Define a proper interface (with the Component Model) and compose filters, making small and shareable units. 

PIPELESS EVOLUTION Python Rust + Pyo3 Rust + Pyo3 + Wasm (exp) Pipeless 

IMPLEMENTATION DETAILS Pipeless 

STATEFUL VS STATELESS A component per thread. N-cores run in parallel. A single component, with internal state, shared among all threads and with sequential sorted processing. Pipeless lock run release Stateless run run run Stateful 

PYTHON ON WASM Pipeless Python user code componetize-py ● (Computer Vision) users love Python. ● Wasm unlocks parallel processing (avoid GIL limitations) at the cost of memory. ● Running a Python script in Wasm is significantly slower. 

AVOID COPYING FRAMES Latency is critical. Copying raw frames to WASM memory is too slow… ptr host-ref HashMap frame resize, normalize, draw, permute, etc. Pipeless common operations API invoke with host-ref WIT 

DEMO Pipeless 

Pipeless Original: ● FPS: 24 ● Duration: 9s Processing: ● FPS: ~8 ● Duration: 29s 

Benefits ● Securely run untrusted code ● Easy to share filters ● Multi-language support ● Get rid of Python GIL limitations Challenges ● Python components are slow ● Missing support for critical libraries ● Complex workarounds for no-copy RECAP A bright future awaits us with Wasm! Pipeless 

THANKS! Miguel Angel Cabrera @miguelaeh_ Angel M De Miguel @_angelmm Pipeless ⭐ Star Pipeless on GitHub! https://github.com/pipeless-ai/pipeless