AI in the Browser Smarter Angular Apps With WebNN & Prompt API Christian Liebel @christianliebel Consultant 

Hello, it’s me. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Christian Liebel W3C WebML WG & CG TAG Associate X: @christianliebel Bluesky: @christianliebel.com Angular, PWA & Generative AI Microsoft MVP & Google GDE (Angular, Web) 

Examples AI in the Browser Smarter Angular Apps With WebNN & Prompt API Generative AI Cloud Providers 

Drawbacks AI in the Browser Smarter Angular Apps With WebNN & Prompt API Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription 

Can we run GenAI models locally? AI in the Browser Smarter Angular Apps With WebNN & Prompt API 

Bring Your Own AI (BYOAI) – Libraries – WebLLM – Transfomers.js – Frameworks – ONNX Runtime – TensorFlow.js – APIs – WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Translator & Language Detector APIs – Prompt API AI in the Browser Smarter Angular Apps With WebNN & Prompt API Local AI Inference 

https://webllm.mlc.ai/ AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM DEMO 

On NPM AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM 

Storing model files locally AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM Internet Website HTML/JS Cache with model files Hugging Face Note: Due to the Same-Origin Policy, models cannot be shared across origins. 

Model Size Comparison Model:Parameters Size phi3:3b 2.2 GB mistral:7b 4.1 GB llama3:8b 4.7 GB gemma2:9b 5.4 GB gemma2:27b 16 GB llama3:70b 40 GB AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM 

https://huggingface.co/docs/transformers.js/index AI in the Browser Smarter Angular Apps With WebNN & Prompt API Transformers.js DEMO 

– Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Better performance for specific workloads AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebNN Source: https://webmachinelearning.github.io/webnn-intro/ DEMO 

AI in the Browser Smarter Angular Apps With WebNN & Prompt API Why should you care? DEMO 

about://flags Enables WebNN API à Enabled Enables experimental WebNN API features à Enabled AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebNN 

Drawbacks AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebNN Models can’t be shared across origins Inference is fast, but doesn’t reach full native speed 

https://github.com/explainers-by-googlers/cross-origin-storage AI in the Browser Smarter Angular Apps With WebNN & Prompt API Cross-Origin Storage 

– Initiative by Google Chrome – Exploratory APIs for local experiments and use case determination – Downloads AI models into Google Chrome – Models are shared across origins – Uses native APIs directly (full performance) AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI https://developer.chrome.com/docs/ai/built-in 

Incubated by the WebML CG AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs https://webmachinelearning.github.io/incubations/ DEMO 

AI in the Browser Smarter Angular Apps With WebNN & Prompt API Multimodal Models 

AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano 

about://on-device-internals https://www.google.com/chrome/canary/ about://flags Enables optimization guide on device à EnabledBypassPerfRequirement (API) for Gemini Nano à Enabled AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs 

TypeScript Definitions AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs 

Rule-based algorithms are limited in their capabilities. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Why should you care? 

Use AI to implement use cases that are difficult or impossible to implement using rule-based algorithms. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Why should you care? 

AI in the Browser Smarter Angular Apps With WebNN & Prompt API Chatbots DEMO 

AI in the Browser Smarter Angular Apps With WebNN & Prompt API Categorization DEMO 

(Cloud only!) AI in the Browser Smarter Angular Apps With WebNN & Prompt API Multimodal Realtime Models DEMO 

Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower response quality – Less capable – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental AI in the Browser Smarter Angular Apps With WebNN & Prompt API On-device AI Models 

Thank you for your kind attention! Christian Liebel @christianliebel [email protected]