AI in the Browser: Smarter Angular apps with Prompt API and WebNN

In this session, we will explore the integration of Generative AI functions into Angular applications using the Prompt API and the Web Neural Network (WebNN) API. These APIs enable the execution of Large Language Models (LLM) on the user’s device. The primary benefits of local execution include offline availability and data security, provided that the user’s device has sufficient power to run the AI models. During the presentation, we will discuss different use cases and compare the advantages and disadvantages of each solution. Join us to learn how to make your Angular app smarter.

Transcript

1. AI in the Browser Smarter Angular apps with Prompt API and WebNN Christian Liebel @christianliebel Consultant
2. Hello, it’s me. AI in the Browser Smarter Angular apps with Prompt API and WebNN Christian Liebel X: @christianliebel Bluesky: @christianliebel.com Email: christian.liebel @thinktecture.com Angular, PWA & Generative AI Slides: thinktecture.com /christian-liebel
3. Rule-based algorithms are limited in their capabilities. AI in the Browser Why should you care about AI? Smarter Angular apps with Prompt API and WebNN
4. Use AI to implement use cases that are difficult or impossible to implement using rule-based algorithms. AI in the Browser Why should you care about AI? Smarter Angular apps with Prompt API and WebNN
5. Examples AI in the Browser Generative AI Cloud Providers Smarter Angular apps with Prompt API and WebNN
6. Drawbacks AI in the Browser Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription Smarter Angular apps with Prompt API and WebNN
7. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Transfomers.js – Frameworks – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage NEW! Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API NEW! – Translator & Language Detector APIs – Prompt API NEW! Multimodal – WebMCP NEW! AI in the Browser Web AI Landscape Smarter Angular apps with Prompt API and WebNN
8. https://webllm.mlc.ai/ AI in the Browser WebLLM DEMO Smarter Angular apps with Prompt API and WebNN
9. On NPM AI in the Browser WebLLM Smarter Angular apps with Prompt API and WebNN
10. Storing model files locally AI in the Browser WebLLM Internet Website HTML/JS Cache with model files Hugging Face Note: Due to the Same-Origin Policy, models cannot be shared across origins. Smarter Angular apps with Prompt API and WebNN
11. Model Size Comparison Model:Parameters Size phi3:3.8b 2.2 GB mistral:7b 4.1 GB deepseek-r1:8b 5.2 GB gemma3n:e4b 7.5 GB gemma3:12b 8.1 GB llama4:16x17b 67 GB AI in the Browser WebLLM Smarter Angular apps with Prompt API and WebNN
12. Drawbacks AI in the Browser WebNN Models can’t be shared across origins Inference is fast, but doesn’t reach full native speed Smarter Angular apps with Prompt API and WebNN
13. https://github.com/explainers-by-googlers/cross-origin-storage AI in the Browser Cross-Origin Storage Smarter Angular apps with Prompt API and WebNN DEMO
14. https://huggingface.co/docs/transformers.js/index AI in the Browser Transformers.js DEMO Smarter Angular apps with Prompt API and WebNN
15. Cloud only… AI in the Browser Realtime Models DEMO Smarter Angular apps with Prompt API and WebNN
16. …or is it? NEW! AI in the Browser Realtime Models DEMO Smarter Angular apps with Prompt API and WebNN
17. – Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Better performance for specific workloads AI in the Browser WebNN Source: https://webmachinelearning.github.io/webnn-intro/ Smarter Angular apps with Prompt API and WebNN DEMO
18. about://flags Enables WebNN API à Enabled Enables experimental WebNN API features à Enabled AI in the Browser WebNN Smarter Angular apps with Prompt API and WebNN
19. – Initiative by Google Chrome – Exploratory APIs for local experiments and use case determination – Downloads AI models into Google Chrome – Models are shared across origins – Uses native APIs directly (full performance) AI in the Browser Built-in AI https://developer.chrome.com/docs/ai/built-in Smarter Angular apps with Prompt API and WebNN
20. Incubated by the WebML CG AI in the Browser Built-in AI APIs https://webmachinelearning.github.io/incubations/ DEMO Smarter Angular apps with Prompt API and WebNN
21. AI in the Browser Multimodal Models Smarter Angular apps with Prompt API and WebNN
22. AI in the Browser Multimodal Models Smarter Angular apps with Prompt API and WebNN DEMO
23. AI in the Browser Structured Output Smarter Angular apps with Prompt API and WebNN DEMO https://bandarra.me/apps/flyby/
24. AI in the Browser Built-in AI APIs Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano Smarter Angular apps with Prompt API and WebNN
25. about://on-device-internals https://www.google.com/chrome/canary/ about://flags Enables optimization guide on device à EnabledBypassPerfRequirement (API) for Gemini Nano à Enabled AI in the Browser Built-in AI APIs Smarter Angular apps with Prompt API and WebNN
26. TypeScript Definitions AI in the Browser Built-in AI APIs Smarter Angular apps with Prompt API and WebNN
27. AI in the Browser Smarter Angular apps with Prompt API and WebNN WebMCP – Allows websites to expose tools to browser or external agents – Joint effort by Microsoft and Google https://github.com/webmachinelearning/webmcp
28. AI in the Browser Smarter Angular apps with Prompt API and WebNN WebMCP https://www.youtube.com/watch?v=gbu9kyY2B60
29. Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower response quality – Less capable – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental AI in the Browser Web AI Smarter Angular apps with Prompt API and WebNN
30. AI in the Browser Smarter Angular apps with Prompt API and WebNN WebLLM & Prompt API Code Samples
31. Thank you for your kind attention! Christian Liebel @christianliebel christian.liebel@thinktecture.com