AI in the Browser: Smarter Angular apps with Prompt API and WebNN

In this session, we will explore the integration of Generative AI functions into Angular applications using the Prompt API and the Web Neural Network (WebNN) API. These APIs enable the execution of Large Language Models (LLMs) on the user’s device. The primary benefits of local execution include offline availability and data security, provided that the user’s device has sufficient power to run the AI models. During the presentation, we will discuss different use cases and compare the advantages and disadvantages of each solution. Join us to learn how to make your Angular app smarter.

Transcript

1. AI in the Browser Smarter Angular Apps with Prompt API and WebNN Christian Liebel @christianliebel Consultant
2. AI in the Browser Smarter Angular Apps with Prompt API and WebNN Schedule 🕒 Room Title 10:15 Regency D AI in the Browser: Smarter Angular Apps with Prompt API and WebNN 11:30 Regency D Built-in AI & WebMCP: The AI Revolution Right in Your Browser
3. Hello, it’s me. AI in the Browser Christian Liebel W3C WebApps, WebML & TAG christian.liebel @thinktecture.com Angular, PWA & Generative AI Microsoft MVP & Google GDE Smarter Angular Apps with Prompt API and WebNN
4. Rule-based algorithms are limited in their capabilities. AI in the Browser Why should you care about AI? Smarter Angular Apps with Prompt API and WebNN
5. Use AI to implement use cases that are difficult or impossible to implement using rule-based algorithms. AI in the Browser Why should you care about AI? Smarter Angular Apps with Prompt API and WebNN
6. Examples AI in the Browser Generative AI Cloud Providers Smarter Angular Apps with Prompt API and WebNN
7. Drawbacks AI in the Browser Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription Smarter Angular Apps with Prompt API and WebNN
8. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Frameworks – Transformers.js – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API – Translator & Language Detector APIs – Prompt API – Embedding API AI in the Browser Web AI Landscape Smarter Angular Apps with Prompt API and WebNN
9. AI in the Browser Smarter Angular Apps with Prompt API and WebNN WebGPU – Grants low-level access to the Graphics Processing Unit (GPU) – Near native performance for machine learning applications – Supported by Chromium-based browsers on Windows and macOS from version 113, Safari 26, and Firefox 141 on Windows/Firefox 145 on macOS 26 Tahoe on Apple Silicon
10. https://huggingface.co/docs/transformers.js/index AI in the Browser Transformers.js DEMO Smarter Angular Apps with Prompt API and WebNN
11. & Angular AI in the Browser Smarter Angular Apps with Prompt API and WebNN Transformers.js DEMO
12. – Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Positive standard positions by Chromium and Firefox AI in the Browser WebNN Source: https://webmachinelearning.github.io/webnn-intro/ Smarter Angular Apps with Prompt API and WebNN
13. WebNN AI in the Browser Transformers.js Smarter Angular Apps with Prompt API and WebNN https://webnn.io/en/learn/tutorials/transformers-js/transformers-js
14. AI in the Browser WebNN https://huggingface.co/webnn/spaces Smarter Angular Apps with Prompt API and WebNN DEMO
15. AI in the Browser Smarter Angular Apps with Prompt API and WebNN WebNN: near-native inference performance Source: Intel. Browser: Chrome Canary 118.0.5943.0, DUT: Dell/Linux/i7-1260P, single p-core, Workloads: MediaPipe solution models (FP32, batch=1)
16. Drawbacks AI in the Browser Bring Your Own AI Models can’t be shared across origins Inference is fast, but doesn’t reach full native speed Smarter Angular Apps with Prompt API and WebNN
17. https://github.com/wicg/cross-origin-storage AI in the Browser Cross-Origin Storage DEMO Smarter Angular Apps with Prompt API and WebNN
18. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Frameworks – Transformers.js – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API – Translator & Language Detector APIs – Prompt API – Embedding API AI in the Browser Web AI Landscape Smarter Angular Apps with Prompt API and WebNN
19. AI in the Browser Built-in AI Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano Smarter Angular Apps with Prompt API and WebNN
20. – Initiative by Google Chrome – Exploratory APIs for local experiments and use case determination – Downloads AI models into Google Chrome – Models are shared across origins – Uses native APIs directly (full performance) AI in the Browser Built-in AI https://developer.chrome.com/docs/ai/built-in Smarter Angular Apps with Prompt API and WebNN
21. Prompt API Standard Positions AI in the Browser Smarter Angular Apps with Prompt API and WebNN Built-in AI https://github.com/WebKit/standards-positions/issues/495 https://github.com/mozilla/standards-positions/issues/1213
22. Prompt API Polyfill AI in the Browser Smarter Angular Apps with Prompt API and WebNN Built-in AI Supports two backends: – Cloud backend: needs an API key (no longer local/offline-capable) – Local backend: uses Transformers.js (via WebGPU or Wasm) npm install prompt-api-polyfill https://developer.chrome.com/docs/ai/prompt-api-polyfill
23. Prompt API const languageModel = await LanguageModel.create(); console.log(await languageModel.prompt("Capital of the US?")); AI in the Browser Smarter Angular Apps with Prompt API and WebNN Built-in AI
24. AI in the Browser Multimodal Models Smarter Angular Apps with Prompt API and WebNN
25. AI in the Browser Multimodal Models DEMO Smarter Angular Apps with Prompt API and WebNN
26. AI in the Browser Structured Output DEMO https://bandarra.me/apps/flyby/ Smarter Angular Apps with Prompt API and WebNN
27. TypeScript Definitions AI in the Browser Built-in AI APIs Smarter Angular Apps with Prompt API and WebNN
28. Structured Output AI in the Browser Built-in AI APIs DEMO Smarter Angular Apps with Prompt API and WebNN
29. AI in the Browser Smarter Angular Apps with Prompt API and WebNN
30. Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower response quality – Less capable – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental AI in the Browser Web AI Smarter Angular Apps with Prompt API and WebNN
31. “ AI in the Browser Smarter Angular Apps with Prompt API and WebNN Web AI A new NVIDIA chip combines an ultra- efficient CPU with a powerful RTX GPU to deliver up to one petaflop of AI compute. A new kind of power for world makers, built to create what’s next and run the latest games https://www.microsoft.com/en-us/surface/devices/surface-laptop-ultra
32. Q&A AI in the Browser Smarter Angular Apps with Prompt API and WebNN
33. Thank you for your kind attention! Christian Liebel @christianliebel christian.liebel@thinktecture.com