Built-in AI APIs & WebNN: AI Right in Your Browser, Local and Offline-Capable

More and more developers intend to integrate Generative AI features into their applications. Until now, this path has practically always led to the cloud—but it doesn't have to be like that!

Currently, various promising approaches exist for running AI models directly on the user's computer. With Chromium's new built-in AI APIs, we can now bring Large Language Models to your browser: locally and offline-capable. The W3C's Web Neural Network API (WebNN) will grant AI models access to the device's Neural Processing Unit (NPU). The advantages of these approaches are obvious: Locally executed AI models are available offline, the user data does not leave the device, and all this is even free of charge thanks to open-source models.

In this talk, Christian Liebel, Thinktecture's representative at W3C, will present the approaches to make your single-page app smarter. He will discuss use cases and show the advantages and disadvantages of each solution.

Transcript

1. Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable Christian Liebel @christianliebel Consultant
2. Hello, it’s me. Built-in AI APIs & WebNN Christian Liebel W3C WebApps, WebML & TAG christian.liebel @thinktecture.com Angular, PWA & Generative AI Microsoft MVP & Google GDE AI Right in Your Browser, Local and Offline-Capable
3. Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable
4. Rule-based algorithms are limited in their capabilities. Built-in AI APIs & WebNN Why should you care about AI? AI Right in Your Browser, Local and Offline-Capable
5. Use AI to implement use cases that are difficult or impossible to implement using rule-based algorithms. Built-in AI APIs & WebNN Why should you care about AI? AI Right in Your Browser, Local and Offline-Capable
6. Examples Built-in AI APIs & WebNN Generative AI Cloud Providers AI Right in Your Browser, Local and Offline-Capable
7. Drawbacks Built-in AI APIs & WebNN Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription AI Right in Your Browser, Local and Offline-Capable
8. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Frameworks – Transformers.js – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API – Translator & Language Detector APIs – Prompt API – WebMCP Built-in AI APIs & WebNN Web AI Landscape AI Right in Your Browser, Local and Offline-Capable
9. https://huggingface.co/docs/transformers.js/index Built-in AI APIs & WebNN Transformers.js DEMO AI Right in Your Browser, Local and Offline-Capable
10. Built-in AI APIs & WebNN WebGPU – Grants low-level access to the Graphics Processing Unit (GPU) – Near native performance for machine learning applications – Supported by Chromium-based browsers on Windows and macOS from version 113, Safari 26 NEW!, and Firefox 141 on Windows AI Right in Your Browser, Local and Offline-Capable
11. – Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Positive standard positions by Chromium and Firefox Built-in AI APIs & WebNN WebNN Source: https://webmachinelearning.github.io/webnn-intro/ AI Right in Your Browser, Local and Offline-Capable
12. Built-in AI APIs & WebNN WebNN https://huggingface.co/webnn/spaces AI Right in Your Browser, Local and Offline-Capable DEMO
13. Drawbacks Built-in AI APIs & WebNN Bring Your Own AI Models can’t be shared across origins Inference is fast, but doesn’t reach full native speed AI Right in Your Browser, Local and Offline-Capable
14. https://github.com/explainers-by-googlers/cross-origin-storage Built-in AI APIs & WebNN Cross-Origin Storage DEMO AI Right in Your Browser, Local and Offline-Capable
15. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Frameworks – Transformers.js – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API – Translator & Language Detector APIs – Prompt API – WebMCP Built-in AI APIs & WebNN Web AI Landscape AI Right in Your Browser, Local and Offline-Capable
16. Built-in AI APIs & WebNN Built-in AI Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano AI Right in Your Browser, Local and Offline-Capable
17. – Initiative by Google Chrome – Exploratory APIs for local experiments and use case determination – Downloads AI models into Google Chrome – Models are shared across origins – Uses native APIs directly (full performance) Built-in AI APIs & WebNN Built-in AI https://developer.chrome.com/docs/ai/built-in AI Right in Your Browser, Local and Offline-Capable
18. Incubated by the WebML CG Built-in AI APIs & WebNN Built-in AI APIs https://webmachinelearning.github.io/incubations/ DEMO AI Right in Your Browser, Local and Offline-Capable
19. Prompt API Standard Positions Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable Built-in AI https://github.com/WebKit/standards-positions/issues/495 https://github.com/mozilla/standards-positions/issues/1213
20. Prompt API const languageModel = await LanguageModel.create(); console.log(await languageModel.prompt("Capital of the UK?")); Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable Built-in AI
21. Built-in AI APIs & WebNN Multimodal Models AI Right in Your Browser, Local and Offline-Capable
22. Built-in AI APIs & WebNN Multimodal Models DEMO AI Right in Your Browser, Local and Offline-Capable
23. Built-in AI APIs & WebNN Structured Output DEMO https://bandarra.me/apps/flyby/ AI Right in Your Browser, Local and Offline-Capable
24. about://flags Enables optimization guide on device à EnabledBypassPerfRequirement (API) for Gemini Nano à Enabled Built-in AI APIs & WebNN Built-in AI APIs AI Right in Your Browser, Local and Offline-Capable
25. TypeScript Definitions Built-in AI APIs & WebNN Built-in AI APIs AI Right in Your Browser, Local and Offline-Capable
26. Built-in AI APIs & WebNN Agentic Browsers AI Right in Your Browser, Local and Offline-Capable
27. Built-in AI APIs & WebNN WebMCP – Allows websites to expose tools to the browser or external agents – Joint effort by Microsoft and Google https://github.com/webmachinelearning/webmcp AI Right in Your Browser, Local and Offline-Capable
28. Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable WebMCP https://webmachinelearning.github.io/webmcp/
29. API Imperative navigator.modelContext .provideContext({ tools: [{ "name": "start_game", "description": "Start a new game.", "inputSchema": {}, "execute": () => {}, } ]}); Declarative <form id="reservationForm" toolname="book_table_le_petit_bistr o" tooldescription=...> <input name="name" toolparamdescription="Customer's full name (min 2 chars)" /> </form> Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable WebMCP
30. Agent-only! Built-in AI APIs & WebNN AI Right in Your Browser, Local and Offline-Capable WebMCP Maze https://webmcp-maze.bandarra.me/ DEMO
31. Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower response quality – Less capable – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental Built-in AI APIs & WebNN Web AI AI Right in Your Browser, Local and Offline-Capable
32. Thank you for your kind attention! Christian Liebel @christianliebel christian.liebel@thinktecture.com