Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Christian Liebel @christianliebel Consultant 

Hello, it’s me. Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Christian Liebel X: @christianliebel Bluesky: @christianliebel.com Email: christian.liebel @thinktecture.com Angular, PWA & Generative AI Slides: thinktecture.com /christian-liebel 

13:30–15:00 Block 1 15:00–15:30 Coffee Break 15:30–17:00 Block 2 Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Timetable 

What to expect Focus on web app development Focus on Generative AI Up-to-date insights: the ML/AI field is evolving fast Live demos on real hardware 12 hands-on labs What not to expect Deep dive into AI specifics, RAG, model finetuning or training Stable libraries or specifications Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Expectations Huge downloads! High requirements! Things may break! 

Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) DEMO 

(Workshop Edition) Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Demo Use Case DEMO 

Setup complete? (Node.js, Google Chrome, Editor, Git, macOS/Windows, 20 GB free disk space, 6 GB VRAM) Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Setup 

webgpureport.org Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebGPU 

git clone https://github.com/thinktecture/angular- days-2025-fall-genai.git cd angular-days-2025-fall-genai npm i Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Setup LAB #0 

Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Generative AI everywhere Source: https://www.apple.com/chde/apple-intelligence/ 

Run locally on the user’s system Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Single-Page Applications Server- Logik Web API Push Service Web API DBs HTML, JS, CSS, Assets Webserver Webbrowser SPA Client- Logik View HTML/CSS View HTML/CSS View HTML/CSS HTTPS WebSockets HTTPS HTTPS 

Make SPAs offline-capable Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Progressive Web Apps Service Worker Internet Website HTML/JS Cache fetch 

Overview Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Generative AI Text OpenAI GPT Mistral … Audio/Music Musico Soundraw … Images DALL·E Firefly … Video Sora Runway … Speech Whisper tortoise-tts … 

Overview Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Generative AI Text OpenAI GPT Mistral … Audio/Music Musico Soundraw … Images DALL·E Firefly … Video Sora Runway … Speech Whisper tortoise-tts … 

Examples Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Generative AI Cloud Providers 

Drawbacks Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription 

Can we run GenAI models locally? Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) 

Size Comparison Model:Parameters Size phi3:3.8b 2.2 GB mistral:7b 4.1 GB deepseek-r1:8b 5.2 GB gemma3n:e4b 7.5 GB gemma3:12b 8.1 GB llama4:16x17b 67 GB Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Large Language Models 

https://webllm.mlc.ai/ Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebLLM DEMO 

On NPM Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebLLM 

npm i @mlc-ai/web-llm npm start -- --open Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) LAB #1 

(1/4) In src/app/todo/todo.ts, add the following lines at the top of the class: protected readonly progress = signal(0); protected readonly ready = signal(false); protected engine?: MLCEngine; Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Downloading a model LAB #2 

(2/4) In todo.ts (ngOnInit()), add the following lines: this.engine = await CreateMLCEngine(MODEL, { initProgressCallback: ({ progress }) => this.progress.set(progress) }); this.ready.set(true); Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Downloading a model LAB #2 

(3/4) In todo.html, change the following lines: @if(!ready()) { } … Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Downloading a model LAB #2 

(4/4) The progress bar should now begin to move. Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Downloading a model LAB #2 

Storing model files locally Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Cache API Internet Website HTML/JS Cache with model files Hugging Face Note: Due to the Same-Origin Policy, models cannot be shared across origins. 

Parameter cache Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Cache API 

Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebAssembly (Wasm) – Bytecode for the web – Compile target for arbitrary languages – Can be faster than JavaScript – WebLLM uses a model- specific Wasm library to accelerate model computations 

Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebGPU – Grants low-level access to the Graphics Processing Unit (GPU) – Near native performance for machine learning applications – Supported by Chromium-based browsers on Windows and macOS from version 113, Safari 26, and Firefox 141 on Windows 

– Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Even better performance compared to WebGPU Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebNN Source: https://webmachinelearning.github.io/webnn-intro/ DEMO 

Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) WebNN: near-native inference performance Source: Intel. Browser: Chrome Canary 118.0.5943.0, DUT: Dell/Linux/i7-1260P, single p-core, Workloads: MediaPipe solution models (FP32, batch=1) 

(1/4) In todo.ts, add the following lines at the top of the class: protected readonly reply = signal(''); Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Model inference LAB #3 

(2/4) In the runPrompt() method, add the following code: this.reply.set('…'); const chunks = languageModel === 'webllm' ? this.inferWebLLM(userPrompt) : this.inferPromptApi(userPrompt); for await (const chunk of chunks) { this.reply.set(chunk); } Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Model inference LAB #3 

(3/4) In the inferWebLLM() method, add the following code: await this.engine!.resetChat(); const messages: ChatCompletionMessageParam[] = [{role: "user", content: userPrompt}]; const chunks = await this.engine!.chat.completions.create({messages, stream: true}); let reply = ''; for await (const chunk of chunks) { reply += chunk.choices[0]?.delta.content ?? ''; yield reply; } Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Model inference LAB #3 

(4/4) In todo.html, change the following line:

{{ reply() }}

You should now be able to send prompts to the model and see the responses in the template. Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Model inference LAB #3 

Stop the development server (Ctrl+C) and run npm run build Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) LAB #4 

1. In angular.json, increase the bundle size for the Angular project (property architect.build.configurations.production.budgets[0] .maximumError) to 10MB. 2. Then, run npm run build again. This time, the build should succeed. 3. If you stopped the development server, don’t forget to bring it back up again (npm start). Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Build issues LAB #4 

(1/2) In todo.ts, add the following signal at the top: protected readonly todos = signal([]); Add the following line to the addTodo() method: const text = prompt() ?? ''; this.todos.update(todos => [...todos, { done: false, text }]); Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Todo management LAB #5 

(2/2) In todo.html, add the following lines to add todos from the UI: @for (todo of todos(); track $index) { {{ todo.text }} } Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Todo management LAB #5 

@for (todo of todos(); track $index) { {{ todo.text }} } ⚠ Boo! This pattern is not recommended. Instead, you should set the changed values on the signal. But this messes up with Angular Material… Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Todo management (extended) LAB #6 

Concept and limitations For the sake of simplicity, we will add all TODOs to the prompt, in JSON format. Not all models understand JSON. Remember: LLMs have a context window (Mistral-7B: 8K). If you need to chat with larger sets of text, refer to Retrieval Augmented Generation (RAG). These are the todos: [ {name: "Wash clothes"}, {name: "Pet the dog"}, {name: "Take out the trash"} ] Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Chat with data 

System prompt Metaprompt that defines… – character – capabilities/limitations – output format – behavior – grounding data Hallucinations and prompt injections cannot be eliminated. You are a helpful assistant. Answer user questions on todos. Generate a valid JSON object. Avoid negative content. These are the user’s todos: … Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Chat with data 

Flow System message • The user has these todos: 1. … 2. … 3. … User message • How many todos do I have? Assistant message • You have three todos. Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Chat with data 

Using a system & user prompt Adjust the code in inferWebLLM() to include the system prompt: const systemPrompt = `Here's the user's todo list: ${JSON.stringify(this.todos())}`; const messages: ChatCompletionMessageParam[] = [ { role: "system", content: systemPrompt }, { role: "user", content: userPrompt } ]; Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Chat with data LAB #7 

Techniques – Providing examples (single shot, few shot, …) – Priming outputs – Specify output structure – Repeating instructions – Chain of thought – … Success also depends on the model. Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt Engineering https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/prompt-engineering 

const systemPrompt = `You are a helpful assistant. The user will ask questions about their todo list. Briefly answer the questions. Don't try to make up an answer if you don't know it. Here's the user's todo list: ${JSON.stringify(this.todos())}`; Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt Engineering LAB #8 

Alternatives Prompt Engineering Retrieval Augmented Generation Fine-tuning Custom model Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt Engineering Effort 

about://flags Enables optimization guide on device à EnabledBypassPerfRequirement Prompt API for Gemini Nano à Enabled await LanguageModel.create(); about://components about://on-device-internals Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt API LAB #9 

Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt API Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano 

Part of Chrome’s Built-In AI initiative – Exploratory API for local experiments and use case determination – Downloads Gemini Nano into Google Chrome – Model can be shared across origins – Uses native APIs directly – Fine-tuning API might follow in the future Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt API https://developer.chrome.com/docs/ai/built-in 

npm i -D @types/dom-chromium-ai add "dom-chromium-ai" to the types in tsconfig.app.json Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Prompt API LAB #10 

Add the following lines to inferPromptApi(): const systemPrompt = `The user will ask questions about their todo list. Here's the user's todo list: ${JSON.stringify(this.todos())}`; const languageModel = await LanguageModel.create({ initialPrompts: [{ role: "system", content: systemPrompt }]}); const chunks = languageModel.promptStreaming(userPrompt); let reply = ''; for await (const chunk of chunks) { reply += chunk; yield reply; } Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Local AI Models LAB #11 

npm run ng add @angular/pwa npm run build npx http-server -p 8080 -c-1 dist/genai-app/browser Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Progressive Web Apps LAB #12 

Alternatives: Ollama – Local runner for AI models – Offers a local server a website can connect to à allows sharing models across origins – Supported on macOS and Linux (Windows in Preview) https://webml-demo.vercel.app/ https://ollama.ai/ Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Local AI Models 

https://github.com/jacoblee93/fully-local-pdf-chatbot/ Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Local RAG Demo 

Alternatives: Hugging Face Transformers Pre-trained, specialized, significantly smaller models beyond GenAI Examples: – Text generation – Image classification – Translation – Speech recognition – Image-to-text Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Local AI Models 

Alternatives: Transformers.js – Pre-trained, specialized, significantly smaller models beyond GenAI – JavaScript library to run Hugging Face transformers in the browser – Supports most of the models https://huggingface.co/docs/transformers.js Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Local AI Models 

Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower quality – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Summary 

– Cloud-based models remain the most powerful models – Due to their size and high system requirements, local generative AI models are currently rather interesting for very special scenarios (e.g., high privacy demands, offline availability) – Small, specialized models are an interesting alternative (if available) – Large language models are becoming more compact and efficient – Vendors are shipping AI models with their devices – Devices are becoming more powerful for running AI workloads – Experiment with the AI APIs and make your Angular App smarter! Prompt trifft PWA Smartere Web-Apps mit Angular und lokaler KI (hands-on) Summary 

Thank you for your kind attention! Christian Liebel @christianliebel [email protected]