Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intelligente Formulare: Angular Forms mit AI & ...

Intelligente Formulare: Angular Forms mit AI & LLMs aufpeppen

Ob Adressdaten, Bankdaten oder Bewerbungsinformationen – das Ausfüllen umfangreicher Formulare kann für Benutzer eine zeitraubende und mühsame Aufgabe sein. Wie wäre es, wenn Benutzer einfach einen Text in die Zwischenablage kopieren könnten und das Formular sich automatisch ausfüllt? Ein Smart Form Filler macht genau das möglich: Mit Hilfe eines Large Language Models (LLM), das sowohl den Inhalt der Zwischenablage als auch die Struktur des Formulars erkennt, können Ihre Angular-Formulare vollautomatisch ausgefüllt werden – auf Wunsch auch vollständig lokal und offlinefähig. Christian Liebel zeigt, wie dieses Generative-AI-Feature in Angular implementiert werden kann.

Christian Liebel

March 05, 2025
Tweet

More Decks by Christian Liebel

Other Decks in Programming

Transcript

  1. Hello, it’s me. Intelligente Formulare Angular-Forms mit AI & LLMs

    aufpeppen Christian Liebel X: @christianliebel Bluesky: @christianliebel.com Email: christian.liebel @thinktecture.com Angular, PWA & Generative AI Slides: thinktecture.com /christian-liebel
  2. Overview Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Generative

    AI Text OpenAI GPT Mistral … Audio/Music Musico Soundraw … Images DALL·E Firefly … Video Sora Runway … Speech Whisper tortoise-tts …
  3. Overview Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Generative

    AI Text OpenAI GPT Mistral … Audio/Music Musico Soundraw … Images DALL·E Firefly … Video Sora Runway … Speech Whisper tortoise-tts …
  4. Large: Trained on lots of data Language: Process and generate

    text in natural language Models: Programs/neural networks Examples: – GPT (ChatGPT, Bing Chat, …) – Gemini, Gemma (Google) – LLaMa (Meta AI) Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Large Language Models
  5. Prompts serve as the universal interface Unstructured text conveying specific

    semantics Paradigm shift in software architecture Natural language becomes a first-class citizen Caveats Non-determinism and hallucination, prompt injections Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Large Language Models
  6. Just transfer the 17.34 euros to me, my IBAN is

    DE02200505501015871393. I am with Hamburger Sparkasse (HASPDEHH). Data Extraction Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Use Case Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz
  7. Just transfer the 17.34 euros to me, my IBAN is

    DE02200505501015871393. I am with Hamburger Sparkasse (HASPDEHH). Data Extraction Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Use Case Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz
  8. protected readonly formGroup = this.fb.group({ firstName: [''], lastName: [''], addressLine1:

    [''], addressLine2: [''], city: [''], state: [''], zip: [''], country: [''], }); Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Idea Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz Smart Form Filler (LLM)
  9. Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Form Field

    “The insurance number starts with INS, followed by six characters.” “Try to determine the post code based on the input.”
  10. Async Clipboard API Allows reading from/writing to the clipboard in

    an asynchronous manner Reading from the clipboard requires user consent first (privacy!) Supported by Chrome, Edge and Safari and Firefox Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator
  11. Message types System message • The form has the following

    setup: { "firstName": "", "lastName": "" } User message • Passt, dann hier schonmal meine Adresse: Peter Müller Assistant message • { "firstName": "Peter", "lastName": "Müller" } Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator
  12. Message types Some models add additional output or may hallucinate

    properties, so the response cannot be parsed back to JSON: Sure! I’m glad to help. Here’s your JSON: { "firstName": "Peter", "lastName": "Müller" } There are two alternatives: 1. JSON mode: Special output mode that guarantees valid JSON. 2. Tool/function calling: Well-defined API for the model to call a tool. Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator
  13. Tool calling { "name": "fillForm", "description": "Fills in the form.",

    "parameters": { "type": "object", "properties": { "firstName": { "type": "string", "description": "The user's first name." }, }, } Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator Natural language descriptions evaluated by the LLM JSON schema that defines the tool’s API shape
  14. Tool messages System message • You are a helpful assistant

    that helps filling in forms. • You can call these tools if needed: User message • Passt, dann hier schonmal meine Adresse: Peter Müller Tool message • { "tool_name": "fillForm", "tool_args": { "firstName": "Peter", "lastName": "Müller" } } Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator { "name": "fillForm", "description": "Fills in the form.", "parameters": { "type": "object", "properties": { "firstName": { "type": "string", "description": "The user's first name." }, }, }
  15. Multimodal models Recent LLMs are capable of processing other content

    besides just text. Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator
  16. Multimodal models System message • You are a helpful assistant

    that helps filling in forms. • You can call these tools if needed: User message Tool message • { "tool_name": "fillForm", "tool_args": { "firstName": "Peter", "lastName": "Müller" } } Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator { "name": "fillForm", "description": "Fills in the form.", "parameters": { "type": "object", "properties": { "firstName": { "type": "string", "description": "The user's first name." }, }, }
  17. Realtime models Recent models are capable of having realtime conversations.

    Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Generator
  18. Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt API

    Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano
  19. Part of Chrome’s Built-In AI initiative – Exploratory API for

    local experiments and use case determination – Downloads Gemini Nano into Google Chrome – Model can be shared across origins – Multimodality in development – Fine-tuning API might follow in the future Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt API https://developer.chrome.com/docs/ai/built-in
  20. Comparison 45 33 1200 0 200 400 600 800 1000

    1200 1400 WebLLM (Llama3-8b, M4) Azure OpenAI (gpt-4o-mini) Groq (Llama3-8b) Tokens/sec Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Performance WebLLM/Groq: Own tests (14.11.2024), OpenAI/Azure OpenAI: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/provisioned-throughput (18.07.2024)
  21. Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Parser

    Assistant message • { "firstName": "Peter", "lastName": "Müller" }
  22. Assistant message Parsing the assistant message as text/JSON/… Tool calling

    Specifying a well-defined interface via a JSON schema called by the LLM (safer, growing support) Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Prompt Parser
  23. – Super accurate – Versatile thanks to layered architecture –

    Runs in the browser, locally, or in the cloud – Very fast execution is possible via Groq – Multimodal models can process content types other than text – Realtime models allow conversations Intelligente Formulare Angular-Forms mit AI & LLMs aufpeppen Summary