Smart Form Filler Spice up your Forms With Generative AI & LLMs Christian Liebel | Consultant | Thinktecture AG | [email protected]

Hello, it’s me. Smart Form Filler Spice up your Forms With Generative AI & LLMs Christian Liebel X: @christianliebel Email: christian.liebel @thinktecture.com Angular & PWA Slides: thinktecture.com /christian-liebel

Overview Smart Form Filler Spice up your Forms With Generative AI & LLMs Generative AI Text OpenAI GPT Mistral … Speech OpenAI Whisper tortoise-tts … Images DALL·E Stable Diffusion … Audio/Music Musico Soundraw …

Large: Trained on lots of data Language: Process and generate text in natural language Models: Programs/neural networks Examples: – GPT (ChatGPT, Bing Chat, …) – Gemini, Gemma (Google) – LLaMa (Meta AI) Smart Form Filler Spice up your Forms With Generative AI & LLMs Large Language Models

Prompts serve as the universal interface Unstructured text conveying specific semantics Paradigm shift in software architecture Natural language becomes a first-class citizen Caveats Non-determinism and hallucination, prompt injections Smart Form Filler Spice up your Forms With Generative AI & LLMs Large Language Models

Smart Form Filler Spice up your Forms With Generative AI & LLMs Use cases beyond chat bots…? Source: https://www.apple.com/chde/apple-intelligence/

Just transfer the 17.34 euros to me, my IBAN is DE02200505501015871393. I am with Hamburger Sparkasse (HASPDEHH). Data Extraction Smart Form Filler Spice up your Forms With Generative AI & LLMs Use Case Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz

Just transfer the 17.34 euros to me, my IBAN is DE02200505501015871393. I am with Hamburger Sparkasse (HASPDEHH). Data Extraction Smart Form Filler Spice up your Forms With Generative AI & LLMs Use Case Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz

protected readonly formGroup = this.fb.group({ firstName: [''], lastName: [''], addressLine1: [''], addressLine2: [''], city: [''], state: [''], zip: [''], country: [''], }); Smart Form Filler Spice up your Forms With Generative AI & LLMs Idea Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz Smart Form Filler (LLM)

Smart Form Filler Spice up your Forms With Generative AI & LLMs Idea DEMO

Form Field Prompt Generator Model Backend Response Parser Smart Form Filler Spice up your Forms With Generative AI & LLMs Architecture

Smart Form Filler Spice up your Forms With Generative AI & LLMs Form Field “Try to determine the country based on the user’s input.” “If present, put the department in this field.”

Async Clipboard API Allows reading from/writing to the clipboard in an asynchronous manner Reading from the clipboard requires user consent first (privacy!) Supported by Chrome, Edge, Safari and Firefox Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Generator

System message • The form has the following setup: { "firstName": "", "lastName": "" } User message • Passt, dann hier schonmal meine Adresse: Peter Müller Assistant message • { "firstName": "Peter", "lastName": "Müller" } Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Generator

Cloud backends Smart Form Filler Spice up your Forms With Generative AI & LLMs Model Backend

Local backend (WebLLM) Smart Form Filler Spice up your Forms With Generative AI & LLMs Model Backend

https://webllm.mlc.ai/ Smart Form Filler Spice up your Forms With Generative AI & LLMs WebLLM

Smart Form Filler Spice up your Forms With Generative AI & LLMs

Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt API Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano

Part of Chrome’s Built-In AI initiative – Exploratory API for local experiments and use case determination – Downloads Gemini Nano into Google Chrome – Model can be shared across origins – Uses native APIs directly – Fine-tuning API might follow in the future Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt API https://developer.chrome.com/docs/ai/built-in

Demo: Smart Form Filler Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt API DEMO

Comparison 22.98 33.96 568.06 909.62 0 100 200 300 400 500 600 700 800 900 1000 WebLLM (Mistral-7b, M1) WebLLM (Mistral-7b, M3) Groq (Mixtral-8x7b) Groq (Llama3-8b) Tokens/sec Smart Form Filler Spice up your Forms With Generative AI & LLMs Performance Own tests (WebLLM: 23.03.2024, Groq: 01.05.2024)

Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Parser Assistant message • { "firstName": "Peter", "lastName": "Müller" }

Assistant message Parsing the assistant message as text/JSON/… Tool calling Specifying a well-defined interface via a JSON schema called by the LLM (safer, growing support) Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Parser

– Super accurate – Versatile thanks to layered architecture – Runs in the browser, locally, or in the cloud – Very fast execution is possible via Groq https://github.com/thinktecture-labs/smart-form-filler Smart Form Filler Spice up your Forms With Generative AI & LLMs Summary

Thank you for your kind attention! Christian Liebel @christianliebel [email protected]