'Talk to your systems‘ Integrating Gen AI into your architectures with structured LLM output Christian Weyer | Co-Founder & CTO | Thinktecture AG | [email protected]

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output Using Language Models – two major scenarios 2 ‘Talk to your data’ RAG ‘Talk to your systems’ Structured Output

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output TALK TO YOUR SYSTEMS WHY? 3

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output 4 Human language rocks Extending access to software

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output A classical UI – strong UX for certain use cases 5

One possible extended UX pattern 'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output Language-enabled UIs 6

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output 7 LLMs Enabling new scenarios

§ LLMs are always part of end-to-end architectures § Client apps (Web, desktop, mobile) § Services with APIs § Databases § etc. § LLMs / SLMs enable human language as a first-class citizen 🤯 'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output End-to-end architectures with LLMs 8 Clients Services LLMs Desktop Web Mobile Service A Service B Service C API Gateway Monitoring LLM 1 LLM 2

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output Goal: Extracting structured data from unstructured semantic input (aka human language text) 9 Automating GUI-based processes & workflows Creating efficient workflows with universal interfaces

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output TALK TO YOUR SYSTEMS HOW? 10

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output 11 Prompting Talk to me!

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output 12 ‘Function Calling’ Give me schema!

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output 13 Pydantic & Instructor Make it easier!

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output TALK TO YOUR SYSTEMS WHAT? 14

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output Creating efficient workflows with universal interfaces: Talk-to-Your-Company 15 Angular PWA Speech-to-Text Gen AI Gateway (Python FastAPI) LLM / SLM Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Tool calling) Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio Internal Business API (node.js – veeeery old) Query Availability API Availability When is CL…? CL will be…

Filling Angular forms with human language input – TypeScript-only protected readonly formGroup = this.fb.group({ firstName: [’’], lastName: [’’], addressLine1: [’’], addressLine2: [’’], city: [’’], state: [’’], zip: [’’], country: [’’] }); 'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output Automating GUI-based processes: Smart form filling OK, nice – so here is my address then:: Peter Schmitt, Rheinstr. 7 in Schkeuditz – postcode is 04435, BTW. 16 Smart Form Filler (TS code & LLM)

'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output TALK TO YOUR SYSTEMS RECAP 17

§ Human language enables new powerful use cases & access to our software § Automating GUI-based processes & workflows § Creating efficient workflows with universal interfaces § Always use structured output § Secret sauce for integrating LLMs into your application architectures § Instructor is a helpful library to boost LLM use cases § Implements best practices § Uses ‘Function Calling’ & other approaches § Supports any LLM / SLM § Integrates with FastAPI 'Talk to your systems' Integrating Gen AI into your architectures with structured LLM output Recap & Recommendations 18

Thank you! Christian Weyer https://thinktecture.com/christian-weyer 19 https://github.com/thinktecture-labs/talk-to-your-systems https://github.com/thinktecture-labs/smart-form-filler