RWEST DevCon: AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs)

Human language as a universal interface (UI) for software solutions – written or even spoken. Sounds exciting! Beyond the ChatGPT hype, Christian Weyer dives into the world of Large Language Models (LLMs) and focuses on how Generative AI based on LLMs can be sensibly integrated into your own application solutions via data and APIs. He presents pragmatic scenarios and use cases that demonstrate the potential of LLMs like GPT-4 or open-source variants. Modern "user interfaces" beyond GUIs & colorful designs: and action!

Transcript

1. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Christian Weyer Co-Founder & CTO @christianweyer
2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
3. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Our journey today AI all-the- things? LLMs in your Solutions Talk to your Data Exciting Times… Democratizing Generative AI Talk to your Systems 3
4. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) AI all-the-things? 4
5. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) A.I. 5
6. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) AI all-the-things? Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers NLP (Natural Language Processing) 6
7. § LLMs generate text based on input § LLMs can understand text – this changes a lot § Without having to train them on data, domains, or use cases § Prompts are the universal interface (“UI”): unstructured text with semantics as input § Human language evolves as a first-class citizen in software architecture 🤯 AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Large Language Models (LLMs) – like GPT powering ChatGPT Text… – really, just text? 7
8. § LLMs are programs § LLMs are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Large Language Models demystified 🔍 8
9. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) LLMs in your Solutions 9
10. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Using LLMs: It’s just HTTP APIs Inference, FTW. 10
11. GPT API access OpenAI Playground AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) DEMO 11
12. Barebones SDKs § E.g. Open AI SDK § Available for any programming language § Basic abstraction over HTTP APIs § Also available from other providers § Mistral § Cohere § Etc. Frameworks – e.g. LangChain § Large community § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs § Memory § Vector stores § Tools § Loading text from a wide range of sources AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Building LLM-based end-to-end applications 12
13. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Talk to your Data 13
14. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Answering Questions on Data Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM Embedding model Embedding model 💡 Indexing / Embedding Question Answering Vector DB 14
15. RAG: Learning about company’s policies via Slack LangChain Python, Weaviate – Mixtral-8x7B on Groq AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) DEMO 15
16. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Talk to your Systems 16
17. § Write or speak your input → get structured data for your programs & systems § Clever & strict prompting § Schema description: JSON, TypeScript types, etc. § Framework or tools support § Kor, TypeChat, etc. § Open AI Function Calling AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Extract structured data from textual information 17
18. Extracting structured data from text: Flight search OpenAI Function Calling, LangChain Python, FastAPI, JS - GPT-4o AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) DEMO 18
19. Extracting structured data from voice: Form filling JSON extraction, Angular Forms - Mixtral-8x7B on Groq AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) DEMO 19
20. § Integrate LLM-external systems to aid LLMs § Tool / function calling standard established by OpenAI § LLM outputs JSON containing arguments to call one or many functions § LLM does not call the function § All major libs support tool calling § OpenAI SDKs § LangChain § Semantic Kernel § etc. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Extending LLM capabilities curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "What is the weather like in Boston?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } } ], "tool_choice": "auto" }' 20
21. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Talking to internal APIs – Ask for experts availability Angular PWA Open AI Speech-to-Text Internal Systems Gateway Open AI GPT-4-T Open AI Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Tool calling) Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio Internal Company API Query Availability API Availability When is CL…? CL will be… 21
22. Ask for experts availability in my company systems Angular, Speech-to-text, internal HTTP API, node.js OpenAI SDK + Tool Calling, Text-to-speech – GPT-4-Turbo AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) DEMO 22
23. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Democratizing Generative AI 23
24. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Open-source Edge IoT Server Desktop Mobile Web Other providers Antrophic Cohere Mistral AI Hugging Face Open-source 24
25. § Open-source community drives innovation in Generative AI § Important factors § Use case § Parameter size § Quantization § Processing power needed § Llama- & Mistral-based families show big potential for local use cases AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Open-source LLMs thrive 25
26. Local RAG: Mistral-7B open-source LLM llama.cpp, ollama, LangChain Python, StreamLit AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) DEMO 26
27. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) Exciting Times… 27
28. AI in Action: 'UIs' with a difference Language as access to your own software with Large Language Models (LLMs) 2024: The year of SLMs & CPU inference 28
29. Thank you! Christian Weyer christian.weyer@thinktecture.com https://thinktecture.com/christian-weyer 29