BASTA! Spring 2024: Generative AI: Large Language Models - Szenarien, Use Cases und Patterns für Businessanwendungen

Jenseits des ChatGPT-Hypes taucht Christian Weyer in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie Sie AI-Funktionalität sinnvoll in Ihre Architekturen und Anwendungen integrieren können. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (wie GPT oder Llama) demonstrieren und erörtern, wie AI-Techniken in bestehende Lösungen einbezogen werden können. Die Teilnehmer erhalten erste Einblicke sowohl in LangChain als Python-Toolkit als auch Semantic Kernel als .NET-basiertes SDK. Zudem werden wir nicht nur Closed-Source-Systeme (wie OpenAI) betrachten, sondern auch Open-Source-Optionen evaluieren, um unterschiedlichen Anforderungen gerecht werden zu können (und nein, dieser Talk-Titel und Abstract wurde nicht von ChatGPT geschrieben).

Transcript

1. Generative AI Large Language Models: Szenarien, Use Cases und Patterns für Businessanwendungen Christian Weyer @christianweyer CTO, Technology Catalyst
2. The image part with relationship ID rId12 was not found in the file. The image part with relationship ID rId12 was not found in the file. Special Day Generative AI für Business-Anwendungen Thema Sprecher Datum, Uhrzeit Generative AI: Large Language Models – Szenarien, Use Cases und Patterns für Business-Anwendungen Christian Weyer DI, 13. Februar 2024, 10.45 bis 11.45 Generative AI: A Story About LLM Prompting (and how Tools like TypeChat Can Help) Rainer Stropek DI, 13. Februar 2024, 12.15 bis 13.15 Generative AI: Semantische Suche und LLMs jenseits des Hello World- RAG-Tutorials Sebastian Gingter DI, 13. Februar 2024, 15.30 bis 16.30 Generative AI: Optimierte Informationssuche durch AI-gesteuerte Datenquellenwahl Marco Frodl DI, 13. Februar 2024, 17.00 bis 18.00 Generative AI: Private GPT LLMs: Azure OpenAI Service sicher deployen mit Terraform Kenny Pflug DI, 13. Februar 2024, 19.00 bis 20.00
3. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Christian Weyer Co-Founder & CTO @ Thinktecture AG 3
4. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Our journey 4 AI all-the- things? LLMs in your Solutions Selected Scenarios Exciting Times… Democratizing Generative AI
5. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen AI all-the-things? 5
6. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen AI all-the-things? 6 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
7. § LLMs generate text based on input § LLMs can understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Large Language Models (LLMs) 7 Text… – really, just text?
8. § LLMs are programs § LLMs are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Large Language Models demystified 8
9. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen LLMs in your Solutions 9
10. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Using LLMs: It’s just APIs ! Inference, FTW. 10
11. GPT-4 API access OpenAI Playground Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 11
12. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen The best tool for .NET developers to talk to LLMs! 12 🙈
13. § OSS framework for developing applications powered by LLMs § > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen LangChain - building LLM-based applications 13
14. § Microsoft’s OSS framework to integrate LLMs into applications § .NET, Python, and Java versions § Plugins encapsulate AI capabilities § Semantic functions for prompting § Native functions to run local code § Planners are orchestrating LLM interactions § Not as broad feature set as LangChain § E.g., no concept/abstraction for loading data Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Semantic Kernel - building LLM-based applications 14
15. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Talk to your Data 15
16. Real-World Generative AI Sprachzentrierte Anwendungen mit Large Language Models, Python & .NET Answering Questions on Data Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 16 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA Intro
17. Learning about my company’s policies via Slack LangChain (Python), Weaviate, Open AI GPT Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 17
18. Chat with web site documents Semantic Kernel (.NET), SqLite, Open AI GPT Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 18
19. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Talk to your Systems 19
20. § LLM can be instructed to, e.g. § Do sentiment analysis § Extract information from text § Extract structured information § Schema description: JSON, TypeScript types, etc. § Implement via tools like Kor, TypeChat, or Open AI Function / Tool Calling Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Extracting meaning in text for structured data 20
21. Extracting structured data OpenAI Tool Calling with LangChain Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 21
22. § Tool / function calling standard established by OpenAI § Functions for interactions § LLM chooses to output JSON object containing arguments to call one or many functions § LLM does not call the function § You do it in your code § All major libs support tool calling with abstractions § OpenAI SDKs § Langchain § Semantic Kernel Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Extending LLM capabilities 22 curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "What is the weather like in Boston?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } } ], "tool_choice": "auto" }'
23. Extending LLM capabilities OpenAI Tool Calling with Semantic Kernel Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 23
24. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Democratizing Generative AI 24
25. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Open-source Edge IoT Server Desktop Mobile Web 25 Other providers Antrophic Cohere Mistral AI Hugging Face Open-source
26. § Open-source community drives innovation in Generative AI § HuggingFace is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Open-source LLMs thrive 26
27. Local RAG with Zephyr open-source LLM llama.cpp, ollama, LangChain Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 27
28. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen End-to-End 28
29. Ask for expert availability in my company systems Angular, Speech-to-text, internal HTTP API, node.js OpenAI SDK + Tool Calling, Open AI GPT, Text-to-speech Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen DEMO 29
30. 30 Angular PWA OpenAI Speech-to-Text TT Panorama Gateway OpenAI GPT-4 OpenAI Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio TT Panorama Query Panorama API Availability Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen
31. Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Exciting Times… 31
32. § LLMs enable new scenarios & use cases to incorporate human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § Security topics need to be addressed § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Current state 32
33. § Beginning of a long way Generative AI - Large Language Models Szenarien, Use Cases und Patterns für Business-Anwendungen Generative to Interactive 33 https://www.technologyreview.com/2023/09/15/1079624/deepmind-inflection-generative-ai-whats-next-mustafa-suleyman
34. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 34