programmier.bar: Large Language Models, Daten & APIs - Integration von Generative AI in eigene Anwendungen

Menschliche Sprache als Universal Interface für Software-Lösungen - hört sich spannend an! Jenseits des ChatGPT-Hypes taucht Christian in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie man AI-Funktionalität sinnvoll in eigene Anwendungen integrieren kann. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (wie GPT oder Llama) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Die Teilnehmer erhalten erste Einblicke in Frameworks wie LangChain aus der Python-Welt zur Programmierung LLM-basierter Systeme. Zudem werden wir darauf eingehen, nicht nur Closed-Source-Systeme (wie OpenAI) zu nutzen, sondern auch Open-Source-Optionen (wie Llama) in Betracht zu ziehen, um unterschiedlichen Anforderungen gerecht werden zu können.

Transcript

1. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Christian Weyer @christianweyer CTO, Technology Catalyst
2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
3. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Our journey 3 AI all-the- things? Integrating LLMs Selected Scenarios End-to-End Exciting Times… Democratizing Generative AI
4. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen AI all-the-things? 4
5. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen AI all-the-things? 5 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
6. § LLMs generate text based on input § LLMs can understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models (LLMs) 6 Text… – really, just text?
7. § LLMs are programs predicting the next token § LLMs are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models demystified 7
8. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Integrating LLMs 8
9. § LLMs are always part of end-to-end architectures § HTTP/Web/REST APIs § Databases § Client apps (Web, desktop, mobile) § etc. § An LLM is ‘just’ an additional asset in your architecture § It is not the Holy Grail for everything! Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen End-to-end architectures with LLMs 9
10. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Using LLMs: It’s just APIs ! Inference, FTW. 10
11. GPT-4 API access via OpenAI Playground Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 11
12. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen The best tool for developers to talk to LLMs! 12
13. § OSS framework for developing applications powered by LLMs § > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen LangChain - building LLM-based applications 13
14. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Selected Scenarios 14
15. Text generation § LLMs are good in generating text § Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Typical LLM scenarios: 15
16. Extracting meaning in text § LLM can be instructed to, e.g. § Do sentiment analysis § Extract information from text § Extracting structured information § JSON, TypeScript types, etc. § Via tools like Kor, TypeChat, or Open AI Function Calling Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Typical LLM scenarios: 16
17. Extracting structured data (LangChain + Kor) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 17
18. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Answering Questions on Data - Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 18 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
19. Learning about my company’s policies via Slack (LangChain) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 19
20. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Democratizing Generative AI 20
21. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Open-source Edge IoT Server Desktop Mobile Web 21 Other providers Antrophic Cohere Mistral AI Hugging Face Open-source
22. § Open-source community drives innovation in Generative AI § HuggingFace is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Open-source LLMs thrive 22
23. Local RAG with Mistral OSS LLM (llama.cpp & LM Studio) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 23
24. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen End-to-End 24
25. Ask for expert availability in my company systems (Speech-to-text, internal API, node.js OpenAI SDK, GPT-4, Text-to-speech) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 25
26. 26 Angular PWA OpenAI Speech-to-Text TT Panorama Gateway OpenAI GPT-4 OpenAI Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio TT Panorama Query Panorama API Availability Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen
27. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Exciting Times… 27
28. § LLMs enable new scenarios & use cases to incorporate human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Current state 28
29. Potential for LLM-powered human-machine workflows via universal interface agents Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Outlook 29
30. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 30 Selected demos: Extract structured information: https://github.com/thinktecture-labs/llm-extract-structured-information-langchain-kor Local RAG with PDFs: https://github.com/thinktecture-labs/rag-chat-with-pdf-local-llm