Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Webinar: Large Language Models, Daten & APIs: I...

Webinar: Large Language Models, Daten & APIs: Integration von Generative AI Power – mit Python & .NET

Jenseits des ChatGPT-Hypes taucht Christian in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie Sie AI-Funktionalität sinnvoll in Ihre Anwendungenintegrieren können. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (wie GPT oder Llama) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Die Teilnehmer erhalten erste Einblicke sowohl in LangChain als Python-Toolkit als auch Semantic Kernel als .NET-basiertes SDK.
Zudem werden wir darauf eingehen, nicht nur Closed-Source-Systeme (wie OpenAI) zu nutzen, sondern auch Open-Source-Optionen in Betracht zu ziehen, um unterschiedlichen Anforderungen gerecht werden zu können.

Christian Weyer

October 05, 2023
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. Large Language Models, Daten & APIs: Integration von Generative AI

    Power – mit Python & .NET Christian Weyer @christianweyer CTO, Technology Catalyst
  2. § What & why? § LLM integration aspects § Selected

    use cases § Agents & LLMs § LLM landscape § Current state & Outlook Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Talking points 2
  3. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures

    § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Christian Weyer Co-Founder & CTO @ Thinktecture AG 3
  4. Speech OpenAI Whisper tortoise-tts … Large Language Models, Daten &

    APIs Integration von Generative AI Power - mit Python & .NET Generative AI all-the-things Images Midjourney DALL·E … Audio/Music Musico Soundraw … Text OpenAI GPT Anthropic Claude Codex Llama … 5
  5. Speech OpenAI Whisper tortoise-tts … Large Language Models, Daten &

    APIs Integration von Generative AI Power - mit Python & .NET Generative AI all-the-things Images Midjourney DALL·E … Audio/Music Musico Soundraw … Text OpenAI GPT Anthropic Claude Codex Llama … 6 P
  6. § LLMs generate text based on input § LLMs can

    understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 * LLMs are not “perfect” – errors may occur, caveats like non-determinism & hallucination – these are topics to be dealt with Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Text… – really, just text? 7 Yeah: Large Language Models (LLMs)
  7. “Hm… what … how… erm… who could know this…? 🤔”

    Learning about my company’s policies in Slack Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET DEMO 8 Demos are using OpenAI GPT-3.5-turbo or GPT-4
  8. § LLMs are programs § LLMs are highly specialized neural

    networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Large Language Models demystified 9
  9. Large Language Models, Daten & APIs Integration von Generative AI

    Power - mit Python & .NET Using LLMs: It’s just APIs ! Inference, FTW. 11
  10. GPT-4 API access via OpenAI Playground Large Language Models, Daten

    & APIs Integration von Generative AI Power - mit Python & .NET DEMO 12
  11. Large Language Models, Daten & APIs Integration von Generative AI

    Power - mit Python & .NET The best tool for .NET developers to talk to LLMs! 13 🙈
  12. § Most popular open-source framework for developing applications powered by

    LLMs § Several releases per week § > 1000 contributors § Python and Typescript versions § Chains as a central concept § Sequences of actions in code § Abstractions for § Loading text from a wide range of sources § Vector stores § Prompts & LLMs (local and remote) § Tools and Agents Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET LangChain 14
  13. § Microsoft’s open-source framework to integrate LLMs into applications §

    .NET, Python, and Java versions § Plugins encapsulate AI capabilities into a single unit of functionality § Semantic functions for prompting § Native functions to run local code § Chain is collection of Plugins § Planners are similar to Agents in LangChain § Not as broad feature set as LangChain § E.g., no concept/abstraction for loading data Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Semantic Kernel 15
  14. Text generation § LLMs are good in generating text §

    Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Typical LLM use cases 17
  15. Generating text & images (multimodal) (Semantic Kernel) Large Language Models,

    Daten & APIs Integration von Generative AI Power - mit Python & .NET DEMO 18
  16. Understanding text § LLM can be instructed to e.g. §

    do sentiment analysis § extract information from text § Tools (like Kor, Open AI Functions – or TypeChat) for extracting information in a structured way reliably § JSON, Python classes, TypeScript types etc. Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Typical LLM use cases 19
  17. Extracting structured data (LangChain + Kor) Large Language Models, Daten

    & APIs Integration von Generative AI Power - mit Python & .NET DEMO 20
  18. Answering Questions / Chat with documents § Mixed usage of

    text embedding and a large language model § Retrieval-augmented generation (RAG) § Index one or more documents § Use text embedding to index documents § Store embeddings in a vector database § User asks question § Create embedding for the question § Search vector database to find relevant text § Use LLM with relevant input text & question to create an answer Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Typical LLM use cases 21
  19. Large Language Models, Daten & APIs Integration von Generative AI

    Power - mit Python & .NET Answering Questions / Chat with documents - RAG Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 22 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
  20. Chat with web site documents (Semantic Kernel) Large Language Models,

    Daten & APIs Integration von Generative AI Power - mit Python & .NET DEMO 23
  21. § Involve an LLM making decisions (via ReAct – Reasoning

    & Acting) § Which actions to take (thought) § Taking that action (executed via your code) § Seeing an observation § Repeating until done § Tools to interact with the workflow from the LLM § LangChain tools and agents § Semantic Kernel native functions § OpenAI function calling § Reliable (structured) way to interact from the LLM with your code § Proprietary to OpenAI models (fine-tuned) Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Agents: Multi-steps workflows with LLM interaction 25
  22. Large Language Models, Daten & APIs Integration von Generative AI

    Power - mit Python & .NET ReAct – with function calling 26 LLM My code Query Some API Some database Prompt Function definitions Final answer Answer ❓ ❓ ❗ 💡
  23. Stock market agent with OpenAI function calling (Python OpenAI SDK)

    Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET DEMO 27
  24. § Open AI § Azure Open AI Service § Google

    Model Garden on Vertex AI § Amazon Sagemaker § Hugging Face § Fermyon Serverless AI § Self-hosting (hard to scale) § Device / browser Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET LLM providers 29
  25. Running an open-source LLM offline in the browser (with Wasm)

    (web-llm) Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET DEMO 30
  26. § Fast moving and changing field § Every week something

    “big” happens in LLM space § Frameworks are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § Bias of the models § Which data was the model trained on? § Finding the right prompt can be tedious § Prompting differs between LLMs § Prompt injection as a threat § Will improve over time § Context size is currently limited § Will likely diminish over time § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Current state 32
  27. Huge potential for LLM-driven human-machine workflows via UI (universal interface)

    agents Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Outlook 33 P
  28. § Semantic Kernel § https://learn.microsoft.com/en-us/semantic-kernel/overview/ § LangChain § https://www.langchain.com/ §

    LangChain Agents § https://python.langchain.com/docs/modules/agents/ § ReAct: Synergizing Reasoning and Acting in Language Models § https://react-lm.github.io/ § Prompt Engineering Guide § https://www.promptingguide.ai/ § OpenAI API reference § https://platform.openai.com/docs/api-reference § Azure OpenAI Service REST API reference § https://learn.microsoft.com/en-us/azure/ai-services/openai/reference § Hugging Face Inference Endpoints (for various OSS LLMs) § https://huggingface.co/docs/inference-endpoints/api_reference § Llama2 on Hugging Face § https://huggingface.co/blog/llama2 § OWASP Top 10 for LLM Applications § https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-slides-v1_0_1.pdf Large Language Models, Daten & APIs Integration von Generative AI Power - mit Python & .NET Links 35