Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Large Language Models, Daten & APIs: Integratio...

Large Language Models, Daten & APIs: Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co.

Jajaja, der ChatGPT-Hype... Jenseits dessen taucht Christian in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie Sie AI-Funktionalität sinnvoll in Ihre Anwendungen integrieren können. Wir werden pragmatische Anwendungsfälle untersuchen, die das Potenzial von LLMs (wie GPT) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Die Teilnehmer erhalten Einblicke in LangChain als Python-Toolkit und die Möglichkeiten der Integration von LLMs. Zudem werden wir uns damit befassen, nicht nur Cloud-basierte und Closed-Source-Systeme (wie OpenAI) zu nutzen, sondern auch On-Premises- und Open-Source-Optionen in Betracht zu ziehen, um unterschiedlichen organisatorischen Anforderungen gerecht zu werden. Come in and find out!

Christian Weyer

November 08, 2023
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. Large Language Models, Daten & APIs: Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Christian Weyer @christianweyer CTO, Technology Catalyst
  2. § What & why? § LLM integration aspects § Selected

    use cases § Agents & LLMs § LLM landscape § Current state & Outlook Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Talking points 2
  3. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures

    § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Christian Weyer Co-Founder & CTO @ Thinktecture AG 3
  4. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. What’s in a name? 5
  5. § LLMs generate text based on input § LLMs can

    understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 * LLMs are not “perfect” – errors may occur, caveats like non-determinism & hallucination – these are topics to be dealt with Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Text… – really, just text? 6 Yeah: Large Language Models (LLMs)
  6. § LLMs are programs § LLMs are highly specialized neural

    networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Large Language Models demystified 7
  7. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Integration 8
  8. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Using LLMs: It’s just APIs! Inference, FTW. 9
  9. GPT-4 API access via OpenAI Playground Large Language Models, Daten

    & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. DEMO 10 Demos are using OpenAI GPT-3.5-turbo or GPT-4(-Turbo)
  10. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. The best tool for Java developers to talk to LLMs! 11 🙈
  11. § Open-source framework for developing applications powered by LLMs §

    > 1000 contributors § Python and Typescript versions § Chains as a central concept § Sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. LangChain 12
  12. § No official Java version of LangChain available § LangChain4J

    is community-driven § Works with Java 8 or higher § Supports Spring Boot 2 and 3 Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. LangChain4J 13
  13. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Use cases 14
  14. Text generation § LLMs are good in generating text §

    Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Typical LLM use cases 15
  15. Understanding text § LLM can be instructed to e.g. §

    do sentiment analysis § extract information from text § Tools (like Kor, Open AI Functions – or TypeChat) for extracting information in a structured way reliably § JSON, Python classes, TypeScript types etc. Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Typical LLM use cases 16
  16. Extracting structured data (LangChain + Kor) Large Language Models, Daten

    & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. DEMO 17
  17. Answering Questions / Chat with documents § Mixed usage of

    text embedding and a large language model § Retrieval-augmented generation (RAG) § Index one or more documents § Use text embedding to index documents § Store embeddings in a vector database § User asks question § Create embedding for the question § Search vector database to find relevant text § Use LLM with relevant input text & question to create an answer Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Typical LLM use cases 18
  18. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Answering Questions / Chat with documents - RAG Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 19 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
  19. Chat with my company’s policies (LangChain) Large Language Models, Daten

    & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. DEMO 20
  20. § Involve an LLM making decisions (via ReAct – Reasoning

    & Acting) § Which actions to take (thought) § Taking that action (executed via your code) § Seeing an observation § Repeating until done § Tools to interact with the workflow from the LLM § LangChain tools and agents § Semantic Kernel native functions § OpenAI function calling § Reliable (structured) way to interact from the LLM with your code § Proprietary to OpenAI models (fine-tuned) Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Agents: Multi-steps workflows with LLM interaction 22
  21. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. ReAct – with function calling 23 LLM My code Query Some API Some database Prompt Function definitions Final answer Answer ❓ ❓ ❗ 💡
  22. Stock market agent with OpenAI function calling (Python OpenAI SDK)

    Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. DEMO 24
  23. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Landscape 25
  24. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. LLMs everywhere 26 OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Other providers Antrophic Cohere HuggingFace … Open-source Edge IoT Server Desktop Mobile Web Open-source
  25. § Open-source community drives innovation in Generative AI § HuggingFace

    is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Open-source LLMs thrive 27
  26. Mistral OSS LLM locally on Apple M1 Max (llama.cpp &

    LM Studio) Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. DEMO 28
  27. Running OSS LLM offline in the browser - with Wasm

    & WebGPU (web-llm) Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. DEMO 29
  28. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Wrap-Up 30
  29. § Fast moving and changing field § Every week something

    “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Current state 31
  30. Huge potential for LLM-driven human-machine workflows via universal interface agents

    Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Outlook 32
  31. Large Language Models, Daten & APIs Integration von Generative-AI-Power in

    Ihre Anwendungen mit LangChain & Co. Outlook 33
  32. § Prompt Engineering Guide § https://www.promptingguide.ai/ § OpenAI API reference

    § https://platform.openai.com/docs/api-reference § Azure OpenAI Service REST API reference § https://learn.microsoft.com/en-us/azure/ai- services/openai/reference § Hugging Face Inference Endpoints § https://huggingface.co/docs/inference- endpoints/api_reference § LangChain § https://www.langchain.com/ § LangChain4J § https://github.com/langchain4j/langchain4j § ReAct: Synergizing Reasoning and Acting in Language Models § https://react-lm.github.io/ § Llama.cpp § https://github.com/ggerganov/llama.cpp § Web-llm § https://webllm.mlc.ai/ § The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) § https://arxiv.org/pdf/2309.17421.pdf § OWASP Top 10 for LLM Applications § https://owasp.org/www-project-top-10-for- large-language-model- applications/assets/PDF/OWASP-Top-10-for- LLMs-2023-slides-v1_0_1.pdf Large Language Models, Daten & APIs Integration von Generative-AI-Power in Ihre Anwendungen mit LangChain & Co. Links 34