Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Keynote Die coolste neue Programmiersprache: Englisch!? - Was bringen Generative AI & LLMs für meine Software?

Keynote Die coolste neue Programmiersprache: Englisch!? - Was bringen Generative AI & LLMs für meine Software?

Jajaja, der ChatGPT-Wahnsinn. Einfach (meist englischen) Text in ein Chat-Fenster tippen und staunen ... Jo, nice. Aber was ist das wirklich Interessante daran für uns Softwarearchitekten und -entwickler? Christian zeichnet in dieser Keynote jenseits des aktuellen Hypes ein Bild, in dem die menschliche Sprache ein wichtiger und zentraler Bestandteil zukünftiger Softwarelösungen sein kann. Die Mächtigkeit semantischer Schnittstellen, gepaart mit klassischen Daten-Architekturen, APIs und Workflows, eröffnet interessante Use Cases für die Mensch-Maschine- respektive Mensch-Code-Interaktion. Die Session bringt die neuen Ideen von generativer AI & Large Language Models in Einklang mit existierenden Architekturansätzen, damit wir als Developer das mögliche Potenzial für künftige Lösungen und Produkte besser einschätzen können.

(Und nein - dieser Abstract wurde nicht von ChatGPT geschrieben... 😉)

Christian Weyer

November 29, 2023
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. Die coolste neue Programmiersprache:
    Englisch !?
    Was bringen Generative AI & LLMs
    für meine Software?
    Christian Weyer
    @christianweyer
    CTO, Technology Catalyst

    View full-size slide

  2. § Technology catalyst
    § AI-powered solutions
    § Pragmatic end-to-end architectures
    § Microsoft Regional Director
    § Microsoft MVP for Developer Technologies & Azure
    ASPInsider, AzureInsider
    § Google GDE for Web Technologies
    [email protected] @christianweyer https://www.thinktecture.com
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Christian Weyer
    Co-Founder & CTO @ Thinktecture AG
    2

    View full-size slide

  3. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Our journey
    3
    (Gen) AI all-the-
    things?
    Integrating LLMs
    Selected
    Scenarios
    Democratizing
    Generative AI
    Exciting Times…

    View full-size slide

  4. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    AI all-the-things?
    4

    View full-size slide

  5. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    AI all-the-things?
    5
    Data
    Science
    Artificial Intelligence
    Machine Learning
    Unsupervised, supervised, reinforcement learning
    Deep Learning
    ANN, CNN, RNN etc.
    NLP
    Generative AI
    GAN, VAE, Transformers etc.
    Image / Video Generation
    GAN, VAE
    Large Language Models
    Transformers

    View full-size slide

  6. § LLMs generate text based on input
    § LLMs can understand text – this changes a lot
    § Prompts are the universal interface (“UI”) →
    unstructured text with semantics
    § Human language evolves as a first-class citizen in
    software architecture 🤯
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Large Language Models (LLMs)
    6
    Text… – really, just text?

    View full-size slide

  7. § LLMs are programs
    § LLMs are highly specialized neural networks
    § LLMs use(d) lots of data
    § LLMs need a lot of resources to be operated
    § LLMs have an API to be used through
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Large Language Models demystified
    7

    View full-size slide

  8. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Integrating LLMs
    8

    View full-size slide

  9. § LLMs are always part of end-to-end architectures
    § HTTP/Web/REST APIs
    § Databases
    § Client apps (Web, desktop, mobile)
    § etc.
    § An LLM is ‘just’ an additional asset in your architecture
    § It is not the Holy Grail for everything!
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    End-to-end architectures with LLMs
    9

    View full-size slide

  10. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Using LLMs: It’s just APIs !
    Inference, FTW.
    10

    View full-size slide

  11. GPT-4 API access via OpenAI Playground
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    DEMO
    11

    View full-size slide

  12. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    The best tool for .NET developers to talk to LLMs!
    12
    🙈

    View full-size slide

  13. § OSS framework for developing applications powered by LLMs
    § > 1000 contributors
    § Python and Typescript versions
    § Chains as a central concept
    § Sequences of LLM-related actions in code
    § Abstractions for
    § Prompts & LLMs (local and remote)
    § Memory
    § Vector stores
    § Tools
    § Loading text from a wide range of sources
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    LangChain - building LLM-based applications
    13

    View full-size slide

  14. § Microsoft’s OSS framework to integrate LLMs into applications
    § .NET, Python, and Java versions
    § Plugins encapsulate AI capabilities
    § Semantic functions for prompting
    § Native functions to run local code
    § Chain is collection of Plugins
    § Planners are orchestrating LLMS interactions
    § Not as broad feature set as LangChain
    § E.g., no concept/abstraction for loading data
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Semantic Kernel - building LLM-based applications
    14

    View full-size slide

  15. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Selected
    Scenarios
    15

    View full-size slide

  16. Text generation
    § LLMs are good in generating text
    § Regular text
    § Code
    § SQL (beware!)
    § JSON
    § etc.
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Typical LLM scenarios:
    16

    View full-size slide

  17. Extracting meaning in text
    § LLM can be instructed to, e.g.
    § Do sentiment analysis
    § Extract information from text
    § Extracting structured information
    § JSON, Python classes, TypeScript types etc.
    § Via tools like JSON Schema or Open AI Functions
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Typical LLM scenarios:
    17

    View full-size slide

  18. Extracting structured data
    (LangChain + Kor)
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    DEMO
    18

    View full-size slide

  19. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Answering Questions on Data - Retrieval-augmented generation (RAG)
    Cleanup & Split Text Embedding
    Question
    Text Embedding
    Save
    Query
    Relevant Text
    Question
    Answer
    LLM
    19
    Vector DB
    Embedding
    model
    Embedding
    model 💡
    Indexing / Embedding QA

    View full-size slide

  20. Learning about my company’s policies via Slack
    (LangChain)
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    DEMO
    20

    View full-size slide

  21. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Towards
    (half-) Autonomy
    21

    View full-size slide

  22. § Involve an LLM in making decisions (via ReAct – Reasoning & Acting)
    § Which actions to take (thought)
    § Taking that action (executed via your code)
    § Seeing an observation
    § Repeating until done
    § Tools to interact with the workflow from the LLM
    § LangChain tools and agents
    § Semantic Kernel native functions
    § OpenAI function calling
    § Reliable (structured) way to interact from the LLM with your code
    § Proprietary to OpenAI models (fine-tuned)
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Agents: Multi-step workflows with humans & LLMs
    22

    View full-size slide

  23. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Agents & ReAct – with OpenAI function calling
    LLM
    My code
    Query
    Some API Some database
    Prompt Function
    definitions
    Final answer
    Answer



    💡
    23
    Function
    Function

    View full-size slide

  24. Ask for expert availability in my company systems
    (Whisper, Semantic Kernel, TTS)
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    DEMO
    24

    View full-size slide

  25. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Democratizing
    Generative AI
    25

    View full-size slide

  26. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    LLMs everywhere
    OpenAI-related (cloud)
    OpenAI
    Azure OpenAI
    Service
    Big cloud providers
    Google Model Garden
    on Vertex AI
    Amazon Bedrock
    Other providers
    Antrophic
    Cohere
    HuggingFace
    … Open-source
    Edge
    IoT Server
    Desktop Mobile
    Web
    Open-source
    26

    View full-size slide

  27. § Open-source community drives
    innovation in Generative AI
    § HuggingFace is central place for it
    § Literally, every week a new and
    “better” LLM shows up 🤓
    § Important factors
    § Use case
    § Parameter size
    § Quantization
    § Processing power needed
    § Mistral-based family shows big
    potential for local use cases (7B params)
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Open-source LLMs thrive
    27

    View full-size slide

  28. Running Mistral OSS LLM locally
    (llama.cpp & LM Studio)
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    DEMO
    28

    View full-size slide

  29. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Exciting Times
    29

    View full-size slide

  30. § LLMs enable new scenarios & use cases to
    incorporate human language into software solutions
    § Fast moving and changing field
    § Every week something “big” happens in LLM space
    § Frameworks & ecosystem are evolving together with LLMs
    § Closed vs open LLMs
    § Competition drives invention & advancement
    § SISO (sh*t in, sh*t out)
    § Quality of results heavily depends on your data & input
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Current state
    30

    View full-size slide

  31. Potential for
    LLM-powered
    human-machine workflows
    via universal interface agents
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Outlook
    31

    View full-size slide

  32. Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Outlook
    32

    View full-size slide

  33. Thank you!
    Christian Weyer
    https://thinktecture.com/christian-weyer
    33

    View full-size slide

  34. § Semantic Kernel
    § https://learn.microsoft.com/en-us/semantic-kernel/overview/
    § LangChain
    § https://www.langchain.com/
    § LangChain Agents
    § https://python.langchain.com/docs/modules/agents/
    § ReAct: Synergizing Reasoning and Acting in Language Models
    § https://react-lm.github.io/
    § Prompt Engineering Guide
    § https://www.promptingguide.ai/
    § OpenAI API reference
    § https://platform.openai.com/docs/api-reference
    § Azure OpenAI Service REST API reference
    § https://learn.microsoft.com/en-us/azure/ai-services/openai/reference
    § Hugging Face Inference Endpoints (for various OSS LLMs)
    § https://huggingface.co/docs/inference-endpoints/api_reference
    § Llama2 on Hugging Face
    § https://huggingface.co/blog/llama2
    § OWASP Top 10 for LLM Applications
    § https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-slides-v1_0_1.pdf
    Die coolste neue Programmiersprache: Englisch
    Was bringen Generative AI & LLMs für meine Software?
    Links
    34

    View full-size slide