Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to use GPT with your Symfony application [EN]

How to use GPT with your Symfony application [EN]

With the release of ChatGPT, OpenAI has generated a lot of buzz around GPT and AI in general. For software architects and developers, this is not only a game changer when it comes to learning or debugging, but also introduces a new tool to embed into your system's architecture. In this talk, we will take a closer look at the basic concepts of GPT, the relevant models, and how you can effectively utilize them in your Symfony application.

Example Use Cases:
* https://github.com/chr-hertel/symfonycon-bot-2023
* https://github.com/chr-hertel/decomplex

Christopher Hertel

December 08, 2023
Tweet

More Decks by Christopher Hertel

Other Decks in Technology

Transcript

  1. Christopher Hertel - @el_stoffel
    &
    GPT
    How to use
    Symfony
    with your
    application

    View full-size slide

  2. Christopher Hertel - @el_stoffel
    Who is that guy?
    Christopher Hertel
    Software Architect in Berlin
    @el_stoffel

    View full-size slide

  3. Christopher Hertel - @el_stoffel
    What should I talk about?
    Alright, Christopher, that's easy:
    1. LLM & GPT
    Start with the fundamentals of Large Language Models and GPT.
    2. GPT Fundamentals
    Bring in fundamental concepts of GPT itself - best with a simple example.
    3. Retrieval Augmented Generation
    Explain the basic concept of RAG and bring in a real-world example.
    4. Function Calling
    Walk them through function calling - and if you have time, bring an example.
    5. Q & A
    Well, let's see if they're still awake.
    SymfonyCon Brussels 2023

    View full-size slide

  4. Christopher Hertel - @el_stoffel
    LLM & GPT
    ChatGPT and beyond

    View full-size slide

  5. Christopher Hertel - @el_stoffel
    LLM & GPT
    • Model
    • Result of training an AI algorithm with data, e.g. neural network
    • LLM == Large Language Models
    • Specific model to understand and generate language
    • Trained on a large amount of data, e.g. the internet
    • Basic idea is completion on top of probabilities
    • GPT == Generative Pre-Trained Transformer
    • First released by OpenAI
    • Generates language, word by word
    • Pre-trained on large data-sets
    • Transformer is a specific LLM architecture
    ChatGPT => ShopMVC

    View full-size slide

  6. Christopher Hertel - @el_stoffel
    Examples
    • Models
    • Open AI: gpt-3.5-turbo or gpt-4
    • Meta: Llama 2 or CodeLlama
    • Anthropic: Claude
    • OSS: Falcon, GPT-J or GPT4All
    • BYO: Stanford Alpaca
    • Runtime
    • OpenAI & Microsoft Azure
    • huggingface.co
    • replicate.com
    • Self-hosted: GPT-J or GPT4All
    ChatGPT …
    • is not a model
    • is a product, using gpt-3.5 or gpt-4
    • is for free for a reason

    View full-size slide

  7. Christopher Hertel - @el_stoffel
    GPT Fundamentals
    Prompts, Temperature and more

    View full-size slide

  8. Christopher Hertel - @el_stoffel
    Prompts
    • Messages that interact with the model
    • Style of prompts really important => “Prompt Engineering”
    • A lot of “trial & error”
    • Roles
    • User
    Messages created by a user
    • Assistant
    Messages created by a system
    • System
    High level instructions for the conversation

    View full-size slide

  9. Christopher Hertel - @el_stoffel
    System Prompt
    The
    In Action

    View full-size slide

  10. Christopher Hertel - @el_stoffel
    You are ChatGPT, a large language model trained by OpenAI.
    You respond with only one sentence.
    System
    What should I do tomorrow?
    User
    That depends on your personal goals and
    priorities.
    Assistant

    View full-size slide

  11. Christopher Hertel - @el_stoffel
    What should I do tomorrow?
    User
    Do what makes you happy and fulfilled.
    Assistant
    You are my parent and always want "the best" for me.
    You respond only with one sentence.
    System

    View full-size slide

  12. Christopher Hertel - @el_stoffel
    What should I do tomorrow?
    User
    Get up and go for a run or try a new sport!
    Assistant
    You are a sports trainer that helps people to overcome their
    hesitation to do some sports.
    You respond with only one sentence.
    System

    View full-size slide

  13. Christopher Hertel - @el_stoffel
    What should I do tomorrow?
    User
    Make a to-do list and prioritize your tasks
    based on their importance and urgency.
    Assistant
    You are the assistant of Fabien Potencier, who is the creator of the
    PHP framework Symfony, and CPO of Platform.sh.
    You respond with only one sentence.
    System

    View full-size slide

  14. Christopher Hertel - @el_stoffel
    What should I do tomorrow?
    User
    Invest your time in something that will
    generate revenue.
    Assistant
    You are a capitalism fan boy and value money over people.
    You respond only with one sentence.
    System

    View full-size slide

  15. Christopher Hertel - @el_stoffel
    What should I do tomorrow?
    User
    Invest your time in something that will
    generate revenue.
    Assistant
    You are a capitalism fan boy and value money over people.
    You respond only with one sentence.
    System

    View full-size slide

  16. Christopher Hertel - @el_stoffel
    Context
    • Chat history => sum of messages
    • Provided all over again
    • The model interaction is stateless
    • state only training and fine-tuning
    • Limited by the number of “tokens”
    • gpt-3.5-turbo supports 16.385 tokens
    • gpt-4 supports 8.192 tokens
    • gpt-4-32k supports 32.768 tokens
    • gpt-4-turbo supports 128.000 tokens
    User
    Assistant
    System
    User
    Assistant
    User
    Assistant
    Context

    View full-size slide

  17. Christopher Hertel - @el_stoffel
    Context
    Payload will grow and grow …

    View full-size slide

  18. Christopher Hertel - @el_stoffel
    Temperature
    0.0 2.0
    creative
    deterministic
    • Mechanism how GPT handles probabilities
    • Often referred to as “randomness”
    More Options
    max tokens, n [of choices], penalty
    of presence or frequency, format,
    seed, top_p and tools

    View full-size slide

  19. Christopher Hertel - @el_stoffel
    Usage with Symfony
    • Architecture constraints
    • GPT is a component to understand and generate language
    • By default used via Web API
    • Libraries
    • HTTP clients like symfony/http-client or guzzlehttp/guzzle are sufficient
    • API client libraries like
    • openai-php/client with Symfony integration openai-php/symfony
    • orhanerday/open-ai that claims to be better
    • LLM Framework packages
    • Package kambo/langchain based on Langchain
    • Package theodo-group/llphant based on Langchain and LlamaIndex

    View full-size slide

  20. Christopher Hertel - @el_stoffel
    decomplex.me
    Example Use Case

    View full-size slide

  21. Christopher Hertel - @el_stoffel
    Use case: Implementation I
    • Install openai-php/symfony with Composer
    • Configure OpenAI token in environment variable
    $ composer require openai-php/symfony
    ###> openai-php/symfony ###
    # https://platform.openai.com/account/api-keys
    OPENAI_API_KEY=sk-rWYtxvD1dYouR34LlyTH1nkth1sI5real?
    OPENAI_ORGANIZATION=
    ###< openai-php/symfony ###

    View full-size slide

  22. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  23. Christopher Hertel - @el_stoffel
    Retrieval Augmented Generation
    Semantic Vectors and Similarity Search

    View full-size slide

  24. Christopher Hertel - @el_stoffel
    Token
    • Sequences of characters found in texts
    Sequences of characters found in texts
    • Not necessarily same as words, especially around numbers or emojis
    Not necessarily same as words, especially around numbers or emojis
    • Foundation for GPT to understand semantics – every token has an ID
    Foundation for GPT to understand semantics
    Rule of Thumb
    words x 4/3 = tokens
    75 words = 100 tokens

    View full-size slide

  25. Christopher Hertel - @el_stoffel
    Semantics
    negative
    positive
    professional
    jargon
    s**t
    insufficient
    cool
    excellent
    suboptimal
    lit
    boring
    bad
    fresh
    perfect

    View full-size slide

  26. Christopher Hertel - @el_stoffel
    Semantics
    negative
    positive
    professional
    jargon
    s**t insufficient
    cool
    excellent
    suboptimal
    lit
    boring bad
    fresh
    perfect
    High
    Semantic
    Similarity
    Low
    Semantic
    Similarity
    s**t
    insufficient
    cool
    excellent
    suboptimal
    lit
    boring
    bad
    fresh
    perfect

    View full-size slide

  27. Christopher Hertel - @el_stoffel
    Semantics
    negative
    positive
    professional
    jargon
    s**t insufficient
    cool
    excellent
    suboptimal
    lit
    boring bad
    fresh
    perfect
    High
    Semantic
    Similarity
    Low
    Semantic
    Similarity
    s**t
    insufficient
    cool
    excellent
    suboptimal
    lit
    boring
    bad
    fresh
    perfect
    +1
    -1
    +1
    -1
    [-1.0, -1.0]
    [-0.9, 0.8]
    [0.5, -0,4]
    [1.0, 1.0]
    [-0.1, 0.7]
    [0.8, -0.8]
    [-0.4, -0.4]
    [-0.5, 0.3]
    [0.3, -0.7]
    [0.6, 0.6]

    View full-size slide

  28. Christopher Hertel - @el_stoffel
    Vector
    • Used to express tokens, words, phrases and paragraphs as numerical data
    • Enables arithmetical calculations
    • Distance expresses semantic similarity
    • GPT uses 1536 dimensions per vector
    negative
    positive
    professional
    jargon
    s**t
    insufficient
    cool
    excellent
    suboptimal
    lit
    boring
    bad
    fresh

    View full-size slide

  29. Christopher Hertel - @el_stoffel
    Embeddings
    • Conversion/reduction of words & texts to (single) vector
    • The bigger the text, the “blurrier” the vector
    • Embedding models also provided by OpenAI, e.g. text-embedding-ada-002
    Embeddings Model
    [0.547, -0.425, -0.152, …]
    [0.721, 0.111, -0.924, …]
    [-0.821, 0.289, -0.345, …]
    [0.424, 0.775, 0.522, …]
    [0.124, 0.542, -0.152, …]

    View full-size slide

  30. Christopher Hertel - @el_stoffel

    View full-size slide

  31. Christopher Hertel - @el_stoffel

    View full-size slide

  32. Christopher Hertel - @el_stoffel
    SymfonyConBot
    Example Use Case

    View full-size slide

  33. Christopher Hertel - @el_stoffel
    Use case: SymfonyConBot

    View full-size slide

  34. Christopher Hertel - @el_stoffel
    Retrieval Augmented Generation
    Data Source
    Vector Store
    1 Create Embeddings
    Message Response
    2 Create Embeddings
    3 Similarity Search
    4 Prompt & Generation
    Chain
    Chat Application

    View full-size slide

  35. Christopher Hertel - @el_stoffel
    Retrieval Augmented Generation
    Data Source
    Vector Store
    1 Create Embeddings
    Message Response
    2 Create Embeddings
    3 Similarity Search
    4 Prompt & Generation
    Chat Application

    View full-size slide

  36. Christopher Hertel - @el_stoffel
    Use case: Implementation I

    View full-size slide

  37. Christopher Hertel - @el_stoffel
    Retrieval Augmented Generation
    Data Source
    Vector Store
    1 Create Embeddings
    Message Response
    2 Create Embeddings
    3 Similarity Search
    4 Prompt & Generation
    Chat Application
    TODO: - extended sketch

    View full-size slide

  38. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  39. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  40. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  41. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  42. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  43. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  44. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  45. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  46. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  47. Christopher Hertel - @el_stoffel
    Retrieval Augmented Generation
    • Combine GPT capabilities with domain-specific knowledge
    • Great to reduce hallucinations of GPT
    • Way more flexible than fine-tuning pre-trained models
    • Fine-grained control over data provided to the end user, e.g. user-specific
    • Easily combined with real time data

    View full-size slide

  48. Christopher Hertel - @el_stoffel
    Tools & Function Calling
    Inversion of Control with GPT

    View full-size slide

  49. Christopher Hertel - @el_stoffel
    Tools & Function Calling
    • Equipping GPT with functions
    • GPT delegates the call back to app
    • App comes back to GPT with results
    • GPT generates final response
    GPT
    App
    Chat UI
    Functions
    User Message Chain
    Response
    Prompts Model
    Model
    Response
    Function
    Function Call
    Result

    View full-size slide

  50. Christopher Hertel - @el_stoffel
    Use case: Implementation I

    View full-size slide

  51. Christopher Hertel - @el_stoffel
    Use case: Implementation I

    View full-size slide

  52. Christopher Hertel - @el_stoffel
    Use case: Implementation I

    View full-size slide

  53. Christopher Hertel - @el_stoffel
    Use case: Implementation I

    View full-size slide

  54. Christopher Hertel - @el_stoffel
    Use case: Implementation II

    View full-size slide

  55. Christopher Hertel - @el_stoffel

    View full-size slide

  56. Christopher Hertel - @el_stoffel
    Thank you!

    View full-size slide