How to use GPT with your Symfony application [EN]

Slide 1

Slide 1 text

Christopher Hertel - @el_stoffel & GPT How to use Symfony with your application

Slide 2

Slide 2 text

Christopher Hertel - @el_stoffel Who is that guy? Christopher Hertel Software Architect in Berlin @el_stoffel

Slide 3

Slide 3 text

Christopher Hertel - @el_stoffel What should I talk about? Alright, Christopher, that's easy: 1. LLM & GPT Start with the fundamentals of Large Language Models and GPT. 2. GPT Fundamentals Bring in fundamental concepts of GPT itself - best with a simple example. 3. Retrieval Augmented Generation Explain the basic concept of RAG and bring in a real-world example. 4. Function Calling Walk them through function calling - and if you have time, bring an example. 5. Q & A Well, let's see if they're still awake. SymfonyCon Brussels 2023

Slide 4

Slide 4 text

Christopher Hertel - @el_stoffel LLM & GPT ChatGPT and beyond

Slide 5

Slide 5 text

Christopher Hertel - @el_stoffel LLM & GPT • Model • Result of training an AI algorithm with data, e.g. neural network • LLM == Large Language Models • Specific model to understand and generate language • Trained on a large amount of data, e.g. the internet • Basic idea is completion on top of probabilities • GPT == Generative Pre-Trained Transformer • First released by OpenAI • Generates language, word by word • Pre-trained on large data-sets • Transformer is a specific LLM architecture ChatGPT => ShopMVC

Slide 6

Slide 6 text

Christopher Hertel - @el_stoffel Examples • Models • Open AI: gpt-3.5-turbo or gpt-4 • Meta: Llama 2 or CodeLlama • Anthropic: Claude • OSS: Falcon, GPT-J or GPT4All • BYO: Stanford Alpaca • Runtime • OpenAI & Microsoft Azure • huggingface.co • replicate.com • Self-hosted: GPT-J or GPT4All ChatGPT … • is not a model • is a product, using gpt-3.5 or gpt-4 • is for free for a reason

Slide 7

Slide 7 text

Christopher Hertel - @el_stoffel GPT Fundamentals Prompts, Temperature and more

Slide 8

Slide 8 text

Christopher Hertel - @el_stoffel Prompts • Messages that interact with the model • Style of prompts really important => “Prompt Engineering” • A lot of “trial & error” • Roles • User Messages created by a user • Assistant Messages created by a system • System High level instructions for the conversation

Slide 9

Slide 9 text

Christopher Hertel - @el_stoffel System Prompt The In Action

Slide 10

Slide 10 text

Christopher Hertel - @el_stoffel You are ChatGPT, a large language model trained by OpenAI. You respond with only one sentence. System What should I do tomorrow? User That depends on your personal goals and priorities. Assistant

Slide 11

Slide 11 text

Christopher Hertel - @el_stoffel What should I do tomorrow? User Do what makes you happy and fulfilled. Assistant You are my parent and always want "the best" for me. You respond only with one sentence. System

Slide 12

Slide 12 text

Christopher Hertel - @el_stoffel What should I do tomorrow? User Get up and go for a run or try a new sport! Assistant You are a sports trainer that helps people to overcome their hesitation to do some sports. You respond with only one sentence. System

Slide 13

Slide 13 text

Christopher Hertel - @el_stoffel What should I do tomorrow? User Make a to-do list and prioritize your tasks based on their importance and urgency. Assistant You are the assistant of Fabien Potencier, who is the creator of the PHP framework Symfony, and CPO of Platform.sh. You respond with only one sentence. System

Slide 14

Slide 14 text

Christopher Hertel - @el_stoffel What should I do tomorrow? User Invest your time in something that will generate revenue. Assistant You are a capitalism fan boy and value money over people. You respond only with one sentence. System

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Christopher Hertel - @el_stoffel Context • Chat history => sum of messages • Provided all over again • The model interaction is stateless • state only training and fine-tuning • Limited by the number of “tokens” • gpt-3.5-turbo supports 16.385 tokens • gpt-4 supports 8.192 tokens • gpt-4-32k supports 32.768 tokens • gpt-4-turbo supports 128.000 tokens User Assistant System User Assistant User Assistant Context

Slide 17

Slide 17 text

Christopher Hertel - @el_stoffel Context Payload will grow and grow …

Slide 18

Slide 18 text

Christopher Hertel - @el_stoffel Temperature 0.0 2.0 creative deterministic • Mechanism how GPT handles probabilities • Often referred to as “randomness” More Options max tokens, n [of choices], penalty of presence or frequency, format, seed, top_p and tools

Slide 19

Slide 19 text

Christopher Hertel - @el_stoffel Usage with Symfony • Architecture constraints • GPT is a component to understand and generate language • By default used via Web API • Libraries • HTTP clients like symfony/http-client or guzzlehttp/guzzle are sufficient • API client libraries like • openai-php/client with Symfony integration openai-php/symfony • orhanerday/open-ai that claims to be better • LLM Framework packages • Package kambo/langchain based on Langchain • Package theodo-group/llphant based on Langchain and LlamaIndex

Slide 20

Slide 20 text

Christopher Hertel - @el_stoffel decomplex.me Example Use Case

Slide 21

Slide 21 text

Christopher Hertel - @el_stoffel Use case: Implementation I • Install openai-php/symfony with Composer • Configure OpenAI token in environment variable $ composer require openai-php/symfony ###> openai-php/symfony ### # https://platform.openai.com/account/api-keys OPENAI_API_KEY=sk-rWYtxvD1dYouR34LlyTH1nkth1sI5real? OPENAI_ORGANIZATION= ###< openai-php/symfony ###

Slide 22

Slide 22 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 23

Slide 23 text

Christopher Hertel - @el_stoffel Retrieval Augmented Generation Semantic Vectors and Similarity Search

Slide 24

Slide 24 text

Christopher Hertel - @el_stoffel Token • Sequences of characters found in texts Sequences of characters found in texts • Not necessarily same as words, especially around numbers or emojis Not necessarily same as words, especially around numbers or emojis • Foundation for GPT to understand semantics – every token has an ID Foundation for GPT to understand semantics Rule of Thumb words x 4/3 = tokens 75 words = 100 tokens

Slide 25

Slide 25 text

Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh perfect

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh perfect High Semantic Similarity Low Semantic Similarity s**t insufficient cool excellent suboptimal lit boring bad fresh perfect +1 -1 +1 -1 [-1.0, -1.0] [-0.9, 0.8] [0.5, -0,4] [1.0, 1.0] [-0.1, 0.7] [0.8, -0.8] [-0.4, -0.4] [-0.5, 0.3] [0.3, -0.7] [0.6, 0.6]

Slide 28

Slide 28 text

Christopher Hertel - @el_stoffel Vector • Used to express tokens, words, phrases and paragraphs as numerical data • Enables arithmetical calculations • Distance expresses semantic similarity • GPT uses 1536 dimensions per vector negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh

Slide 29

Slide 29 text

Christopher Hertel - @el_stoffel Embeddings • Conversion/reduction of words & texts to (single) vector • The bigger the text, the “blurrier” the vector • Embedding models also provided by OpenAI, e.g. text-embedding-ada-002 Embeddings Model [0.547, -0.425, -0.152, …] [0.721, 0.111, -0.924, …] [-0.821, 0.289, -0.345, …] [0.424, 0.775, 0.522, …] [0.124, 0.542, -0.152, …]

Slide 30

Slide 30 text

Christopher Hertel - @el_stoffel

Slide 31

Slide 31 text

Christopher Hertel - @el_stoffel

Slide 32

Slide 32 text

Christopher Hertel - @el_stoffel SymfonyConBot Example Use Case

Slide 33

Slide 33 text

Christopher Hertel - @el_stoffel Use case: SymfonyConBot

Slide 34

Slide 34 text

Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chain Chat Application

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Christopher Hertel - @el_stoffel Use case: Implementation I

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 39

Slide 39 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 40

Slide 40 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 41

Slide 41 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 42

Slide 42 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 43

Slide 43 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 44

Slide 44 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 45

Slide 45 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 46

Slide 46 text

Christopher Hertel - @el_stoffel Use case: Implementation II

Slide 47

Slide 47 text

Christopher Hertel - @el_stoffel Retrieval Augmented Generation • Combine GPT capabilities with domain-specific knowledge • Great to reduce hallucinations of GPT • Way more flexible than fine-tuning pre-trained models • Fine-grained control over data provided to the end user, e.g. user-specific • Easily combined with real time data

Slide 48

Slide 48 text

Christopher Hertel - @el_stoffel Tools & Function Calling Inversion of Control with GPT

Slide 49

Slide 49 text

Christopher Hertel - @el_stoffel Tools & Function Calling • Equipping GPT with functions • GPT delegates the call back to app • App comes back to GPT with results • GPT generates final response GPT App Chat UI Functions User Message Chain Response Prompts Model Model Response Function Function Call Result