How to use GPT with your Symfony application [EN]

With the release of ChatGPT, OpenAI has generated a lot of buzz around GPT and AI in general. For software architects and developers, this is not only a game changer when it comes to learning or debugging, but also introduces a new tool to embed into your system's architecture. In this talk, we will take a closer look at the basic concepts of GPT, the relevant models, and how you can effectively utilize them in your Symfony application.

Example Use Cases:
* https://github.com/chr-hertel/symfonycon-bot-2023
* https://github.com/chr-hertel/decomplex

Transcript

1. Christopher Hertel - @el_stoffel & GPT How to use Symfony with your application
2. Christopher Hertel - @el_stoffel Who is that guy? Christopher Hertel Software Architect in Berlin @el_stoffel
3. Christopher Hertel - @el_stoffel What should I talk about? Alright, Christopher, that's easy: 1. LLM & GPT Start with the fundamentals of Large Language Models and GPT. 2. GPT Fundamentals Bring in fundamental concepts of GPT itself - best with a simple example. 3. Retrieval Augmented Generation Explain the basic concept of RAG and bring in a real-world example. 4. Function Calling Walk them through function calling - and if you have time, bring an example. 5. Q & A Well, let's see if they're still awake. SymfonyCon Brussels 2023
4. Christopher Hertel - @el_stoffel LLM & GPT ChatGPT and beyond
5. Christopher Hertel - @el_stoffel LLM & GPT • Model • Result of training an AI algorithm with data, e.g. neural network • LLM == Large Language Models • Specific model to understand and generate language • Trained on a large amount of data, e.g. the internet • Basic idea is completion on top of probabilities • GPT == Generative Pre-Trained Transformer • First released by OpenAI • Generates language, word by word • Pre-trained on large data-sets • Transformer is a specific LLM architecture ChatGPT => ShopMVC
6. Christopher Hertel - @el_stoffel Examples • Models • Open AI: gpt-3.5-turbo or gpt-4 • Meta: Llama 2 or CodeLlama • Anthropic: Claude • OSS: Falcon, GPT-J or GPT4All • BYO: Stanford Alpaca • Runtime • OpenAI & Microsoft Azure • huggingface.co • replicate.com • Self-hosted: GPT-J or GPT4All ChatGPT … • is not a model • is a product, using gpt-3.5 or gpt-4 • is for free for a reason
7. Christopher Hertel - @el_stoffel GPT Fundamentals Prompts, Temperature and more
8. Christopher Hertel - @el_stoffel Prompts • Messages that interact with the model • Style of prompts really important => “Prompt Engineering” • A lot of “trial & error” • Roles • User Messages created by a user • Assistant Messages created by a system • System High level instructions for the conversation
9. Christopher Hertel - @el_stoffel System Prompt The In Action
10. Christopher Hertel - @el_stoffel You are ChatGPT, a large language model trained by OpenAI. You respond with only one sentence. System What should I do tomorrow? User That depends on your personal goals and priorities. Assistant
11. Christopher Hertel - @el_stoffel What should I do tomorrow? User Do what makes you happy and fulfilled. Assistant You are my parent and always want "the best" for me. You respond only with one sentence. System
12. Christopher Hertel - @el_stoffel What should I do tomorrow? User Get up and go for a run or try a new sport! Assistant You are a sports trainer that helps people to overcome their hesitation to do some sports. You respond with only one sentence. System
13. Christopher Hertel - @el_stoffel What should I do tomorrow? User Make a to-do list and prioritize your tasks based on their importance and urgency. Assistant You are the assistant of Fabien Potencier, who is the creator of the PHP framework Symfony, and CPO of Platform.sh. You respond with only one sentence. System
14. Christopher Hertel - @el_stoffel What should I do tomorrow? User Invest your time in something that will generate revenue. Assistant You are a capitalism fan boy and value money over people. You respond only with one sentence. System
15. Christopher Hertel - @el_stoffel What should I do tomorrow? User Invest your time in something that will generate revenue. Assistant You are a capitalism fan boy and value money over people. You respond only with one sentence. System
16. Christopher Hertel - @el_stoffel Context • Chat history => sum of messages • Provided all over again • The model interaction is stateless • state only training and fine-tuning • Limited by the number of “tokens” • gpt-3.5-turbo supports 16.385 tokens • gpt-4 supports 8.192 tokens • gpt-4-32k supports 32.768 tokens • gpt-4-turbo supports 128.000 tokens User Assistant System User Assistant User Assistant Context
17. Christopher Hertel - @el_stoffel Context Payload will grow and grow …
18. Christopher Hertel - @el_stoffel Temperature 0.0 2.0 creative deterministic • Mechanism how GPT handles probabilities • Often referred to as “randomness” More Options max tokens, n [of choices], penalty of presence or frequency, format, seed, top_p and tools
19. Christopher Hertel - @el_stoffel Usage with Symfony • Architecture constraints • GPT is a component to understand and generate language • By default used via Web API • Libraries • HTTP clients like symfony/http-client or guzzlehttp/guzzle are sufficient • API client libraries like • openai-php/client with Symfony integration openai-php/symfony • orhanerday/open-ai that claims to be better • LLM Framework packages • Package kambo/langchain based on Langchain • Package theodo-group/llphant based on Langchain and LlamaIndex
20. Christopher Hertel - @el_stoffel decomplex.me Example Use Case
21. Christopher Hertel - @el_stoffel Use case: Implementation I • Install openai-php/symfony with Composer • Configure OpenAI token in environment variable $ composer require openai-php/symfony ###> openai-php/symfony ### # https://platform.openai.com/account/api-keys OPENAI_API_KEY=sk-rWYtxvD1dYouR34LlyTH1nkth1sI5real? OPENAI_ORGANIZATION= ###< openai-php/symfony ###
22. Christopher Hertel - @el_stoffel Use case: Implementation II
23. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Semantic Vectors and Similarity Search
24. Christopher Hertel - @el_stoffel Token • Sequences of characters found in texts Sequences of characters found in texts • Not necessarily same as words, especially around numbers or emojis Not necessarily same as words, especially around numbers or emojis • Foundation for GPT to understand semantics – every token has an ID Foundation for GPT to understand semantics Rule of Thumb words x 4/3 = tokens 75 words = 100 tokens
25. Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh perfect
26. Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh perfect High Semantic Similarity Low Semantic Similarity s**t insufficient cool excellent suboptimal lit boring bad fresh perfect
27. Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh perfect High Semantic Similarity Low Semantic Similarity s**t insufficient cool excellent suboptimal lit boring bad fresh perfect +1 -1 +1 -1 [-1.0, -1.0] [-0.9, 0.8] [0.5, -0,4] [1.0, 1.0] [-0.1, 0.7] [0.8, -0.8] [-0.4, -0.4] [-0.5, 0.3] [0.3, -0.7] [0.6, 0.6]
28. Christopher Hertel - @el_stoffel Vector • Used to express tokens, words, phrases and paragraphs as numerical data • Enables arithmetical calculations • Distance expresses semantic similarity • GPT uses 1536 dimensions per vector negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh
29. Christopher Hertel - @el_stoffel Embeddings • Conversion/reduction of words & texts to (single) vector • The bigger the text, the “blurrier” the vector • Embedding models also provided by OpenAI, e.g. text-embedding-ada-002 Embeddings Model [0.547, -0.425, -0.152, …] [0.721, 0.111, -0.924, …] [-0.821, 0.289, -0.345, …] [0.424, 0.775, 0.522, …] [0.124, 0.542, -0.152, …]
30. Christopher Hertel - @el_stoffel
31. Christopher Hertel - @el_stoffel
32. Christopher Hertel - @el_stoffel SymfonyConBot Example Use Case
33. Christopher Hertel - @el_stoffel Use case: SymfonyConBot
34. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chain Chat Application
35. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chat Application
36. Christopher Hertel - @el_stoffel Use case: Implementation I
37. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chat Application TODO: - extended sketch
38. Christopher Hertel - @el_stoffel Use case: Implementation II
39. Christopher Hertel - @el_stoffel Use case: Implementation II
40. Christopher Hertel - @el_stoffel Use case: Implementation II
41. Christopher Hertel - @el_stoffel Use case: Implementation II
42. Christopher Hertel - @el_stoffel Use case: Implementation II
43. Christopher Hertel - @el_stoffel Use case: Implementation II
44. Christopher Hertel - @el_stoffel Use case: Implementation II
45. Christopher Hertel - @el_stoffel Use case: Implementation II
46. Christopher Hertel - @el_stoffel Use case: Implementation II
47. Christopher Hertel - @el_stoffel Retrieval Augmented Generation • Combine GPT capabilities with domain-specific knowledge • Great to reduce hallucinations of GPT • Way more flexible than fine-tuning pre-trained models • Fine-grained control over data provided to the end user, e.g. user-specific • Easily combined with real time data
48. Christopher Hertel - @el_stoffel Tools & Function Calling Inversion of Control with GPT
49. Christopher Hertel - @el_stoffel Tools & Function Calling • Equipping GPT with functions • GPT delegates the call back to app • App comes back to GPT with results • GPT generates final response GPT App Chat UI Functions User Message Chain Response Prompts Model Model Response Function Function Call Result
50. Christopher Hertel - @el_stoffel Use case: Implementation I
51. Christopher Hertel - @el_stoffel Use case: Implementation I
52. Christopher Hertel - @el_stoffel Use case: Implementation I
53. Christopher Hertel - @el_stoffel Use case: Implementation I
54. Christopher Hertel - @el_stoffel Use case: Implementation II
55. Christopher Hertel - @el_stoffel
56. Christopher Hertel - @el_stoffel Thank you!