Tweet
Tweet

Slide 1

Slide 1 text

Generative AI From Zero to Production: Build your own GenAI solution #WISSENTEILEN powered by Lars Röwekamp | Tim Wüllner | open knowledge GmbH Hands-on Workshop

Slide 2

Slide 2 text

(Architecture, Cloud, AI & ML) Lars Röwekamp LinkedIn: lars-roewekamp CIO New Technologies OPEN KNOWLEDGE

Slide 3

Slide 3 text

Tim Wüllner LinkedIn: tim-wuellner Machine Learning Engineer OPEN KNOWLEDGE

Slide 4

Slide 4 text

What‘s GenAI? How does „Generative AI“ actually work?

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

The Idea: The question and answer are SEMANTICALLY very similar. How does a LLM generate a response?

Slide 7

Slide 7 text

The Idea: The question and answer are SEMANTICALLY very similar. Sematic proximity is represented MATHEMATICALLY by the aid of vectors. How does a LLM generate a response?

Slide 8

Slide 8 text

Dog Cat Tiger Mouse Hamburg Berlin Madrid Amazon Embedding Model 0.6 0.3 0.1 … 0.8 0.5 0.3 … 0.4 0.2 0.9 … Embeddings (semantic vectors)

Slide 9

Slide 9 text

Embedding Model 0.6 0.3 0.1 … 0.8 0.5 0.3 … 0.4 0.2 0.9 … Berlin: City, Europe, Capital of Germany, … Dog Cat Tiger Mouse Hamburg Berlin Madrid Amazon Embeddings (semantic vectors)

Slide 10

Slide 10 text

Dog Cat Mouse Tiger Hamburg Berlin Madrid Amazon Embedding Model 0.6 0.3 0.1 … 0.8 0.5 0.3 … 0.4 0.2 0.9 … Berlin: City, Europe, Capital of Germany, … Dog Cat Tiger Mouse Hamburg Berlin Madrid Amazon

Slide 11

Slide 11 text

Dog Cat Mouse Tiger Hamburg Berlin Madrid Amazon Embedding Model 0.6 0.3 0.1 … 0.8 0.5 0.3 … 0.4 0.2 0.9 … Dog Cat Tiger Mouse Hamburg Berlin Madrid Amazon

Slide 12

Slide 12 text

Dog Cat Mouse Tiger Hamburg Berlin Madrid Amazon Embedding Model 0.6 0.3 0.1 … 0.8 0.5 0.3 … 0.4 0.2 0.9 … Dog Cat Tiger Mouse Hamburg Berlin Madrid Amazon

Slide 13

Slide 13 text

Tiger Hamburg Berlin Madrid Amazon „Can you name a few European capitals?“ Embedding Model 0.8 0.5 0.4 … QUESTION: „Can you name ..?“: City, Europe, Capital of … Dog Cat Mouse

Slide 14

Slide 14 text

Tiger Hamburg Berlin Madrid Amazon „Can you name a few European capitals?“ Embedding Model 0.8 0.5 0.4 … QUESTION: „Can you name ..?“: City, Europe, Capital of … Dog Cat Mouse

Slide 15

Slide 15 text

Tiger Hamburg Berlin Madrid Amazon „Can you name a few European capitals?“ Embedding Model 0.8 0.5 0.4 … QUESTION: „Can you name ..?“: City, Europe, Capital of … Our goal? Dog Cat Mouse

Slide 16

Slide 16 text

Hamburg Berlin Madrid Amazon Our goal? Do everything possible to ensure that the ‚question‘ and the ‚answer‘ are as close together as possible!

Slide 17

Slide 17 text

Source: http://vectors.nlpl.eu/explore/embeddings/en/MOD_enwiki_upos_skipgram_300_2_2021/cat_NOUN/

Slide 18

Slide 18 text

man woman king queen Okay, but what‘s so great about that?

Slide 19

Slide 19 text

man woman king queen queen – woman + man = king Okay, but what‘s so great about that? We can calculate with it and ‚understand‘.

Slide 20

Slide 20 text

man woman king queen doctor – man + woman = ? Okay, but what‘s so great about that? We can calculate with it and ‚understand‘.

Slide 21

Slide 21 text

man woman king queen doctor – man + woman = nurse Okay, but what‘s so great about that? We can calculate with it and ‚understand‘.

Slide 22

Slide 22 text

Generative AI under the hood Transformer (LLM Layer) John wants his bank to cash the … ? word2vec style vectors

Slide 23

Slide 23 text

Generative AI under the hood Transformer (LLM Layer) John wants his bank to cash the … ? John wants his bank to cash the … ? (verb) (verb) context aka hidden state

Slide 24

Slide 24 text

Generative AI under the hood Transformer (LLM Layer) John wants his bank to cash the … ? John wants his bank to cash the … ? (verb) (verb) context aka hidden state Key-Vector: btw I‘am a noun describing a male person. Query-Vector: btw I‘am seeking for a noun describing a male person.

Slide 25

Slide 25 text

Generative AI under the hood Transformer (LLM Layer) John wants his bank to cash the … ? Transformer (LLM Layer) context aka hidden state John wants his bank to cash the … ? (verb) (verb) (John‘s) (finance) John wants his bank to cash the … ? (verb) (verb) (male) ( ... )

Slide 26

Slide 26 text

Generative AI under the hood Transformer (LLM Layer) John wants his bank to cash the … ? John (main character, male, married to Cheryl, cousin of Donald, from Minnesota, currently in Boise, … ) wants his bank to cash the … ?

Slide 27

Slide 27 text

Generative AI under the hood Transformer (LLM Layer) John wants his bank to cash the … ? Transformer (LLM Layer) John wants his bank to cash the … (verb) (verb) (John‘s) (finance) John wants his bank to cash the … ? (verb) (verb) (male) Enough context information to be able to „guess“ the next word. cheque ( ... )

Slide 28

Slide 28 text

Generative AI under the Hood Large Language Model John wants his bank to … Large Language Model Next Token increase cash close … Probability 0.2 0.4 0.1 … Next Token account cheque money order … Probability 0.15 0.75 0.05 … John wants his bank to cash the …

Slide 29

Slide 29 text

GenAI by Example? Where can I realistically use GenAI - and where not?

Slide 30

Slide 30 text

Generative AI by Example

Slide 31

Slide 31 text

Health Care & Pharmaceuticals Advertisment & Marketing Media & Entertainment Production & Manufactoring Financial Services Software Development Generative AI by Example

Slide 32

Slide 32 text

GenAI by Example „Know Your Business!“ *if you want to succeed in the end.

Slide 33

Slide 33 text

GenAI Basics How to build a first, simple GenAI system?

Slide 34

Slide 34 text

GenAI- Model Prompt GenAI Basics „What is the most beautiful holiday destination?“

Slide 35

Slide 35 text

Model A-small Model Provider model-name model-parameter provider-parameter prompt Model A-big Model B e.g. openAI, Google, … Provider Client API GenAI Basics Model Integration „What is the most beautiful holiday destination?“ Reminder: 00-hello-genai.ipynb

Slide 36

Slide 36 text

Hands-On Create a Jupyter Notebook in the Google Colab environment. Calling a GenAI model with a prompt via a provider-specific API. Hello IPYNB*, hello GenAI *Python Notebook aka Jupyter Notebook

Slide 37

Slide 37 text

https://colab.research.google.com/

Slide 38

Slide 38 text

Hands-on Hello Colab https://colab.research.google.com/

Slide 39

Slide 39 text

https://github.com/openknowledge/workshop-genai-student

Slide 40

Slide 40 text

Hands-on Hello GenAI World Jupyter Notebook https://github.com/openknowledge/workshop-genai-student

Slide 41

Slide 41 text

Hands-on Hello GenAI World Jupyter Notebook https://github.com/openknowledge/workshop-genai-student

Slide 42

Slide 42 text

GenAI- Model Prompt GenAI Basics „What is the most beautiful holiday destination?“

Slide 43

Slide 43 text

GenAI- Model Prompt GenAI Basics Model Selection Prompt Engineering 2 1 „What is the most beautiful holiday destination?“

Slide 44

Slide 44 text

GenAI explained MODELS

Slide 45

Slide 45 text

GenAI- Model Prompt GenAI Basics Model Selection Prompt Engineering 2 Model Selection „What is the most beautiful holiday destination?“ Which Model? Which Parameters? Which Infrastructure? Which License? Which Price Model? What Governance? 1

Slide 46

Slide 46 text

GenAI Basics Model Selection „In the world of GenAI, model selection is part of the product design. It affects user experience, cost, scalability, and reliability.“ „While we might not pick the perfect model on the first try, starting with clear priorities like speed, cost, accuracy helps guide the process.“

Slide 47

Slide 47 text

Step 1 Step 2 Step 3 Step 4 GenAI Basics Model Selection aka „How do I find* the right model?“

Slide 48

Slide 48 text

Define specific GenAI use cases Clearly outline the purpose and application of GenAl. 1. Use Case Step 2 Step 3 Step 4 GenAI Basics Model Selection aka „How do I find* the right model?“

Slide 49

Slide 49 text

Define specific GenAI use cases Create a shortlist of foundation models Models that meet the business and technical requirements. 1. Use Case 2. Shortlist Step 3 Step 4 GenAI Basics Model Selection aka „How do I find* the right model?“ Clearly outline the purpose and application of GenAl.

Slide 50

Slide 50 text

Define specific GenAI use cases Create a shortlist of foundation models Evaluate and test selected models Models that meet the business and technical requirements. Conduct thorough testing to evaluate model performance in your own context. 1. Use Case 2. Shortlist 3. Evaluation Step 4 GenAI Basics Model Selection aka „How do I find* the right model?“ Clearly outline the purpose and application of GenAl.

Slide 51

Slide 51 text

Define specific GenAI use cases Create a shortlist of foundation models Evaluate and test selected models Integrate the best models and evaluate them on an ongoing basis Models that meet the business and technical requirements. Conduct thorough testing to evaluate model performance in your own context. Integrate, regularly evaluate and, if necessary, optimise the performance of the model.. 1. Use Case 2. Shortlist 3. Evaluation 4. Integration GenAI Basics Model Selection aka „How do I find* the right model?“ Clearly outline the purpose and application of GenAl.

Slide 52

Slide 52 text

GenAI Basics Model Selection aka „How do I find* the right model?“ „Know Your Use-Case!“ * if you want to succeed in the end.

Slide 53

Slide 53 text

GenAI- Model Prompt GenAI Basics Model Integration Prompt Engineering 2 Model Selection 1 Suitable model found: But how do I address it from my application? „What is the most beautiful holiday destination?“

Slide 54

Slide 54 text

Model A-small Model Provider model-name model-parameter provider-parameter prompt Model A-big Model B e.g. openAI, Google, … Provider Client API GenAI Basics Model Integration „What is the most beautiful holiday destination?“ Reminder: 00-hello-genai.ipynb

Slide 55

Slide 55 text

Model X mini Model Provider II model-name model-parameter provider-parameter prompt Model Y Model Y ++ e.g. openAI, Google, … Provider Client API GenAI Basics Model Integration „What is the most beautiful holiday destination?“

Slide 56

Slide 56 text

Model X mini Model Provider II Provider II Client API model-name model-parameter provider-parameter prompt Model Y Model Y ++ e.g. openAI, Google, … GenAI Basics Model Integration „What is the most beautiful holiday destination?“ The OpenAI API is currently the de facto standard.

Slide 57

Slide 57 text

Hub-Provider Client API model-name model-parameter hub-provider-parameter prompt e.g. Hugging-Face Provider-II hosted Models Provider-I hosted Models Hub-hosted Models* Hub-Provider Dispatcher *Provider X, fine-tuned or adapted, own GenAI Basics Model Integration „What is the most beautiful holiday destination?“ Cloud based Resources

Slide 58

Slide 58 text

Hub-Provider Client API model-name model-parameter hub-provider-parameter prompt e.g. Hugging-Face Provider-II hosted Models Provider-I hosted Models Hub-hosted Models* Hub-Provider Dispatcher *Provider X, fine-tuned or adapted, own GenAI Basics Model Integration „What is the most beautiful holiday destination?“ cloud-not-allowed exception

Slide 59

Slide 59 text

Local-Hub Provider Client API model-name model-parameter hub-provider-parameter prompt e.g. GPT4all, ollama Local hosted Models (Provider III) Local hosted Models (Provider II) Local hosted Models (own or adaptedI) Local-Hub Provider Dispatcher Local Machine GenAI Basics Model Integration „What is the most beautiful holiday destination?“

Slide 60

Slide 60 text

GenAI Basics Model Adaption „ What additional levers do I have?“ • Temperature degree of ‘fantasy‘ • Max Tokens length of answer • Top K selection of hits from the top K hits • Top P selection of hits from the top P percent • Presence Penalty avoid repetition • Frequence Penalty avoid repetition (weighted edition)

Slide 61

Slide 61 text

GenAI explained PROMPTING

Slide 62

Slide 62 text

GenAI- Model Prompt Model Selection Prompt Engineering 2 1 GenAI Basics Prompt Engineering „What is the most beautiful holiday destination?“

Slide 63

Slide 63 text

GenAI- Model Prompt Model Selection Prompt Engineering 1 Parts of a Prompt System vs User Prompt Principles of Prompting Patterns for Prompting Context Window Query vs Chat GenAI Basics Prompt Engineering 2 „What is the most beautiful holiday destination?“

Slide 64

Slide 64 text

GenAI Basics Prompt Engineering Role Instruction Example 1 Context Example n Question Who am I? What is my intention? What are helpful examples? Are there any additional information? BTW: what is the task I ask for? What should a good prompt should look like?

Slide 65

Slide 65 text

GenAI Basics Prompt Engineering Role Instruction Example 1 Context Example n Question What is your role as an assistant? What is my intention? What are helpful examples? Are there any additional information? BTW: what is the task I ask for? What should a good prompt should look like?

Slide 66

Slide 66 text

GenAI Basics Good Prompt vs. Bad Prompt You I want to cook something. LR Topic: Recipe Recommendations

Slide 67

Slide 67 text

GenAI Basics Good Prompt vs. Bad Prompt You I want to cook something. LR You Acting as an expert home cook, for someone who enjoys vegetarian Italian food and has only 30 minutes to prepare dinner, could you recommend a recipe including a list of ingredients and step-by- step instructions? LR Output Example Context Question Role Acting as an expert home cook, for someone who enjoys vegetarian Italian food and has only 30 minutes to prepare dinner, could you recommend a recipe including a list of ingredients and step-by-step instructions? You could point at recipes you like from the BBC’s Good Food guide, providing URL’s to recipes you love. Topic: Recipe Recommendations

Slide 68

Slide 68 text

GenAI Basics Good Prompt vs. Bad Prompt You I want to learn something new. LR You Acting as a coding instructor, for a beginner with a goal to learn Python within 4 weeks, please provide a learning plan including resources and a weekly schedule for 10 hours per week. LR Output Example Context Question Role Acting as a coding instructor, for a beginner with a goal to learn Python within 4 weeks, Please provide a learning plan, including resources and a weekly schedule for 10 hours per week Point it at courses you’ve done in the past that you’ve liked. Tell it you like to learn by reading books, or websites, or by watching videos, or a mixture of both. Topic: Learning new skills

Slide 69

Slide 69 text

GenAI Basics System Prompt vs. User Prompt System Assistant User „You“ Chat-Model Behaviour of Assistant

Slide 70

Slide 70 text

GenAI Basics System Prompt vs. User Prompt System Assistant User ‘You're a friendly assistant.’ „You“ Chat-Model Behaviour of Assistant

Slide 71

Slide 71 text

GenAI Basics System Prompt vs. User Prompt System Assistant User ‘You're a friendly assistant and always keep things brief.’ ‚Which is the most beautiful city in the world?‘ ‚Which is the most beautiful city in the world?‘ ‘You are a philosopher and have a tendency to be verbose.’ ??? ??? „You“ Chat-Model Behaviour of Assistant

Slide 72

Slide 72 text

GenAI Basics System Prompt vs. User Prompt System-Prompt: ‘You're a friendly assistant and always keep things brief.’ Assistant: 65 Tokens

Slide 73

Slide 73 text

GenAI Basics System Prompt vs. User Prompt System-Prompt: ‘You are a philosopher and have a tendency to be verbose.’ Assistant: 434 Tokens

Slide 74

Slide 74 text

GenAI Basics System Prompt vs. User Prompt System-Prompt: Default GPT-4o Assistant: 296 Tokens

Slide 75

Slide 75 text

GenAI Basics Patterns for Prompting Pros & Cons Consideration Stepwise Interaction Flipped Interaction Discussion of Experts I‘m planning to travel to China. I‘m planning a trip to China. I‘m planning a trip to China. I‘m planning a trip to China.

Slide 76

Slide 76 text

GenAI Basics Patterns for Prompting I‘m planning to travel to China. Please analyze the pros and cons of traveling by bike, ship or plane. Pros & Cons Consideration Stepwise Interaction Flipped Interaction Discussion of Experts I‘m planning a trip to China. I‘m planning a trip to China. I‘m planning a trip to China.

Slide 77

Slide 77 text

GenAI Basics Patterns for Prompting I‘m planning to travel to China. Please analyze the pros and cons of traveling by bike, ship or plane. Pros & Cons Consideration Stepwise Interaction Flipped Interaction Discussion of Experts I‘m planning a trip to China. Walk me through the planing process step by step, explaining your reasoning at each stage. Wait for my con- firmation before moving to the next step. I‘m planning a trip to China. I‘m planning a trip to China.

Slide 78

Slide 78 text

GenAI Basics Patterns for Prompting I‘m planning to travel to China. Please analyze the pros and cons of traveling by bike, ship or plane. Pros & Cons Consideration Stepwise Interaction Flipped Interaction Discussion of Experts I‘m planning a trip to China. Walk me through the planing process step by step, explaining your reasoning at each stage. Wait for my con- firmation before moving to the next step. I‘m planning a trip to China. Before providing a solution, please ask me relevant questions about the trip, so that you can give me the most appropriate planing advice.“ I‘m planning a trip to China.

Slide 79

Slide 79 text

GenAI Basics Patterns for Prompting I‘m planning to travel to China. Please analyze the pros and cons of traveling by bike, ship or plane. Pros & Cons Consideration Stepwise Interaction Flipped Interaction Discussion of Experts I‘m planning a trip to China. Walk me through the planing process step by step, explaining your reasoning at each stage. Wait for my con- firmation before moving to the next step. I‘m planning a trip to China. Before providing a solution, please ask me relevant questions about the trip, so that you can give me the most appropriate planing advice.“ I‘m planning a trip to China. I am torn between sustainable and cheap travelling. Can you provide two schools or trends within the travel community that can represent the two approaches.

Slide 80

Slide 80 text

GenAI Basics Prompt Injection https://gandalf.lakera.ai/

Slide 81

Slide 81 text

Hands-On Prompting with a system prompting using various prompting patterns and best practices. Extended prompting via chatbot using a storybook and chat history. Prompting & Models

Slide 82

Slide 82 text

Hands-on Prompting & Models

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

GenAI explained myDOMAIN

Slide 85

Slide 85 text

GenAI- Model Prompt GenAI Basics myDomain „What is the most beautiful holiday destination?“

Slide 86

Slide 86 text

GenAI- Model Prompt GenAI Basics myDomain „How to apply for vacation at open knowledge?“

Slide 87

Slide 87 text

GenAI- Model Prompt GenAI Basics myDomain + „Leave regulation at open knowledge“ „How to apply for vacation at open knowledge?“

Slide 88

Slide 88 text

GenAI- Model Prompt GenAI Basics myDomain + „Leave regulation at open knowledge“ „How to activate the alarm system at open knowledge?“

Slide 89

Slide 89 text

GenAI- Model Prompt GenAI Basics myDomain + + „How to activate the alarm system at open knowledge?“ „Leave regulation + Alarm system usage at open knowledge“

Slide 90

Slide 90 text

GenAI- Model Prompt GenAI Basics myDomain + „All internal knowledge (Wiki, DB, …) at open knowledge“ „How to activate the alarm system at open knowledge?“

Slide 91

Slide 91 text

GenAI- Model Prompt GenAI Basics myDomain + ERROR: Token Limit exceeded Output Size Context Window Size „How to activate the alarm system at open knowledge?“ „All internal knowledge at open knowledge“

Slide 92

Slide 92 text

(Source: https://www.meibel.ai/post/understanding-the-impact-of-increasing-llm-context-windows)

Slide 93

Slide 93 text

GenAI- Model Prompt GenAI Basics myDomain + ERROR: Out-of-Budget Exception Context Window Size „All internal knowledge at open knowledge“ „How to activate the alarm system at open knowledge?“

Slide 94

Slide 94 text

GenAI- Model Prompt GenAI Basics myDomain + ERROR: Lost-in-the-Middle Context Window Size „All internal knowledge at open knowledge“ „How to activate the alarm system at open knowledge?“

Slide 95

Slide 95 text

GenAI- Model Prompt GenAI Basics myDomain + ERROR: Self-fullfilling Prophecy Context Window Size Context Window Size Context Window Size Context Window Size „Alles interne Wissen (Wiki, DB, …) bei open knowledge“ „How to activate the alarm system at open knowledge?“

Slide 96

Slide 96 text

GenAI Advanced How can I contribute and query my own domain knowledge?

Slide 97

Slide 97 text

GenAI- Model Prompt GenAI Advanced myDomain + ERROR: Token Limit exceeded! Output Size Context Window Size „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 98

Slide 98 text

myGenAI Model Prompt GenAI Advanced myDomain „All internal knowledge at open knowledge“ „How to apply for vacation at open knowledge?“

Slide 99

Slide 99 text

myGenAI Model Prompt GenAI Advanced myDomain Option 1: Build own Model Option 2: Fine-tune existing Model ERROR: Way to expensive ERROR: Way to complex WARNING: Out-of-Sync „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 100

Slide 100 text

Prompt GenAI Advanced myDomain „Prompt Enrichment“ „Model Enrichment“ myGenAI Model + „How to apply for vacation at open knowledge?“

Slide 101

Slide 101 text

Prompt GenAI Advanced myDomain GenAI- Model SOME MAGIC „MAGIC Enrichment“ „How to apply for vacation at open knowledge?“

Slide 102

Slide 102 text

Prompt GenAI- Model GenAI Advanced myDomain SOME MAGIC „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 103

Slide 103 text

Prompt Retrieve Augment Ingesting- Pipeline Knowledge Database Retriever GenAI- Model GenAI Advanced myDomain „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 104

Slide 104 text

„How to apply for vacation at open knowledge?“ Prompt Ingesting- Pipeline Knowledge Database Retriever GenAI Advanced myDomain R etrieval Augmented Generation „All internal knowledge at open knowledge“

Slide 105

Slide 105 text

GenAI explained RAG Systems

Slide 106

Slide 106 text

Prompt Ingesting- Pipeline Knowledge Database Retriever GenAI Advanced RAG Systems R etrieval Augmented Generation „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 107

Slide 107 text

Prompt Ingesting- Pipeline Knowledge Database GenAI Advanced RAG Systems R etrieval Augmented Generation Retriever „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 108

Slide 108 text

GenAI Advanced RAG Systems GenAI Model INGESTING Pipeline (Async.) Chunking Embedding RETRIEVAL Pipeline (Sync.) Top-k Fetching Augmenting UI/UX (Prompt) Query- Embedding „How to apply for vacation at open knowledge?“ Retriever

Slide 109

Slide 109 text

GenAI Advanced RAG Systems GenAI Model INGESTING Pipeline (Async.) Chunking Embedding RETRIEVAL Pipeline (Sync.) Top-k Fetching Augmenting UI/UX Query- Embedding It looks complicated, and it is! Except for ‘Hello World’ examples from the Internet. Retriever

Slide 110

Slide 110 text

Chunking Embedding Ingesting Pipeline by Example Document (Doc. Id:1) „It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype. The added value of this technology is too great and its use in the form of a chat is too intuitive. In 2024, it was expected that the leap from exploratory playing around with large language models [...].“

Slide 111

Slide 111 text

Chunking Embedding Ingesting Pipeline by Example [„ It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype.“] [„ The added value of this technology is too great and its use in the form of a chat is too intuitive.“] Document (Doc. Id:1) „It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype. The added value of this technology is too great and its use in the form of a chat is too intuitive. In 2024, it was expected that the leap from exploratory playing around with large language models [...].“ [ … ]

Slide 112

Slide 112 text

Chunking Embedding Ingesting Pipeline by Example [-0.24, 0.15, …, 0.52] [-0.13, 0.95, …, 0.31] […] [„ It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype.“] [„ The added value of this technology is too great and its use in the form of a chat is too intuitive.“] Document (Doc. Id:1) „It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype. The added value of this technology is too great and its use in the form of a chat is too intuitive. In 2024, it was expected that the leap from exploratory playing around with large language models [...].“ [ … ]

Slide 113

Slide 113 text

Chunking Embedding Ingesting Pipeline by Example Id Doc. Id Embedding 1 1 [-0.24, 0.15, …, 0.52] 2 1 [-0.13, 0.95, …, 0.31] […] […] […] [-0.24, 0.15, …, 0.52] [-0.13, 0.95, …, 0.31] […] [„ It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype.“] [„ The added value of this technology is too great and its use in the form of a chat is too intuitive.“] Document (Doc. Id:1) „It will soon be the third anniversary of the release of ChatGPT. There is still no end in sight to the hype. The added value of this technology is too great and its use in the form of a chat is too intuitive. In 2024, it was expected that the leap from exploratory playing around with large language models [...].“ [ … ]

Slide 114

Slide 114 text

Retrieval Pipeline by Example Embedding Top-k Fetching Augmenting ‘What is the lecture ‘The architecture for language models in practice - Retrieval Augmented Generation’ about?’

Slide 115

Slide 115 text

Retrieval Pipeline by Example Embedding Top-k Fetching Augmenting [0.35, 0.25, …, 0.62] ‘What is the lecture ‘The architecture for language models in practice - Retrieval Augmented Generation’ about?’

Slide 116

Slide 116 text

Retrieval Pipeline by Example Embedding Top-k Fetching Augmenting Embeddings from knowledge base Embeddings Dimension 1 Embed. Dim. 2 Top-k=1 document(s) [0.35, 0.25, …, 0.62] ‘What is the lecture ‘The architecture for language models in practice - Retrieval Augmented Generation’ about?’

Slide 117

Slide 117 text

Retrieval Pipeline by Example Embedding Top-k Fetching Augmenting Embeddings from knowledge base Embeddings Dimension 1 Embed. Dim. 2 Top-k=1 document(s) [0.35, 0.25, …, 0.62] & „ What is the lecture […] based on the following context: ‘An upstream retrieval process to pull in relevant information from our own dynamic database can provide well-founded answers.’ ‘What is the lecture ‘The architecture for language models in practice - Retrieval Augmented Generation’ about?’

Slide 118

Slide 118 text

Sherlock RAG in Action

Slide 119

Slide 119 text

No content

Slide 120

Slide 120 text

No content

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

Hands-On Implementing a simple RAG system using ingestion and retrieval pipelines. Applying the RAG system to questions from your own domain. RAG System Simple Version

Slide 123

Slide 123 text

Hands-on Simple RAG

Slide 124

Slide 124 text

Retrieval Augmented Knowledge Database Retriever Generation Ingesting- Pipeline GenAI Advanced RAG Systems „How to apply for vacation at open knowledge?“

Slide 125

Slide 125 text

Retrieval Augmented Knowledge Database Retriever Generation Ingesting- Pipeline GenAI Advanced RAG Systems „How to apply for vacation at open knowledge?“ „Working like this you doesn't need a holiday. Lazy piece!“

Slide 126

Slide 126 text

GenAI Professional What is still missing for a sustainable and productive operation?

Slide 127

Slide 127 text

Retrieval Augmented Knowledge Database Retriever Generation Ingesting- Pipeline GenAI Professional Guardrails „How to apply for vacation at open knowledge?“ „Working like this you doesn't need a holiday. Lazy piece!“

Slide 128

Slide 128 text

Retrieval Augmented Knowledge Database Retriever Generation Guarding Output Ingesting- Pipeline GenAI Professional Guardrails Output Profanity Check „How to apply for vacation at open knowledge?“ „Working like this you doesn't need a holiday. Lazy piece!“

Slide 129

Slide 129 text

Retrieval Augmented Knowledge Database Retriever Generation Guarding Output Ingesting- Pipeline GenAI Professional Guardrails „Unfortunately, I can't answer that!“ Output Profanity Check „…. Lazy piece! …“ „How to apply for vacation at open knowledge?“ „Working like this you doesn't need a holiday. Lazy piece!“

Slide 130

Slide 130 text

Retrieval Augmented Knowledge Database Retriever Generation Input Output Ingesting- Pipeline GenAI Professional Guardrails Guarding Topic Check „Which Netflix series do you recommend?“

Slide 131

Slide 131 text

Retrieval Augmented Knowledge Database Retriever Generation Input Output Ingesting- Pipeline GenAI Professional Guardrails Guarding Topic Check „Which Netflix series do you recommend?“ „Sorry, but this does not fall within my area of responsibility.“

Slide 132

Slide 132 text

Retrieval Augmented Knowledge Database Retriever Generation Input Output Ingesting- Pipeline GenAI Professional Guardrails Output-Governance: Toxic Speech, Fact-Checking, Ethical-Guidelines, Cross-Validation, … Input-Evaluation: Prompt-Injection, Jailbreak-Attemp, Topic-Filter, PII-Cleansing, Rate-Limiting,… Guarding

Slide 133

Slide 133 text

Sherlock RAG in Action

Slide 134

Slide 134 text

No content

Slide 135

Slide 135 text

Hands-On Create your own guardrails for input and output to prevent toxic language. Implementing a simple Output Guardrail as a fact checker. RAG System Guardrails

Slide 136

Slide 136 text

Hands-on Guardrails to the rescue

Slide 137

Slide 137 text

Hands-on Guardrails to the rescue

Slide 138

Slide 138 text

Retrieval Augmented Knowledge Database Retriever Generation Input Output Ingesting- Pipeline GenAI Professional Guarding What else is missing for a professional and productive system?

Slide 139

Slide 139 text

Retrieval Augmented Knowledge Database Retriever Generation Input Output Ingesting- Pipeline RAG-Monitoring Cost- Monitoring Model- Monitoring Response-Quality- Monitoring GenAI Professional Monitoring Guarding Ragas: https://docs.ragas.io/en/stable/concepts/metrics

Slide 140

Slide 140 text

Retrieval Augmented Generation Retrieved Context Generated Answer User Input GenAI Professional Monitoring

Slide 141

Slide 141 text

Generated Answer Retrieved Context User Input ‚Is the answer supported by the context?‘ ‚Is the context found relevant to the question?‘ ‚Is the answer relevant to the question?‘ GenAI Professional Monitoring: The RAG-Triad

Slide 142

Slide 142 text

Generated Answer Retrieved Context User Input ‚Is the given answer similar to the correct answer?‘ ‚Has the entire context to be found been found?‘ Ground Truth Answer Ground Truth Context ‚Is the answer supported by the context?‘ ‚Is the context found relevant to the question?‘ ‚Is the answer relevant to the question?‘ GenAI Professional Monitoring: The RAG-Triad+

Slide 143

Slide 143 text

Retrieval Augmented Generation GenAI Professional Response Quality Monitoring https://docs.confident-ai.com/docs/metrics-llm-evals G-Eval Deep Acyclic Graph Answer Relevancy Faithfulness Summerization Hallucination Contextual Recall Contextual Relevancy Contextual Precision Retrieved Context Generated Answer User Input

Slide 144

Slide 144 text

Retrieval Augmented Generation GenAI Professional Response Quality Monitoring Faithfulness Is the supported by the ? Retrieved Context Generated Answer User Input

Slide 145

Slide 145 text

Retrieval Augmented Generation GenAI Professional Response Quality Monitoring Does the match the ? Answer Relevancy Retrieved Context Generated Answer User Input

Slide 146

Slide 146 text

GenAI Professional Response Quality Monitoring Step 1: Generate possible questions ( ) from Embeddings Dimension 1 Embed. Dim. 2 Answer Relevancy Does the match the ?

Slide 147

Slide 147 text

GenAI Professional Response Quality Monitoring Step 1: Generate possible questions ( ) from Step 2: Determine the average distance from Embeddings Dimension 1 Embed. Dim. 2 Answer Relevancy Does the match the ?

Slide 148

Slide 148 text

RAG in Action aka Sherlock RAG

Slide 149

Slide 149 text

No content

Slide 150

Slide 150 text

Best Practices What are the typical pitfalls that I will run into - and how do I deal with them?

Slide 151

Slide 151 text

Retrieval Augmented Knowledge Database Retriever Generation Guarding Input Output Ingesting- Pipeline RAG-Monitoring Cost- Monitoring Model- Monitoring Response-Quality- Monitoring GenAI Best Practices Real Life Survival Guide

Slide 152

Slide 152 text

‘The answers sound conclusive, but they are wrong!’ GenAI Best Practices Real Life Survival Guide

Slide 153

Slide 153 text

GenAI Model UI/UX Guardrails Output Ingesting-Pipeline (Async.) Chunking Embedding Retrieval-Pipeline (Sync.) Query- Embedding Top-k Fetching Augmenting Input Hallucination (via Faithfulness / Answer Relevancy) GenAI Best Practices Real Life Survival Guide

Slide 154

Slide 154 text

GenAI Model UI/UX Guardrails Output Ingesting-Pipeline (Async.) Chunking Embedding Retrieval-Pipeline (Sync.) Query- Embedding Top-k Fetching Augmenting Input Missing Data GenAI Best Practices Real Life Survival Guide

Slide 155

Slide 155 text

GenAI Model UI/UX Guardrails Output Ingesting-Pipeline (Async.) Chunking Embedding Retrieval-Pipeline (Sync.) Query- Embedding Top-k Fetching Augmenting Input Missing Data GenAI Best Practices Real Life Survival Guide Chunk not found

Slide 156

Slide 156 text

GenAI Model UI/UX Guardrails Output Ingesting-Pipeline (Async.) Chunking Embedding Retrieval-Pipeline (Sync.) Query- Embedding Top-k Fetching Augmenting Input Missing Data GenAI Best Practices Real Life Survival Guide Chunk not found Chunk not relevant

Slide 157

Slide 157 text

Ingesting-Pipeline (Async.) Chunking Embedding Real Life RAG #1 Missing-Data Challenge Missing Data

Slide 158

Slide 158 text

Ingesting-Pipeline (Async.) Chunking Embedding Real Life RAG #1 Missing-Data Challenge Missing Data Problem: RAG does not know the required data.

Slide 159

Slide 159 text

Ingesting-Pipeline (Async.) Chunking Embedding Real Life RAG #1 Missing-Data Challenge Missing Data Problem: RAG does not know the required data. Solution : Implement a suitable ingestor.

Slide 160

Slide 160 text

Ingesting-Pipeline (Async.) Chunking Embedding Real Life RAG #1 Missing-Data Challenge Missing Data Problem: Snapshot of the data is not up-to-date.

Slide 161

Slide 161 text

Ingesting-Pipeline (Async.) Chunking Embedding Real Life RAG #1 Missing-Data Challenge Missing Data Problem: Snapshot of the data is not up-to-date. Solution: Interface for ‘continuous’ data sync.

Slide 162

Slide 162 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found

Slide 163

Slide 163 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found Problem: Existing chunks will not be found.

Slide 164

Slide 164 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found Problem: Existing chunks will not be found. Solution #1: Adapt chunk-sizes and overlapping.

Slide 165

Slide 165 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found Problem: Existing chunks will not be found. Solution #2: Choose a different splitter (e.g. Semantic-Splitter).

Slide 166

Slide 166 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found Problem: Existing chunks will not be found. Solution #3: Optimize the distance threshold or k from top-k.

Slide 167

Slide 167 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found Problem: Existing chunks will not be found. Solution #4: Selection of a more suitable embedding model.

Slide 168

Slide 168 text

Query- Embedding Top-k Fetching Ingesting-Pipeline (Async.) Chunking Embedding Augmenting Retrieval-Pipeline (Sync.) Real Life RAG #2 Chunk-not-found Challenge Chunk not found Problem: Existing chunks will not be found. Solution #5: Hypothetical document embeddings (HyDE) Input HyDE

Slide 169

Slide 169 text

Guardrails Input Output Gao et al. (2023) HyDE = Hypothetical Document Embeddings Real Life RAG #2 Chunk-not-found Challenge HyDE

Slide 170

Slide 170 text

Real Life RAG #3 Chunk-not-relevant Challenge Chunk not relevant GenAI Model Query- Embedding Top-k Fetching Augmenting Retrieval-Pipeline (Sync.)

Slide 171

Slide 171 text

Real Life RAG #3 Chunk-not-relevant Challenge Problem: Chunk is not considered to be relevant by the model. Chunk not relevant GenAI Model Query- Embedding Top-k Fetching Augmenting Retrieval-Pipeline (Sync.)

Slide 172

Slide 172 text

Real Life RAG #3 Chunk-not-relevant Challenge Chunk not relevant Query- Embedding Top-k Fetching Augmenting Retrieval-Pipeline (Sync.) Problem: Chunk is not considered to be relevant by the model. Solution #1: Select a more suitable model for the use case. GenAI Model

Slide 173

Slide 173 text

Real Life RAG #3 Chunk-not-relevant Challenge Chunk not relevant Query- Embedding Top-k Fetching Augmenting Retrieval-Pipeline (Sync.) GenAI Model Problem: Chunk is not considered to be relevant by the model. Solution #2: Add additional context to the chunk (small-to-big). Chunk

Slide 174

Slide 174 text

GenAI Model UI/UX Guardrails Input Output Ingesting-Pipeline (Async.) Chunking Embedding Retrieval-Pipeline (Sync.) Query- Embedding Top-k Fetching Augmenting

Slide 175

Slide 175 text

Hands-On Surviving the RAG jungle: Using LLM and RAG metrics to optimise the retrieval process. RAG System typical Pitfalls

Slide 176

Slide 176 text

Hands-on Surviving the RAG jungle

Slide 177

Slide 177 text

Generated Answer Retrieved Context User Input ‚Is the given answer similar to the correct answer?‘ ‚Has the entire context to be found been found?‘ Ground Truth Answer Ground Truth Context ‚Is the answer supported by the context?‘ ‚Is the context found relevant to the question?‘ ‚Is the answer relevant to the question?‘ GenAI Professional Monitoring: The RAG-Triad+

Slide 178

Slide 178 text

1 Provide evaluation data that includes the ground truth context. 2 Generate responses to evaluation input and collect context. Real Life RAG Metrics in Information Retrieval

Slide 179

Slide 179 text

1 2 Generate responses to evaluation input and collect context. 3 Determine whether context is included in ground truth. 1 0 Hit Rate Real Life RAG Metrics in Information Retrieval – Hit Rate Provide evaluation data that includes the ground truth context.

Slide 180

Slide 180 text

1 2 Generate responses to evaluation input and collect context based on relevance. 3 Determine the proportion of relevant documents in relation to all documents. 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑟𝑒𝑙𝑒𝑣𝑎𝑛𝑡 𝑅𝑒𝑠𝑢𝑙𝑡𝑠 𝑡𝑜𝑡𝑎𝑙 𝑅𝑒𝑠𝑢𝑙𝑡𝑠 0.00 0.33 Real Life RAG Metrics in Information Retrieval – Precision Provide evaluation data that includes the ground truth context.

Slide 181

Slide 181 text

1 2 Generate responses to evaluation input and collect context based on relevance. … 3 Determine the rank of the first relevant document. Calculate and average the reciprocal rank (RR): 𝑅𝑅 = 1 𝑅𝑎𝑛𝑔 1 2 3 1 2 3 1 2 0 𝑀𝑅𝑅 = 1 𝑛 σ𝑖=1 𝑛 𝑅𝑅𝑖 𝑀𝑅𝑅 = 0.25 1 2 3 1 2 3 Real Life RAG Metrics in Information Retrieval – Mean Reciprocal Rank (MRR) Provide evaluation data that includes the ground truth context.

Slide 182

Slide 182 text

Outlook What else is possible in addition to the pure knowledge query?

Slide 183

Slide 183 text

Prompt Retrieve Augment Ingesting- Pipeline Knowledge Database Retriever GenAI- Model GenAI Outlook What else is there to consider? „All internal knowledge at open knowledge“ „How to apply for vacation at open knowledge?“

Slide 184

Slide 184 text

GenAI explained I18N

Slide 185

Slide 185 text

UI/UX Enterprise Integration „Comment demander des vacances chez ok?“ Retrieve Augment Ingesting- Pipeline Retriever GenAI- Model GenAI Outlook Multi-Language RAG Knowledge Database „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 186

Slide 186 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Retriever GenAI- Model Language- Detection Translation Translation Translation GenAI Outlook Multi-Language RAG Knowledge Database „All internal knowledge at open knowledge“ „Comment demander des vacances chez ok?“

Slide 187

Slide 187 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Knowledge Database Retriever GenAI- Model Translation Translation Translation Translation FR_fr language not supported domain specific wording GenAI Outlook Multi-Language RAG Language- Detection „Comment demander des vacances chez ok?“ „All internal knowledge at open knowledge in multiple languages.“

Slide 188

Slide 188 text

GenAI explained ACCESS CONTROL

Slide 189

Slide 189 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Retriever GenAI- Model Knowledge Database GenAI Outlook Access Control „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 190

Slide 190 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Retriever GenAI- Model Knowledge Database GenAI Outlook Access Control „What are the top secret KPIs of open knowledge?“ „All internal knowledge at open knowledge“ incl C-Level information.“

Slide 191

Slide 191 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Retriever GenAI- Model Token Knowledge Database Input Guardrail Some kind of Attack? Prompt-Injection? Prompt-Abuse? Rate-Limit? GenAI Outlook Access Control „What are the top secret KPIs of open knowledge?“ „All internal knowledge at open knowledge“ incl C-Level information.“

Slide 192

Slide 192 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Knowledge Database with AC Meta Data Retriever GenAI- Model RBAC via PBF Token Token Input Guardrail GenAI Outlook Access Control RBAC = Role based Access Control PBF = Permission based Filtering „What are the top secret KPIs of open knowledge?“ „All internal knowledge at open knowledge“ incl C-Level information.“

Slide 193

Slide 193 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Knowledge Database with AC Meta Data Retriever GenAI- Model RBAC via PBF Token Token Input Guardrail Output Guardrail PII: Personally identifiable information? GenAI Outlook Access Control „What does the personal data of X look like at ok?“ „All internal knowledge at open knowledge“ incl C-Level information.“

Slide 194

Slide 194 text

GenAI explained INTEGRATION

Slide 195

Slide 195 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Retriever GenAI- Model Knowledge Database GenAI Outlook Integration „All internal knowledge at open knowledge“ „How to apply for vacation at open knowledge?“

Slide 196

Slide 196 text

GenAI Outlook Integration GenAI based Service A P I ML as a Service Input Output UI/UX Enterprise Integration „All internal knowledge at open knowledge“ „How to apply for vacation at open knowledge?“

Slide 197

Slide 197 text

Hands-On Provision of a GenAI-based RAG system as a service. Integration of the service into a proprietary web-based interface. RAG System GenAI as a Service

Slide 198

Slide 198 text

Hands-on GenAI-as-a-Service

Slide 199

Slide 199 text

Hands-on GenAI-as-a-Service

Slide 200

Slide 200 text

GenAI explained AGENTS

Slide 201

Slide 201 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Knowledge Database Retriever GenAI- Model GenAI Outlook Multi-Agenten Systeme „How to apply for vacation at open knowledge?“ „All internal knowledge at open knowledge“

Slide 202

Slide 202 text

UI/UX Enterprise Integration Retrieve Augment Ingesting- Pipeline Knowledge Database Retriever GenAI- Model GenAI Outlook Multi-Agenten Systeme „All internal knowledge at open knowledge“ ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 203

Slide 203 text

Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX Agentic AI ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 204

Slide 204 text

Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX ‚You are an assistant for employees. Perform the following steps to complete the tasks: [...]‘ Agentic AI ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 205

Slide 205 text

Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX I should first retrieve past interactions with the user and user information. Agentic AI ‚You are an assistant for employees. Perform the following steps to complete the tasks: [...]‘ ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 206

Slide 206 text

Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX User : {Name: „Tim Wüllner“, ID: „123“} […] Agentic AI ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 207

Slide 207 text

Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX User : {Name: „Tim Wüllner“, ID: „123“} […] I should now obtain information on how to apply for leave at open knowledge. Agentic AI ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 208

Slide 208 text

Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX ‚You can apply for holiday leave at open knowledge via the BCS system. Leave is applied for as follows […]. Agentic AI ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 209

Slide 209 text

Agentic AI Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX I should now use the proposed tool(s) to apply for the holiday for the user as described. ‚You can apply for holiday leave at open knowledge via the BCS system. Leave is applied for as follows […]. ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 210

Slide 210 text

Agentic AI Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX requestVacation(userId=„123“,from=„x“,to=„y“) ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 211

Slide 211 text

Agentic AI Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX requestVacation(userId=„123“,from=„x“,to=„y“) I should now inform the user about the holiday request. ‚Apply for my vacation from […] to [… ] at ok, please.‘

Slide 212

Slide 212 text

Agentic AI Retriever Memory Tools Planning Database Prompting GenAI- Model API-Call UI/UX ‚Apply for my vacation from […] to [… ] at ok, please.‘ „I applied for … .“

Slide 213

Slide 213 text

UI/UX Expert on holdiays Expert on User- Interaction Expert on coding Expert on Social Media ‚Create a proposal for a contribution on the topic of AI agents.‘ ‚Why does this code result in a NullPointerException ?‘ Agentic AI Multi Agent Systems

Slide 214

Slide 214 text

Conclusion What should I take away from the workshop for myself and my team?

Slide 215

Slide 215 text

GenAI Model UI/UX Guardrails Input Output Ingesting-Pipeline (Async.) Chunking Embedding Retrieval-Pipeline (Sync.) Top-k Fetching Augmenting Query- Embedding

Slide 216

Slide 216 text

GenAI from prototype to production Your personal „takeaways“: • GenAI is powerful. But also expensive. • Professional prompting is your super power. • Every model has its own character. • RAG for your own domain knowledge. • GenAI is also just software. Your use case determines the right path!

Slide 217

Slide 217 text

Time for Questions? Yes, sure!

Slide 218

Slide 218 text

#WISSENTEILEN Many Thanks! #WISSENTEILEN by open knowledge GmbH @_openKnowledge | @mobileLarson Lars Röwekamp, CIO New Technologies

Slide 219

Slide 219 text

Looking beyond the RAG horizon: Cognitive Architecture blog.langchain.dev/what-is-a-cognitive-architecture Wir sind hier

Slide 220

Slide 220 text

#WISSENTEILEN #WISSENTEILEN IMAGE REFERENCE Folie 21: © photoplotnikov - istockphoto.com Folie 23: © Mix und Match Studios - shutterstock.com Folie 23: © Mix und Match Studios - shutterstock.com All other pictures, drawings and icons originate from • pexels.com, • pixabay.com, • unsplash.com, • flaticon.com or are created by my own.