RAG for Small LLM - Speaker Deck

Slide 1

Slide 1 text

Advanced RAG for Small LLM

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

llama.cpp Phi 2 Pico Jarvis

Slide 4

Slide 4 text

github.com/ariya/pico-jarvis

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Zero-Shot Prompt

Slide 7

Slide 7 text

Zero-Shot Prompt

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

You are a helpful assistant. Answer verbosely. user assistant system

Slide 10

Slide 10 text

You are a helpful assistant. Answer concisely.

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Chain of Thought

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

You are a research assistant with access to Google search. Given the question from the user, your task is to use Google to search for the answer. Think step by step.

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

JSON Mode

Slide 17

Slide 17 text

JSON Mode

Slide 18

Slide 18 text

You are a research assistant with access to Google search. Given the question from the user, your task is to use Google to search for the answer. Think step by step. Always output a valid JSON as follows: { "thought": describe your thoughts about the question, "tool": the search engine to use (must be Google), "input": important key phrases to give to search for, "observation": concise result from Google } MAKE BELIEVE

Slide 19

Slide 19 text

thought: This is about Star Trek, I will use Google search tool: Google input: Captain Kirk first name observation: James

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

WRONG

Slide 22

Slide 22 text

Retrieval-Augmented Generation

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Generation

Slide 25

Slide 25 text

You are given a question from human and you have to answer it concisely using only the following reference document. Avoid using any external information or recalling from memory. Now use the above reference to answer this question.

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

This is the weather observation for Palo Alto. The current weather condition is broken clouds. * barometric pressure: 1013 mbars. * temperature (in Celcius): 17 °C. * temperature (in Fahrenheit): 63 °F. * humidity: 60%.

Slide 28

Slide 28 text

Retrieval Improvement

Slide 29

Slide 29 text

Query Transformation

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

You are a language assistant. The user specifies a conversation history followed by a statement. Your task is to rephrase the statement as a new inquiry, with only the relevant references in the conversation history. Explain the reason behind the rephrasing. Always respond in JSON format, such as: { "rephrased": a reformulated statement of the original, "explanation": reason to rephrase it that way }

Slide 34

Slide 34 text

User: What is a dwarf planet? Assistant: A dwarf planet is a celestial body that orbits the sun, is large enough to be rounded by its own gravity, but has not cleared the neighborhood around its orbit of other debris. User: Give an example! User: What is a dwarf planet? Assistant: A dwarf planet is a celestial body that orbits the sun, is large enough to be rounded by its own gravity, but has not cleared the neighborhood around its orbit of other debris. User: Can you provide an example of a dwarf planet?

Slide 35

Slide 35 text

rephrased: Can you provide an example of a dwarf planet? explanation: The user wants to know an example of a dwarf planet, as they have just learned about what a dwarf planet is.

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

HYDE Hypothetical Document Embeddings

Slide 39

Slide 39 text

1 _ .. __ .. __ .. __ __ .. __ .. __ 2 _ .. __ .. __ .. __ __ .. __ .. __ 3 _ .. __ .. __ .. __ __ .. __ .. __ … __ .. __ .. __ .. __ __ .. __ .. __ .. __ 434 Dwarf planets are considered planets by some planetologists but not by the IAU. .. __ .. __ .. What is a dwarf planet? 77%

Slide 40

Slide 40 text

1 _ .. __ .. __ .. __ __ .. __ .. __ 2 _ .. __ .. __ .. __ __ .. __ .. __ 3 _ .. __ .. __ .. __ __ .. __ .. __ … __ .. __ .. __ .. __ __ .. __ .. __ .. __ 433 A dwarf planet is a body orbiting the Sun that is massive enough to be made near-spherical by its own gravity but that has not cleared planetesimals from its neighborhood and is also not a satellite. .. __ .. __ .. A dwarf planet is a celestial body that orbits the sun, is large enough to be rounded by its own gravity, but has not cleared the neighborhood around its orbit of other debris. 87%

Slide 41

Slide 41 text

Keyphrases Search

Slide 42

Slide 42 text

Keyphrases Citations

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

thought: This is about Star Trek, I will use Google search tool: Google input: Captain Kirk first name observation: James

Slide 45

Slide 45 text

Knowledge Base First Retriever Relevant Passages Rerank

Slide 46

Slide 46 text

Intent Classifier

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Reason-Act Thought: This is about weather and I will use weather action. Action: weather: Palo Alto Observation: No, it's not currently snowing in Palo Alto. Answer: It's not currently snowing in Palo Alto. 1

Slide 50

Slide 50 text

Reason-Act Thought: This is about weather and I will use weather action. Action: weather: Palo Alto Observation: The current weather in Palo Alto is clear sky at 17.5 C and humidity 48% Answer: No, it is not. 2 Weather API