Tweet
Tweet

Slide 1

Slide 1 text

Vector Search & LLM

Slide 2

Slide 2 text

Quick Recap

Slide 3

Slide 3 text

llama.cpp Mistral 7B Pico Jarvis

Slide 4

Slide 4 text

github.com/ariya/pico-jarvis

Slide 5

Slide 5 text

llama.cpp

Slide 6

Slide 6 text

Mistral 7B

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Fancy Autocomplete The CEO of Google is a fan of the company's new Pixel 4a phone, which has a $399 price tag. Google CEO Sundar Pichai said in a tweet that he was "excited" about the new phone and that iwas a great value for consumers.

Slide 11

Slide 11 text

Fancy Autocomplete The CEO of Google is a fan of the company's new Pixel 4a phone, which has a $399 price tag. Google CEO Sundar Pichai said in a tweet that he was "excited" about the new phone and that iwas a great value for consumers.

Slide 12

Slide 12 text

Stage Director This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: Who is the CEO of Google? Llama: ___ ___ ___ ___ ___ ___ ___

Slide 13

Slide 13 text

Stage Director This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: Who is the CEO of Google? Llama: The CEO of Google is Sundar Pichai.

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

Chain of Thought You run in a process of Question, Thought, Action, Observation. Use Thought to describe your thoughts about the question you have been asked. Observation will be the result of running those actions. Finally at the end, state the Answer. Thought: This is about science, I can recall the answer from my memory. Action: lookup: largest planet. Observation: Jupiter is the largest planet. Answer: The largest planet is Jupiter.

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Reason-Act Thought: This is about weather and I will use weather action. Action: weather: Palo Alto Observation: No, it's not currently snowing in Palo Alto. Answer: It's not currently snowing in Palo Alto. 1

Slide 18

Slide 18 text

Reason-Act Thought: This is about weather and I will use weather action. Action: weather: Palo Alto Observation: The current weather in Palo Alto is clear sky at 17.5 °C and humidity 48% Answer: No, it is not currently snowing in Palo Alto. 2 Weather API

Slide 19

Slide 19 text

Retrieval-Augmented Generation (RAG) 1 2

Slide 20

Slide 20 text

Multi-turn Conversation

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Sans History This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: How far is it from Earth? Llama: ????

Slide 23

Slide 23 text

With History This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: Which planet is the largest? Llama: Jupiter. User: How far is it from Earth? Llama: ___ ___ ___ ___ ___ ___ ___ ___

Slide 24

Slide 24 text

Vector Search

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

cities id name coord 731 Mountain View [37.39, -122.08] 987 Redwood City [37.49, -122.24] 819 Menlo Park [37.45, -122.18] 056 San Jose [37.34, -121.89]

Slide 28

Slide 28 text

SELECT name, DISTANCE(coord, REF) AS distance FROM cities ORDER BY distance LIMIT 5

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

When was our solar system formed? [0.776,0.192,0.183,0.221,0.362,0.731,0.398,0.006,0.617

Slide 31

Slide 31 text

When was our solar system formed? [0.776,0.192,0.183,0.221,0.362,0.731,0.398,0.006,0.617 [0.693,0.254,0.725,0.576,0.766,0.018,0.211,0.915,0.783 The solar system was formed 4.6 billions years ago. 92%

Slide 32

Slide 32 text

1 The Solar System was formed 4.6 billion years ago 2 The Solar System will remain roughly as it is known 3 The outer Solar System is beyond the asteroids, incl … __ .. __ .. __ .. __ __ .. __ .. __ .. __ 408 Humanity's perspective Humanity's knowledge of the When was our solar system formed?

Slide 33

Slide 33 text

1 [0.811,0.232,0.064,0.471,0.87,0.242,0.61,0.537,0.844, 2 [0.536,0.215,0.357,0.913,0.9,0.331,0.994,0.869,0.563, 3 [0.416,0.704,0.702,0.713,0.407,0.8,0.312,0.997,0.722, … 408 [0.369,0.523,0.151,0.636,0.766,0.836,0.296,0.259,0.75 [0.776,0.192,0.183,0.221,0.362,0.731,

Slide 34

Slide 34 text

1 [0.811,0.232,0.064,0.471,0.87,0.242,0.61,0.537,0.844, 2 [0.536,0.215,0.357,0.913,0.9,0.331,0.994,0.869,0.563, 3 [0.416,0.704,0.702,0.713,0.407,0.8,0.312,0.997,0.722, … 408 [0.369,0.523,0.151,0.636,0.766,0.836,0.296,0.259,0.75 [0.776,0.192,0.183,0.221,0.362,0.731, 92% 83% 75%

Slide 35

Slide 35 text

Relevant Chunks The Solar System was formed 4.6 billion years ago … The Solar System will remain roughly as it is known … The outer Solar System is [40] beyond the asteroids …

Slide 36

Slide 36 text

Prompt, with Supporting Document You are good at answering question. Use the following supporting document. The Solar System was formed 4.6 billion years ago … The Solar System will remain roughly as it is known … The outer Solar System is [40] beyond the asteroids … User: When was solar system formed? Llama: ___ ___ ___ ___ ___ ___ ___ ___

Slide 37

Slide 37 text

HYDE Hypotethical Document Embeddings

Slide 38

Slide 38 text

pgvector github.com/pgvector/pgvector qdrant.tech vespa.ai

Slide 39

Slide 39 text

Thank You speakerdeck.com/ariya