Vector Search and LLM

Transcript

1. Vector Search & LLM
2. Quick Recap
3. llama.cpp Mistral 7B Pico Jarvis
4. github.com/ariya/pico-jarvis
5. llama.cpp
6. Mistral 7B
10. Fancy Autocomplete The CEO of Google is a fan of the company's new Pixel 4a phone, which has a $399 price tag. Google CEO Sundar Pichai said in a tweet that he was "excited" about the new phone and that iwas a great value for consumers.
11. Fancy Autocomplete The CEO of Google is a fan of the company's new Pixel 4a phone, which has a $399 price tag. Google CEO Sundar Pichai said in a tweet that he was "excited" about the new phone and that iwas a great value for consumers.
12. Stage Director This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: Who is the CEO of Google? Llama: ___ ___ ___ ___ ___ ___ ___
13. Stage Director This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: Who is the CEO of Google? Llama: The CEO of Google is Sundar Pichai.
15. Chain of Thought You run in a process of Question, Thought, Action, Observation. Use Thought to describe your thoughts about the question you have been asked. Observation will be the result of running those actions. Finally at the end, state the Answer. Thought: This is about science, I can recall the answer from my memory. Action: lookup: largest planet. Observation: Jupiter is the largest planet. Answer: The largest planet is Jupiter.
17. Reason-Act Thought: This is about weather and I will use weather action. Action: weather: Palo Alto Observation: No, it's not currently snowing in Palo Alto. Answer: It's not currently snowing in Palo Alto. 1
18. Reason-Act Thought: This is about weather and I will use weather action. Action: weather: Palo Alto Observation: The current weather in Palo Alto is clear sky at 17.5 °C and humidity 48% Answer: No, it is not currently snowing in Palo Alto. 2 Weather API
19. Retrieval-Augmented Generation (RAG) 1 2
20. Multi-turn Conversation
22. Sans History This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: How far is it from Earth? Llama: ????
23. With History This is a conversation between User and Llama, a friendly chatbot. Llama is helpful, kind, honest, good at writing, and never fails to answer any requests immediately and with precision. User: Which planet is the largest? Llama: Jupiter. User: How far is it from Earth? Llama: ___ ___ ___ ___ ___ ___ ___ ___
24. Vector Search
27. cities id name coord 731 Mountain View [37.39, -122.08] 987 Redwood City [37.49, -122.24] 819 Menlo Park [37.45, -122.18] 056 San Jose [37.34, -121.89]
28. SELECT name, DISTANCE(coord, REF) AS distance FROM cities ORDER BY distance LIMIT 5
30. When was our solar system formed? [0.776,0.192,0.183,0.221,0.362,0.731,0.398,0.006,0.617
31. When was our solar system formed? [0.776,0.192,0.183,0.221,0.362,0.731,0.398,0.006,0.617 [0.693,0.254,0.725,0.576,0.766,0.018,0.211,0.915,0.783 The solar system was formed 4.6 billions years ago. 92%
32. 1 The Solar System was formed 4.6 billion years ago 2 The Solar System will remain roughly as it is known 3 The outer Solar System is beyond the asteroids, incl … __ .. __ .. __ .. __ __ .. __ .. __ .. __ 408 Humanity's perspective Humanity's knowledge of the When was our solar system formed?
33. 1 [0.811,0.232,0.064,0.471,0.87,0.242,0.61,0.537,0.844, 2 [0.536,0.215,0.357,0.913,0.9,0.331,0.994,0.869,0.563, 3 [0.416,0.704,0.702,0.713,0.407,0.8,0.312,0.997,0.722, … 408 [0.369,0.523,0.151,0.636,0.766,0.836,0.296,0.259,0.75 [0.776,0.192,0.183,0.221,0.362,0.731,
34. 1 [0.811,0.232,0.064,0.471,0.87,0.242,0.61,0.537,0.844, 2 [0.536,0.215,0.357,0.913,0.9,0.331,0.994,0.869,0.563, 3 [0.416,0.704,0.702,0.713,0.407,0.8,0.312,0.997,0.722, … 408 [0.369,0.523,0.151,0.636,0.766,0.836,0.296,0.259,0.75 [0.776,0.192,0.183,0.221,0.362,0.731, 92% 83% 75%
35. Relevant Chunks The Solar System was formed 4.6 billion years ago … The Solar System will remain roughly as it is known … The outer Solar System is [40] beyond the asteroids …
36. Prompt, with Supporting Document You are good at answering question. Use the following supporting document. The Solar System was formed 4.6 billion years ago … The Solar System will remain roughly as it is known … The outer Solar System is [40] beyond the asteroids … User: When was solar system formed? Llama: ___ ___ ___ ___ ___ ___ ___ ___
37. HYDE Hypotethical Document Embeddings
38. pgvector github.com/pgvector/pgvector qdrant.tech vespa.ai
39. Thank You speakerdeck.com/ariya