Building RAG powered applications - PyData London 2nd April

Transcript

1. Building RAG powered applications Aniket Maurya, AI Engineer at Lightning AI April 2024
2. 2 Aniket Maurya AI Engineer @ /aniketmaurya /in/aniketmaurya
3. • RAG and its components • Building a RAG application • Questions Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. 3 Agenda
4. 4 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Prompt Engineering Finetune LLMs a b
5. 5 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Prompt Engineering Finetune LLMs a b • Easy to get started • LLMs lack latest information • Hard to scale • LLMs have latest information • Incurs cost of finetuning
6. 6 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved.
7. 7 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Prompt Engineering Finetune LLMs a b • Easy to get started • LLMs lack latest information • Hard to scale • LLMs have latest information • Incurs cost of finetuning
8. 8 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. RAG • LLMs have latest information • No finetuning cost • Reduce hallucination • Cite sources
9. What is RAG 9 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved.
10. 10 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. User query LLM Generate output DB Prompt(Retrieved context + User query)
11. 11 Retrieval Augmented Generation Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. 1. Data Ingestion 2. Data Indexing and Retrieval 3. Chaining LLM with Retrieval
12. 12 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs
13. 13 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs • Load unstructured data • Split document • Fit model’s context window • Small document = Accurate embedding
14. 14 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. https://chunkviz.up.railway.app/
15. 15 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs 0.2 0.1 0.8 0.3
16. 16 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs source: Sentence Transformers
17. 17 Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs In state after state, new laws have been passed, not only to suppress the vote, but to subvert entire elections. Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. RAG - Parent Document Retriever
18. 18 RAG - Parent Document Retriever Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs In state after state, new laws have been passed, not only to suppress the vote, but to subvert entire elections. Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans.
19. 19 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs
20. 20 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs from langchain_core.prompts import ChatPromptTemplate prompt_template = ChatPromptTemplate.from_template( ( "Please answer the following question based on the provided `context` that follows the question.\n" "Think step by step before coming to answer. If you do not know the answer then just say 'I do not know'\n" "question: {question}\n" "context: ```{context}```\n" ) )
21. 21 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs from langchain_community.chat_models import ChatOllama from langchain_core.output_parsers import StrOutputParser llm = ChatOllama(model="mistral") chain = prompt_template | llm | StrOutputParser()
22. 22 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs from rag_101.retriever import retrieve_context question = "What is the source of the dataset the model was trained on?" context, similarity_score = retrieve_context(query, retriever_model, reranker_model)[0] context = context.page_content chain.invoke({"context": context, "question": question})
23. 23 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs from rag_101.retriever import retrieve_context question = "What is the source of the dataset the model was trained on?" context, similarity_score = retrieve_context(query, retriever_model, reranker_model)[0] context = context.page_content chain.invoke({"context": context, "question": question})
24. 24 RAG Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved. Data Ingestion Data Indexing & Retrieval Chaining LLMs from langchain_core.prompts import ChatPromptTemplate prompt_template = ChatPromptTemplate.from_template( ( "Please answer the following question based on the provided `context` that follows the question.\n" "Think step by step before coming to answer. If you do not know the answer then just say 'I do not know'\n" "question: {question}\n" "context: ```{context}```\n" ) )
25. 25 Aniket Maurya Lightning AI ©2024 Proprietary and Confidential. All Rights Reserved.