Building Performant AI Applications with Nvidia NIMs and Haystack

Transcript

2. Introduction to inferencing with Mark 1. Intro to NIMs with Mark 2. LLM NIMs, Embedding NIMs and Re- ranking NIMs a. Building RAG with NIMs and Haystack, with Tuana 3. Deploy and Scale with Anshul 4. Q&A 5. Agenda @TUANACELIK /IN/TUANACELIK
3. FAQ The code and examples we show are all available on GitHub Yes this is recorded and will be up in the Haystack YouTube channel ASAP
4. RAG @TUANACELIK /IN/TUANACELIK
5. RAG Question Answering Given the documents, answer the question. Documents: {{ documents}} Question: {{ question }} @TUANACELIK /IN/TUANACELIK
6. RAG Question Answering Given the documents, answer the question. Documents: {{ documents}} Question: {{ question }} Question @TUANACELIK /IN/TUANACELIK
7. RAG Question Answering Given the documents, answer the question. Documents: {{ documents}} Question: {{ question }} Question @TUANACELIK /IN/TUANACELIK
8. RAG Question Question Answering Given the documents, answer the question. Documents: Question: {{ question }} @TUANACELIK /IN/TUANACELIK
9. ✨OUR ✨ RAG @TUANACELIK /IN/TUANACELIK
10. EMBEDDING AND STORING DATA Clean Split Embed @TUANACELIK /IN/TUANACELIK
11. EMBEDDING AND STORING DATA Clean Split Embed Convert PDF @TUANACELIK /IN/TUANACELIK snowflake/arctic-embed-l
12. RAG Embed Query Build Prompt Query Retrieve Document(s) Prompt Query Embedding Embeddings Documents Given the documents, answer the question. Documents: {{ documents}} Question: {{ query }} Query Prompt Prompt Response Generate Response @TUANACELIK /IN/TUANACELIK
13. RAG Embed Query Build Prompt Query Retrieve Document(s) Prompt Query Embedding Embeddings Documents Answer the question given the context. Question: {{ query }} Context: {% for document in documents %} {{ document.content }} {% endfor %} Answer: Query Prompt Prompt Response Generate Response @TUANACELIK /IN/TUANACELIK meta/llama-3.1-70b-instruct snowflake/arctic-embed-l
14. @TUANACELIK /IN/TUANACELIK build.nvidia.com
15. @TUANACELIK /IN/TUANACELIK RAG
16. @TUANACELIK /IN/TUANACELIK RAG
17. @TUANACELIK /IN/TUANACELIK RAG
18. @TUANACELIK /IN/TUANACELIK RAG
19. @TUANACELIK /IN/TUANACELIK RAG
20. @TUANACELIK /IN/TUANACELIK
21. 🗂️ DOCUMENTATION PIP INSTALL HAYSTACK-AI @HAYSTACK_AI EXAMPLE REPOSITORY
22. CONDITIONAL ACTIONS SEARCH THE WEB IF... ConditonalRouter WebSearch go_to_websearch answer Answer Generation Retrieval Prompt Response @TUANACELIK /IN/TUANACELIK
23. Fully open-source framework built in Python for custom AI applications Provides tools that developers need to build and customize state-of-the-art AI systems Building blocks: Pipelines & Components Component Component Pipeline @TUANACELIK /IN/TUANACELIK
24. Component Haystack 1.x: DAG Component Pipeline @TUANACELIK /IN/TUANACELIK
25. Haystack 2.0: DAG Pipeline 📚 Announced March 11, 2024 Component Component Component @TUANACELIK /IN/TUANACELIK
26. 📚 Announced March 11, 2024 Agentic Pipelines that can: Invoke the right tool, pipeline/branch Self reflect Correct Iterate through tasks Haystack 2.0: DAG @TUANACELIK /IN/TUANACELIK
27. SELF-REFLECTION PromptBuilder Generator Output Validator @TUANACELIK /IN/TUANACELIK
28. BUILDING TIME SOME PREP Question Answering Given the documents, answer the question. Documents: {{ documents}} Question: {{ question }} Summarization Summarize the following text. Text: {{ text}} Question Generation Given the following document, generate some questions Document: {{ document}} Question: Translation Translate the following text to French Text: {{ text }} PROMPTING (JINJA TEMPLATING) @TUANACELIK /IN/TUANACELIK
29. BUILDING TIME SOME PREP Input 1 Input 2 Output COMPONENTS @TUANACELIK /IN/TUANACELIK
30. BUILDING TIME SOME PREP Input 1 Input 2 Output COMPONENTS @TUANACELIK /IN/TUANACELIK
31. BUILDING TIME SOME PREP COMPONENTS from_lang to_lang documents documents @TUANACELIK /IN/TUANACELIK
32. BUILDING TIME SOME PREP COMPONENTS EMBEDDERS GENERATORS PREPROCESSORS PROMPT BUILDERS CLASSIFIERS CONVERTERS .... @TUANACELIK /IN/TUANACELIK
33. 💻
34. Pipeline @TUANACELIK /IN/TUANACELIK
35. Pipeline @TUANACELIK /IN/TUANACELIK
36. by Stefano Fiorucci 🚀 @TUANACELIK /IN/TUANACELIK
37. 🗂️ DOCUMENTATION 📚 TUTORIALS PIP INSTALL HAYSTACK-AI @TUANACELIK /IN/TUANACELIK TUANACELIK
38. HACKER NEWS SUMMARIES Summaries HackerNewsFetcher PromptBuilder Generator top_k