RAG: Accuracy and Explainability in GenAI Applications

Accuracy and explainability are critical in GenAI applications. When information from AI-integrated solutions is inaccurate, it can impact business, people’s health, financial decisions, and even legal policies, which causes cascading repercussions. Having the best data at the right time is vital.
LLMs are not able to handle this on their own, but retrieval augmented generation (RAG) can help by providing curated data as context to an LLM, guiding it to an appropriate answer. This session will explore how vector and graph RAG address the shortcomings of LLMs, explaining their shared functionality as well as some ways they handle it differently. Finally, we will see how to build a GenAI application with RAG to see these concepts in action.
Code: https://github.com/JMHReif/rag-vector-graph

Transcript

1. RAG: Accuracy and Explainability in GenAI Applications Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Tech speaker, author, blogger, podcaster • Continuous learner • Java champion • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. When AI misses the mark…
4. Who is Jennifer Reif? Public info IS available…
5. But this is good! Generates new from existing patterns
6. Adding some context… Pointing to a speci fi c source as guide
7. Negative AI stories Even well-respected companies get it wrong • Hallucinating non-existent policy, legal cases • Chatbot produces Python • Legally binding vehicle o ff er • Harmful health advice • Threatening users • Inventing new language • Illegal activities (insider trading + local health laws)
8. Standalone LLM Doesn’t often work • Design: • human-consumable output • creative variation (probabilistic answers) • Problems: • too little detail, vague prompt • missing information (recent or private knowledge) • probabilistic ~= inconsistent
9. How do we avoid this? Add as much context as possible • Guide LLM to relevant ideas and content • Focuses / narrows search area • Adds to LLM knowledge • Reduces margin of error Photo by Ali Alauda on Unsplash
10. Adding context…
11. RAG architecture • Retrieval • Data retrieved from external source • Augmented • Augments response with facts • Generation • Response in natural language Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database
12. Types of context • Databases (all kinds) • Documents/folders • Media (music/images/video) • Charts (reports/visuals)
13. • Data -> array of fl oating point numbers • Captures meaning and how it’s used • Comparable format! Embeddings / Vectors Convert data to a point in space
14. • Vectors -> semantic similarity • Proximity in vector space • Example: Library • Book classi fi cation - genre vs location of plot • More speci fi c + more relevant! Photo by Martin Adams on Unsplash Searching the data …e ffi ciently
15. Retrieval sources
16. Vector database Store and retrieve data e ffi ciently • Create/store/search embeddings • Index is top unit of data storage • Prioritize stats • Some metadata/connections • Highly e ff i cient semantic search
17. Where do vectors fall flat? How do you… • Limited metadata / connections • Verify vector representations? • Explain how it got to answer? • Similar not always highest relevance?
18. “Unstructured” data Is it though? • Might be unstructured in appearance, length, styling • Structure is there! • Less obvious + consistent • Structure: • Themes • Moments • Stories
19. Graphs -> how/why connections Analyze data from relationships • Relationships + entities • Designed for storing / navigating networks • Traceable • Explainable
20. Nodes (vertices) Objects or entities • Can have labels • May have properties Person Degree Degree Company Person Person School Degree Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
21. Relationships (edges) Connect entities • Must have type (label) • Must have direction • May have properties Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
22. What is a graph? Answers through relationships • Who has similar journeys/ experiences? • New friends/club members? • Where did an employee/ student come from? • What makes 2 people similar? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
23. Graphs connect the dots Structured + Unstructured • Flexible schema = natural fi t • Document connections, paths, etc • Connect varying data structures
24. GraphRAG Using a graph as data source for RAG • Can be one of many sources • Layer relationships from structured • Caveat - still not silver bullet!
25. Architecting GenAI
26. Layers of AI • More layers = better result • Complexity vs value
27. Layers GenAI systems • Vector search • Additional context retrieval • Lexical search / fi ltering • Polyglot system • Agents / tools • LLM decides which to use (and order) • Range: automation -> autonomous • MCP https://www.anthropic.com/engineering/building-e ff ective-agents
28. Demo!
29. Align retrieval strategy Vector, graph, hybrid, agents? • Determine questions • Match retrieval strategy • Iterate!
31. Resources • Github repository (today’s code): github.com/JMHReif/rag-vector-graph • GraphAcademy LLM courses: graphacademy.neo4j.com/knowledge-graph-rag • Knowledge graph ebook: dev.neo4j.com/jmhreif-kg-ebook Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif