Pass or Play: What does GenAI mean for the developer?

You've been tasked with implementing or learning AI, whether that includes LLMs, RAG, or MLOps. The sea of acronyms in this era of AI can feel like (and sometimes actually are) a black box for the complex logic and processes that underpin them. There’s so much to learn and not enough time.
In this session, we'll explore how these technologies operate and the different practical ways GenAI is being used. We will explain common architectures for applications with GenAI, including RAG and GraphRAG. Live-code examples will show how developers can start using GenAI and help determine where they can be best applied. Come see how to play in the GenAI era.
Code: https://github.com/JMHReif/rag-vector-graph

Transcript

1. Pass or Play What does GenAI mean for the Java developer? Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Tech speaker, blogger, podcaster, author • Java Champion • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Photo by Matt Walsh on Unsplash AI Vector RAG LLM Algorithm Chaining Entity resolution Knowledge graph ML NLP GenAI Hallucination Embedding k-ANN Cosine similarity Euclidean distance Fine-tune Few-shot Grounding Model Prompt Semantic search Similarity Temperature Tokens Natural language ChatGPT Context window Agents GraphRAG MCP
4. Layers of AI • More layers = better result • Complexity vs value
5. Generative Arti fi cial Intelligence Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence
6. Layer #1: Large Language Model (LLM)
7. ChatGPT Public interface to LLM • Chat Generative Pre-trained Transformer • OpenAI, Nov 2022 • Natural language response • Predict next word • Feedback / reward to rank responses • Use cases: professional, personal, everything!
8. Demo, round 1!
9. Who is Jennifer Reif? Public info IS available…
10. But this is good! Generates new from existing patterns
11. Adding some context… Pointing to a speci fi c source as guide
12. Layer #2: “Simply add an LLM” doesn’t work…
13. Strategies to improve LLM accuracy • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG) • All of these involve training an LLM on speci fi c data!
14. RAG Pull data from external data sources • Retrieval • Data retrieved from external source • Augmented • Augments response with facts • Generation • Response in natural language
15. Types of context • Databases (all kinds) • Documents/folders • Media (music/images/video) • Charts (reports/visuals)
16. Embeddings / Vectors • Data -> array of fl oating point numbers • Captures meaning and how it’s used • Comparable format! Convert data to a point in space
17. Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html
18. Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3
19. Searching the data • Vectors -> semantic similarity • Proximity in vector space • Example: Library • Book classi fi cation - genre vs location of plot • More speci fi c + more relevant! Photo by Martin Adams on Unsplash …e ffi ciently
20. Demo data model
21. Demo, round 2! Vector search Data results Prompt phrase+data
22. Benefits With RAG + LLM • Guide probability with context (grounding) • Focuses / narrows probability area • Adds to LLM knowledge • Reduce margin of error • How did the LLM get this answer? Photo by No Revisions on Unsplash
23. Where do vectors fall flat? How do you… • Limited metadata / connections • Verify vector representations? • Explain how it got to answer? • Similar not always highest relevance?
24. Layer #3: Extending RAG
25. Graphs -> how/why connections Analyze data from relationships • Relationships + entities • Designed for storing / navigating networks • Traceable • Explainable
26. Social network LinkedIn Jennifer Priya ??
27. Social network Explicit links Company Jennifer Priya WORKED_FOR WORKED_FOR
28. Social network Explicit links Company Jennifer Priya School W ORKED_FOR WORKED_FOR ATTENDED ATTENDED
29. Skill Degree Company Jennifer Priya School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR H AS_SKILL COM PLETED Degree COM PLETED HAS_SKILL Social network Explicit links
30. Dominic Skill Company Jennifer Priya School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR H AS_SKILL HAS_SKILL Degree C O M PLETED COM PLETED Degree HAS_SKILL Social network Explicit links COM PLETED
31. Person Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR Degree C O M PLETED COM PLETED Work, Inc. Priya Jennifer Hire Me U CMIS Dominic Degree HAS_SKILL Skill H AS_SKILL HAS_SKILL Java CS Social network Explicit links COM PLETED
32. What is a graph? Answers through relationships • How many coworkers shared skills/degrees? • Who is within 2 connections of me and knows Neo4j? • Who adds new skills? • Who might be a good fi t for this project? Edward Jones Adrian Person Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR Degree C O M PLETED COM PLETED Priya Jennifer CMIS Degree HAS_SKILL Skill H AS_SKILL HAS_SKILL Java CS Dominic Work, Inc. Hire Me U COM PLETED
33. Nodes (vertices) Objects or entities • Still have entities • Flexible schema • Retain uniqueness Person Skill Company Person Person School Degree Priya Jennifer CMIS Java Degree CS Dominic Work, Inc. Hire Me U Username: jmhreif
34. Relationships (edges) Connect entities • Equal importance • Paved path > cutting trail • Links already there Person Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR Degree C O M PLETED COM PLETED Priya Jennifer CMIS Skill Java Degree CS Dominic Work, Inc. Hire Me U HAS_SKILL H AS_SKILL HAS_SKILL COM PLETED
35. Graphs add context and meaning Not just point-to-point, but HOW they connect
36. Graphs connect the dots Structured + Unstructured • Flexible schema = natural fi t • Document connections, paths, etc • Connect varying data structures
37. GraphRAG Graph as data source • Accuracy: extra context / related connections • Veri fi ability: check against understandable format • Explainability: trace path through graph for answer
38. Demo, round 3!
39. Layer #4 and beyond!
40. What is an agent? Software system using AI • What it does: • Evaluate -> decide what to do / how to solve • Respond -> execute tools • Work fl ows -> Autonomous system
41. What is MCP? Open standard • AI interact with sources, tools, and services • Modular, consistent access • Metaphors: • Universal adapter for AI • MCP server: microservice • Tool options: kids’ snacks • Vendor MCP servers: Docker containers MCP
42. And more - oh my! • Use case models: specialized for general, code, etc. • Memory: LLMs comprehend context over time • Evaluation: testing / governing LLM accuracy • Ethics: data privacy, bias, training practices • …
43. Nothing is a silver bullet LLM is (of sorts) mind of its own • Can’t guarantee a consistent answer • Model decisions • Prompt construction • Context window limits
45. Resources • Github repository (today’s code): github.com/JMHReif/rag-vector-graph • Github repository: github.com/JMHReif/springai-goodreads • GraphAcademy LLM courses: graphacademy.neo4j.com/knowledge-graph-rag • Knowledge graph ebook: dev.neo4j.com/jmhreif-kg-ebook Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif