Pass or Play: What does GenAI mean for the Java Developer?

You’ve been tasked with implementing or learning AI, whether that includes LLMs, RAG, or MLOps. The sea of acronyms in this era of AI can feel like (and sometimes actually are) a black box for the complex logic and processes that underpin them. There’s so much to learn and not enough time.In this session, we’ll explore how these technologies operate and the different practical ways GenAI is being used. We will explain common architectures for applications with GenAI, including RAG and GraphRAG. Live-code examples will show how Java developers can start using GenAI and help determine where they can be best applied. Come see how to play in the GenAI era.
Code: https://github.com/JMHReif/rag-vector-graph

Transcript

1. Pass or Play What does GenAI mean for the Java developer? Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Tech speaker, blogger, podcaster, author • Java Champion • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Photo by Matt Walsh on Unsplash AI Vector RAG LLM Algorithm Chaining Entity resolution Knowledge graph ML NLP GenAI Hallucination Embedding k-ANN Cosine similarity Euclidean distance Fine-tune Few-shot Grounding Model Prompt Semantic search Similarity Temperature Tokens Natural language ChatGPT Context window Agents GraphRAG MCP
4. Layers of AI • More layers = better result • Complexity vs value
5. Generative Arti fi cial Intelligence Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence
6. Layer #1: Large Language Model (LLM)
7. ChatGPT Public interface to LLM • Chat Generative Pre-trained Transformer • OpenAI, Nov 2022 • Natural language response • Predict next word • Feedback / reward to rank responses • Use cases: professional, personal, everything!
8. Demo, round 1!
9. Who is Jennifer Reif? Public info IS available…
10. But this is good! Generates new from existing patterns
11. Adding some context… Pointing to a speci fi c source as guide
12. Layer #2: “Simply add an LLM” doesn’t work…
13. Strategies to improve LLM accuracy • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG) • All of these involve training an LLM on speci fi c data!
14. RAG Pull data from external data sources • Retrieval • Data retrieved from external source • Augmented • Augments response with facts • Generation • Response in natural language
15. Types of context • Databases (all kinds) • Documents/folders • Media (music/images/video) • Charts (reports/visuals)
16. Embeddings / Vectors • Data -> array of fl oating point numbers • Captures meaning and how it’s used • Comparable format! Convert data to a point in space
17. Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html
18. Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3
19. Searching the data • Vectors -> semantic similarity • Proximity in vector space • Example: Library • Book classi fi cation - genre vs location of plot • More speci fi c + more relevant! Photo by Martin Adams on Unsplash …e ffi ciently
20. Demo, round 2! Vector search Data results Prompt phrase+data
21. Benefits With RAG + LLM • Guide probability with context (grounding) • Focuses / narrows probability area • Adds to LLM knowledge • Reduce margin of error • How did the LLM get this answer? Photo by No Revisions on Unsplash
22. Where do vectors fall flat? How do you… • Limited metadata / connections • Verify vector representations? • Explain how it got to answer? • Similar not always highest relevance?
23. Extending RAG
24. Extending RAG Vectors Vectors Vectors Vectors Similarity
25. Graphs -> how/why connections Analyze data from relationships • Relationships + entities • Designed for storing / navigating networks • Traceable • Explainable
26. What is a graph? Jennifer Michael ??
27. What is a graph? Company Jennifer Michael WORKED_FOR WORKED_FOR
28. What is a graph? Company Jennifer Michael School W ORKED_FOR WORKED_FOR ATTENDED ATTENDED
29. What is a graph? Degree Degree Company Jennifer Michael School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN
30. What is a graph? Dominic Degree Degree Company Jennifer Michael School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED
31. What is a graph? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
32. What is a graph? Answers through relationships • How many coworkers shared classes/degrees? • What are common degree journeys? • How many alumni re- enroll for higher degrees? • Who else went to a school and works for company? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
33. Nodes (vertices) Objects or entities • Can have labels • May have properties Person Degree Degree Company Person Person School Degree Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
34. Relationships (edges) Connect entities • Must have type (label) • Must have direction • May have properties Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Dominic
35. Graphs add context and meaning Not just point-to-point, but HOW they connect
36. Graphs connect the dots Structured + Unstructured • Flexible schema = natural fi t • Document connections, paths, etc • Connect varying data structures
37. GraphRAG Graph as data source • Accuracy: extra context / related connections • Veri fi ability: check against understandable format • Explainability: trace path through graph for answer
38. Demo, round 3!
39. Agents, MCP, and more - oh my! • Use case models: specialized for general, code, etc. • Memory: LLMs comprehend context over time • Evaluation: testing / governing LLM accuracy • Ethics: data privacy, bias, training practices • Agents / Tools: work fl ow -> autonomous • MCP: provide modular, consistent access to LLMs
40. Nothing is a silver bullet LLM is (of sorts) mind of its own • Can’t guarantee a consistent answer • Model decisions • Prompt construction • Context window limits
42. Resources • Github repository (today’s code): github.com/JMHReif/rag-vector-graph • Github repository: github.com/JMHReif/springai-goodreads • GraphAcademy LLM courses: graphacademy.neo4j.com/knowledge-graph-rag • Knowledge graph ebook: dev.neo4j.com/jmhreif-kg-ebook Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif