Pass or Play: What does GenAI mean for the Java developer?

You've been tasked with implementing or learning AI, whether that includes LLMs, RAG, or MLOps. The sea of acronyms in this era of AI can feel like (and sometimes actually are) a black box for the complex logic and processes that underpin them. There's so much to learn and not enough time.
In this session, we'll explore how these technologies operate and the different practical ways GenAI is being used. We will explain common architectures for applications with GenAI, including RAG and GraphRAG. Live-code examples will show how Java developers can start using GenAI and help determine where they can be best applied. Come see how to play in the GenAI era.
Code: https://github.com/JMHReif/springai-goodreads

Transcript

1. Pass or Play What does GenAI mean for the Java developer? Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Conference speaker • Tech blogger • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Photo by Matt Walsh on Unsplash AI Vector RAG LLM Algorithm Chaining Entity resolution Knowledge graph ML NLP GenAI Hallucination Embedding k-ANN Cosine similarity Euclidean distance Fine-tune Few-shot Grounding Model Prompt Semantic search Similarity Temperature Tokens Natural language ChatGPT Context window Agents GraphRAG
4. Generative Arti fi cial Intelligence Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence
5. Large Language Model (LLM)
6. Are LLMs all that? Worth the hype? • General information • Public domain knowledge • Historical data • Creative / arts • Human assistant • Task delegation Photo by Igor Omilaev on Unsplash
7. LLM issues • Lacking most recent data • Not always natural language • Language complexities, sarcasm, emotion • No sources • Hallucinations / Temperature • IP, bias, privacy
8. “Simply add an LLM” doesn’t work…
9. Strategies to improve LLM accuracy • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG) • All of these involve training an LLM on speci fi c data!
10. RAG Pull data from external data sources • Retrieval • Data retrieved from database • Augmented • Augments response with facts • Generation • Response in natural language
11. Explainable AI With RAG + LLM • How did the LLM get this answer? • Grounding LLM answer with veri fi ed data Photo by No Revisions on Unsplash
12. Applying RAG to an LLM
13. LLMs take media, not databases And context window limit
14. What is a vector? Mathematical realm • Line in space • Has length and direction horizontal vertical
15. Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html
16. Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3
17. Vectors to compare things What makes things similar? Length Width
18. Embeddings Convert data to a point in space • Series of numbers • 100s or 1000s of dimensions • Dimension = interesting feature / characteristic
19. LLMs take text, not databases Vectors
20. How do we search the vectors? Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Example: Library • Book classi fi cation - genre vs location of plot • Smaller search set = smaller retrieval time! Photo by Martin Adams on Unsplash
21. Provide prompt + context to LLM Vectors Vectors Vectors Vectors Similarity
22. What is a graph? Natural representation of data • Storing relationships with entities • Documents path • Can answer why • Can fi nd alternates • Understand / fi nd patterns • Web of connections, related info • Bonus: Visual representation of data
23. Book domain • Find authors with reviews for multiple books • Find similar users based on reviews of books and related authors
24. Property graph • Node (vertex) • Relationship (edge)
25. Nodes • Represent objects or entities • Can be labeled • May have properties Book Author title: “Star Wars” isbn: 9756165498 name: “George Lucas” avgRating: 4.72 Review rating: 4.2 reviewText: “Blah” votes: 17
26. Relationships • Must have a type (label) • Must have a direction • May have properties Book Author title: “Star Wars” isbn: 9756165498 name: “George Lucas” avgRating: 4.72 Review rating: 4.2 reviewText: “Blah” votes: 17 AUTHORED WRITTEN_FOR date_added: “Sun Jan 03”
27. GraphRAG Added context • Data + other context • Documenting path • Adding connected context • Logging interactions with data • Network algos • Visualize data + interactions • Analyze performance LLM Responses (pink) and 
 Most Frequently Used Context Documents (red)
28. Example: Agent Neo Chatbot for learning GDS • LLM hallucinating answers • Provide LLM with updated info • Backed by KG of curated documentation • Streamlit + Langchain app • Log conversations with data • Improvements to responses!
29. Provide prompt + context to LLM Vectors Vectors Vectors Vectors Similarity Graph
30. RAG architecture Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database
31. Agentic Workflow Architecture • Uses “agents”/tools • LLM determines next step • Which tool/external source should be called • Uses result from tool as context Prompt + Relevant Information LLM API LLM
 Chat API User Tool Prompt Response Relevant Results / Documents 2 3 1 Source info
32. Nothing is a silver bullet LLM is (of sorts) mind of its own • Can’t guarantee a consistent answer • Prompt engineering • Context window limits
33. Let’s code! Vector search Data results Prompt phrase+data
34. Demo! Our data model
36. Resources • Github repository (today’s code): github.com/JMHReif/springai-goodreads • GraphAcademy LLM courses: graphacademy.neo4j.com/categories/llms/ • Docs for Spring AI: docs.spring.io/spring-ai/reference/api/vectordbs/neo4j.html Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif