Pass or Play: What does GenAI mean for the Java developer?

GenAI, LLM, and other buzzwords are everywhere. The sea of acronyms can feel like (and sometimes actually are) a black box for the complex logic and processes that underpin them. Does/should a Java developer care? In this session, we’ll explore how these technologies operate and cover many of the technical terms that go along with them, such as hallucinations, grounding, and more. We will understand the abilities GenAI can provide to technical solutions alongside some of the struggles they bring, as well. Live-code examples will show how Java developers can utilize GenAI and help determine whether they are worth the hype. Come see if you should pass or play on GenAI.

Code repository: https://github.com/JMHReif/springai-goodreads

Transcript

1. Pass or Play What does GenAI mean for the Java developer? Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Conference speaker • Tech blogger • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Photo by Matt Walsh on Unsplash AI Vector RAG LLM Algorithm Chaining DS Entity resolution Knowledge graph ML NLP GenAI Hallucination Embedding k-ANN Cosine similarity Euclidean distance Fine-tune Few-shot Grounding Model Prompt Semantic search Similarity Temperature Tokens Natural language ChatGPT Chatbot Context window
4. ChatGPT Nov 2022 2023 Now… Vendor LLMs, Coding tools 2024 RAG GraphRAG, agents
5. Generative Arti fi cial Intelligence Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence
6. Large Language Model (LLM)
7. General info… About LLMs • Lots of data • Answers on probabilities • Training takes tons of hardware, money, time • Models • Di ff erent providers / companies train their own
8. Are LLMs all that? Worth the hype? • General information • Public domain knowledge • Historical data • Creative / arts • Human assistant • Task delegation Photo by Igor Omilaev on Unsplash
9. LLM issues • Lacking most recent data • Not always natural language • Language complexities, sarcasm, emotion • No sources • Hallucinations / Temperature • IP, bias, privacy
10. “Simply add an LLM” doesn’t work…
11. Strategies to improve LLM accuracy • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG) • All of these involve training an LLM on speci fi c data!
12. RAG Pull data from external data sources • Retrieval • Data retrieved from database • Augmented • Augments response with facts • Generation • Response in natural language
13. Explainable AI With RAG + LLM • How did the LLM get this answer? • Grounding LLM answer with veri fi ed data Photo by No Revisions on Unsplash
14. Applying RAG to an LLM
15. LLMs take text, not databases And context window limit
16. What is a vector? Mathematical realm • Line in space • Has length and direction horizontal vertical
17. Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html
18. Vector arithmetic C = a + b 1 a b 2 a b 3 a + b
19. Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3
20. Vectors to compare things What makes things similar? Length Width
21. Embeddings Convert data to a point in space • Series of numbers • 100s or 1000s of dimensions • Dimension = interesting feature / characteristic
22. LLMs take text, not databases Vectors
23. How do we search the vectors? Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Example: Library • Book classi fi cation - genre vs location of plot • Smaller search set = smaller retrieval time! Photo by Martin Adams on Unsplash
24. Provide prompt + context to LLM Vectors Text
25. RAG architecture Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database
26. Agentic Workflow Architecture • Uses “agents”/tools • LLM determines next step • Which tool/external source should be called • Uses result from tool as context Prompt + Relevant Information LLM API LLM
 Chat API User Tool Prompt Response Relevant Results / Documents 2 3 1 Source info
27. Nothing is a silver bullet LLM is (of sorts) mind of its own • Can’t guarantee a consistent answer • Prompt engineering • Context window limits
28. Real-World Uses
29. Chatbots Is there anything else? • “Mundane” support questions • Low criticality systems • Repeatable answers to common questions • Provide LLM with updated, correct, speci fi c info • User question -> Related documents -> LLM answer • Monitoring to improve
30. Semantic Search Movie search • Non-keyword search • Combine multiple data sources • Back by non-public or licensed data • Agents can provide additional components • I.e. trailers, summaries, renting, etc • Capabilities as good as curated data
31. Recommendations Product • Recommend based on smaller criteria • Follow user journey for what to try next • Cross-sell • Use similarity from “unrelated” criteria • Personalize experience to customer preferences
32. Other uses • Fraud/Anomaly detection • Monitoring/Logging activities • Bugs and fi xes • Code quality dependencies • Impact • Supply chain • Operations
33. Let’s code! Vector search Data results Prompt phrase+data
34. Demo! Our data model
36. Resources • Github repository (today’s code): github.com/JMHReif/springai-goodreads • GraphAcademy LLM courses: graphacademy.neo4j.com/categories/llms/ • Docs for Spring AI: docs.spring.io/spring-ai/reference/api/vectordbs/neo4j.html • NODES 2024: dev.neo4j.com/nodes24 Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif