Slide 1

Slide 1 text

Pass or Play What does GenAI mean for the Java developer? Jennifer Reif [email protected] @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif

Slide 2

Slide 2 text

Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Conference speaker • Tech blogger • Other: geek Jennifer Reif [email protected] @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif

Slide 3

Slide 3 text

Photo by Matt Walsh on Unsplash AI Vector RAG LLM Algorithm Chaining DS Entity resolution Knowledge graph ML NLP GenAI Hallucination Embedding k-ANN Cosine similarity Euclidean distance Fine-tune Few-shot Grounding Model Prompt Semantic search Similarity Temperature Tokens Natural language ChatGPT Chatbot Context window

Slide 4

Slide 4 text

ChatGPT Nov 2022 2023 Now… Vendor LLMs, Coding tools 2024 RAG GraphRAG, agents

Slide 5

Slide 5 text

Generative Arti fi cial Intelligence Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence

Slide 6

Slide 6 text

Large Language Model (LLM)

Slide 7

Slide 7 text

General info… About LLMs • Lots of data • Answers on probabilities • Training takes tons of hardware, money, time • Models • Di ff erent providers / companies train their own

Slide 8

Slide 8 text

Are LLMs all that? Worth the hype? • General information • Public domain knowledge • Historical data • Creative / arts • Human assistant • Task delegation Photo by Igor Omilaev on Unsplash

Slide 9

Slide 9 text

LLM issues • Lacking most recent data • Not always natural language • Language complexities, sarcasm, emotion • No sources • Hallucinations / Temperature • IP, bias, privacy

Slide 10

Slide 10 text

“Simply add an LLM” doesn’t work…

Slide 11

Slide 11 text

Strategies to improve LLM accuracy • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG) • All of these involve training an LLM on speci fi c data!

Slide 12

Slide 12 text

RAG Pull data from external data sources • Retrieval • Data retrieved from database • Augmented • Augments response with facts • Generation • Response in natural language

Slide 13

Slide 13 text

Explainable AI With RAG + LLM • How did the LLM get this answer? • Grounding LLM answer with veri fi ed data Photo by No Revisions on Unsplash

Slide 14

Slide 14 text

Applying RAG to an LLM

Slide 15

Slide 15 text

LLMs take text, not databases And context window limit

Slide 16

Slide 16 text

What is a vector? Mathematical realm • Line in space • Has length and direction horizontal vertical

Slide 17

Slide 17 text

Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html

Slide 18

Slide 18 text

Vector arithmetic C = a + b 1 a b 2 a b 3 a + b

Slide 19

Slide 19 text

Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3

Slide 20

Slide 20 text

Vectors to compare things What makes things similar? Length Width

Slide 21

Slide 21 text

Embeddings Convert data to a point in space • Series of numbers • 100s or 1000s of dimensions • Dimension = interesting feature / characteristic

Slide 22

Slide 22 text

LLMs take text, not databases Vectors

Slide 23

Slide 23 text

How do we search the vectors? Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Example: Library • Book classi fi cation - genre vs location of plot • Smaller search set = smaller retrieval time! Photo by Martin Adams on Unsplash

Slide 24

Slide 24 text

Provide prompt + context to LLM Vectors Text

Slide 25

Slide 25 text

RAG architecture Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database

Slide 26

Slide 26 text

Agentic Workflow Architecture • Uses “agents”/tools • LLM determines next step • Which tool/external source should be called • Uses result from tool as context Prompt + Relevant Information LLM API LLM
 Chat API User Tool Prompt Response Relevant Results / Documents 2 3 1 Source info

Slide 27

Slide 27 text

Nothing is a silver bullet LLM is (of sorts) mind of its own • Can’t guarantee a consistent answer • Prompt engineering • Context window limits

Slide 28

Slide 28 text

Real-World Uses

Slide 29

Slide 29 text

Chatbots Is there anything else? • “Mundane” support questions • Low criticality systems • Repeatable answers to common questions • Provide LLM with updated, correct, speci fi c info • User question -> Related documents -> LLM answer • Monitoring to improve

Slide 30

Slide 30 text

Semantic Search Movie search • Non-keyword search • Combine multiple data sources • Back by non-public or licensed data • Agents can provide additional components • I.e. trailers, summaries, renting, etc • Capabilities as good as curated data

Slide 31

Slide 31 text

Recommendations Product • Recommend based on smaller criteria • Follow user journey for what to try next • Cross-sell • Use similarity from “unrelated” criteria • Personalize experience to customer preferences

Slide 32

Slide 32 text

Other uses • Fraud/Anomaly detection • Monitoring/Logging activities • Bugs and fi xes • Code quality dependencies • Impact • Supply chain • Operations

Slide 33

Slide 33 text

Let’s code! Vector search Data results Prompt phrase+data

Slide 34

Slide 34 text

Demo! Our data model

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

Resources • Github repository (today’s code): github.com/JMHReif/springai-goodreads • GraphAcademy LLM courses: graphacademy.neo4j.com/categories/llms/ • Docs for Spring AI: docs.spring.io/spring-ai/reference/api/vectordbs/neo4j.html • NODES 2024: dev.neo4j.com/nodes24 Jennifer Reif [email protected] @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif