AI-Enabled Apps Practical Uses of AI in Applications Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif

Who is Jennifer Reif? • Continuous learner • Conference speaker • Tech blogger • Other: geek Developer Advocate, Neo4j Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif

Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. Generative Arti fi cial Intelligence https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence

LLMs What they’re good at… • General information • Public domain knowledge • Historical data • Creative / arts • Human assistant • Task delegation

LLMs What they’re not so good at… • Lacking most recent data • Not always natural language • Language complexities, sarcasm, emotion • No sources • Hallucinations / Temperature • IP, bias, privacy

Improving LLM accuracy

Approaches • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG)

RAG Retrieval Augmented Generation • Retrieval • Data retrieved from database • Augmented • Augments response with facts • Generation • Response in natural language Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database

Agentic Workflow Architecture • Uses “agents”/tools • LLM determines next step • Which tool/external source should be called • Uses result from tool as context Prompt + Relevant Information LLM API LLM
 Chat API User Tool Prompt Response Relevant Results / Documents 2 3 1 Source info

Explainable AI • How did the LLM get this answer? • Grounding LLM answer with veri fi ed data With RAG + LLM

Other benefits With non-LLM data • Information that LLM doesn’t have • More recent than LLM cuto ff • Private data • Ensure accurate, non-con fl icting data • Reduce hallucinations

RAG needs curated data…

Private data set examples Probably data that already exists • Lessons learned • Project docs • Knowledge base / support docs • Onboarding / internal processes • Performance reviews / internal surveys • Job applicants / resumés • Internal trainings, videos, other content

Vectors and Data Sources

How do we use vectors? https://www.mathsisfun.com/algebra/vectors.html

Example Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3

Benefits Of using vectors • Embed nearly any type of data • Image, document, product, text, video, audio, sentence, word • Can be used to feed all sorts of architectures • Non-keyword search • Variety of criteria for search

Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Example: Library • Book classi fi cation - author vs location of plot • Smaller search set = smaller retrieval time! Vector indexes Photo by Martin Adams on Unsplash

Real-World Uses

Chatbot GDS Documentation • LLM hallucinating answers • Provide LLM with updated info • Assist users with learning GDS library • Backed by knowledge graph of documentation text • Streamlit + LangChain app • Log conversations with data • Improvements to responses!

Benefits • Log conversations alongside grounding data • How LLM/users interact with documents • Monitor performance and improve • Analyze data: Algorithms + Visualization • Surface data quality issues in text • Chunk outliers or overlaps • Collapse duplicates

Logging and Visualizing Conversations With a graph database Graph of an actual conversation between an Agent Neo user and the ChatGPT-4 LLM. Context Documents are labeled with their GDS Community.

Semantic Search Movie search • Search movie based on di ff erent criteria • General theme or setting or plot? • Net fl ix search doesn’t work like this • Includes agentic approach (UI, trailers, etc) github.com/datastax/movies_plus_plus

Benefits • Gather multiple data sources into one interface • Search on variety of criteria • Augment traditional search and LLM • Log interactions, likes, paths

Recommendations Product (e.g. Book) • Search based on smaller criteria • Follow user journey for what to try next • Cross-sell • Use similarity from “unrelated” criteria Photo by Carl Raw on Unsplash

Benefits • Combine di ff erent data sources • User preferences/lifestyles, order system, supply chain, etc • Cross-pollinate product categories • Customer 360, real-time markets, trends

Other use cases • Fraud/Anomaly detection • Monitoring/Logging activities • Bugs and fi xes • Dependencies • Impact • Supply chain • Operations

Nothing is a silver bullet • Can’t guarantee a consistent answer • Prompt engineering • Context window limits LLM is (of sorts) mind of its own

Let’s code! Vector search Data results Prompt phrase+data

No content

Resources • Code: github.com/JMHReif/springai-goodreads • GraphAcademy courses: graphacademy.neo4j.com/categories/llms/ • NODES 2024: dev.neo4j.com/nodes24 Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif