Slide 1
Slide 1 text
LangChain4j,
Java,
& You
Holly Cummins
Sr. Technical Staff Member
IBM
LJC Unconference 2025
Slide 2
Slide 2 text
@holly_cummins #Quarkus #IBM #RedHat
1849
Slide 3
Slide 3 text
@holly_cummins #Quarkus #IBM #RedHat
1859
Slide 4
Slide 4 text
@holly_cummins #Quarkus #IBM #RedHat
data is
the new oil
2006
Slide 5
Slide 5 text
@holly_cummins #Quarkus #IBM #RedHat
2006
data is
the new oil
Slide 6
Slide 6 text
@holly_cummins #Quarkus #IBM #RedHat
a data scientist.
what do you call a statistician who lives in san francisco?
Slide 7
Slide 7 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 8
Slide 8 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 9
Slide 9 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 10
Slide 10 text
@holly_cummins #Quarkus #IBM #RedHat
the large
language model
Slide 11
Slide 11 text
@holly_cummins #Quarkus #IBM #RedHat
the large
language model
Slide 12
Slide 12 text
@holly_cummins #Quarkus #IBM #RedHat
deploy to prod
rest api
train the model
langchain
use output in code
standardize prompt structures
switch from one LLM model to another
memory (context)
Slide 13
Slide 13 text
@holly_cummins #Quarkus #IBM #RedHat
but there’s still
a problem
Slide 14
Slide 14 text
@holly_cummins #Quarkus #IBM #RedHat
deploy to prod
rest api
train the model
use output in code
standardize prompt structures
switch from one LLM model to another
memory (context)
langchain
langchain4j
Slide 15
Slide 15 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 16
Slide 16 text
@holly_cummins #Quarkus #IBM #RedHat
this is an integration
challenge
Slide 17
Slide 17 text
@holly_cummins #Quarkus #IBM #RedHat
use output in code
standardize prompt structures
switch from one LLM model to another
memory (context)
quarkus langchain4j
langchain4j
easy access to data
dependency injection
annotations
live coding
failover
mockability
observability
Slide 18
Slide 18 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 19
Slide 19 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 20
Slide 20 text
@holly_cummins #Quarkus #IBM #RedHat
… but not live code
more
Slide 21
Slide 21 text
@holly_cummins #Quarkus #IBM #RedHat
bootstrapping
io.quarkiverse.langchain4j
quarkus-langchain4j-openai
quarkus.langchain4j.openai.api-key=sk-...
configure an api key
Slide 22
Slide 22 text
@holly_cummins #Quarkus #IBM #RedHat
@RegisterAiService
interface Assistant {
String chat(String message);
}
define ai service
use DI to instantiate
assistant
@Inject
private final Assistant assistant;
Slide 23
Slide 23 text
@holly_cummins #Quarkus #IBM #RedHat
@SystemMessage("You are a professional poet")
@UserMessage("""
Write a poem about {topic}. The poem should be {lines} lines long.
""")
String writeAPoem(String topic, int lines);
add context to the calls
main message to send
Slide 24
Slide 24 text
@holly_cummins #Quarkus #IBM #RedHat
interface TransactionExtractor {
@UserMessage("Extract information about a transaction from {it}")
TransactionInfo extractTransaction(String text);
}
class TransactionInfo {
@Description("full name")
public String name;
@Description("IBAN value")
public String iban;
@Description("Date of the transaction")
public LocalDate transactionDate;
@Description("Amount in dollars of the transaction")
public double amount;
}
unmarshalling
objects
Slide 25
Slide 25 text
@holly_cummins #Quarkus #IBM #RedHat
memory
@RegisterAiService(chatMemoryProviderSupplier =
BeanChatMemoryProviderSupplier.class)
interface AiServiceWithMemory {
String chat(@UserMessage String msg);
}
---------------------------------
@Inject
private AiServiceWithMemory ai;
String userMessage1 = "Can you give a brief explanation of Kubernetes?";
String answer1 = ai.chat(userMessage1);
String userMessage2 = "Can you give me a YAML example to deploy an app
for this?";
String answer2 = ai.chat(userMessage2);
possibility to customize memory provider
remember previous
interactions
Slide 26
Slide 26 text
@holly_cummins #Quarkus #IBM #RedHat
@RegisterAiService(/*chatMemoryProviderSupplier =
BeanChatMemoryProviderSupplier.class*/)
interface AiServiceWithMemory {
String chat(@MemoryId Integer id, @UserMessage String msg);
}
---------------------------------
@Inject
private AiServiceWithMemory ai;
String answer1 = ai.chat(1,"I'm Frank");
String answer2 = ai.chat(2,"I'm Betty");
String answer3 = ai.chat(1,"Who Am I?");
default memory provider
refers to conversation with id == 1, ie. Frank
keep track of multiple parallel
conversations
Slide 27
Slide 27 text
@holly_cummins #Quarkus #IBM #RedHat
@RegisterAiService(tools = EmailService.class)
public interface MyAiService {
@SystemMessage("You are a professional poet")
@UserMessage("Write a poem about {topic}. Then send this poem by email.")
String writeAPoem(String topic);
public class EmailService {
@Inject
Mailer mailer;
@Tool("send the given content by email")
public void sendAnEmail(String content) {
mailer.send(Mail.withText("[email protected]", "A poem", content));
}
}
tools,
functions,
agents
describe when to use the tool
register the tool
ties it back to the tool description
Slide 28
Slide 28 text
@holly_cummins #Quarkus #IBM #RedHat
what could possibly go wrong?
(why raw models aren’t enough)
Slide 29
Slide 29 text
@holly_cummins #Quarkus #IBM #RedHat
the demo fails you didn’t get to see
Slide 30
Slide 30 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 31
Slide 31 text
@holly_cummins #Quarkus #IBM #RedHat
Guardrail messages: A Noodle is not a SNAKE.
Slide 32
Slide 32 text
@holly_cummins #Quarkus #IBM #RedHat
Slide 33
Slide 33 text
solution:
problem:
@holly_cummins #Quarkus #IBM #RedHat
hallucinations guard rails
RAG
Slide 34
Slide 34 text
solution:
problem:
@holly_cummins #Quarkus #IBM #RedHat
latency fault tolerance
fallbacks
Slide 35
Slide 35 text
solution:
problem:
@holly_cummins #Quarkus #IBM #RedHat
attacks enterprise security
Slide 36
Slide 36 text
solution:
problem:
@holly_cummins #Quarkus #IBM #RedHat
cost observability
smaller models
hybrid with rules engines
Slide 37
Slide 37 text
@holly_cummins #Quarkus #IBM #RedHat
really: data is the new oil
Slide 38
Slide 38 text
@holly_cummins #Quarkus #IBM #RedHat
- small models are cheaper + greener
- consider hybrid with rules engines
- keep an eye on memory build-up; use
no more context than needed
- limit how much retrieved content is
included in RAG queries
Slide 39
Slide 39 text
@holly_cummins #Quarkus #IBM #RedHat
this is not magic …
but it is amazing
Slide 40
Slide 40 text
https://ibm.biz/BdnRNh
Are you adding AI to existing Java applications?
Are you developing new AI applications in Java?
We would love to talk to you!
Use the QR code to provide your contact details and
we’ll be in touch.
Slide 41
Slide 41 text
slides + demo repo
@hollycummins.com
http://hollycummins.com/langchain4j-ljc-unconference/
survey
https://ibm.biz/BdnRNh