LangChain4j, Java, and You

If you’re not experimenting with AI, were you even developing in 2025? LangChain4j is emerging as a de facto standard
for LLM integration in Java. In this demo-driven talk, Holly will explore a range of cool LangChain4j capabilities, such
as type safe object mapping, stateful context, agents, RAG, guard rails, and fault tolerance.

Transcript

1. LangChain4j, Java, & You Holly Cummins Sr. Technical Staff Member IBM LJC Unconference 2025
2. @holly_cummins #Quarkus #IBM #RedHat 1849
3. @holly_cummins #Quarkus #IBM #RedHat 1859
4. @holly_cummins #Quarkus #IBM #RedHat data is the new oil 2006
5. @holly_cummins #Quarkus #IBM #RedHat 2006 data is the new oil
6. @holly_cummins #Quarkus #IBM #RedHat a data scientist. what do you call a statistician who lives in san francisco?
7. @holly_cummins #Quarkus #IBM #RedHat
8. @holly_cummins #Quarkus #IBM #RedHat
9. @holly_cummins #Quarkus #IBM #RedHat
10. @holly_cummins #Quarkus #IBM #RedHat the large language model
11. @holly_cummins #Quarkus #IBM #RedHat the large language model
12. @holly_cummins #Quarkus #IBM #RedHat deploy to prod rest api train the model langchain use output in code standardize prompt structures switch from one LLM model to another memory (context)
13. @holly_cummins #Quarkus #IBM #RedHat but there’s still a problem
14. @holly_cummins #Quarkus #IBM #RedHat deploy to prod rest api train the model use output in code standardize prompt structures switch from one LLM model to another memory (context) langchain langchain4j
15. @holly_cummins #Quarkus #IBM #RedHat
16. @holly_cummins #Quarkus #IBM #RedHat this is an integration challenge
17. @holly_cummins #Quarkus #IBM #RedHat use output in code standardize prompt structures switch from one LLM model to another memory (context) quarkus langchain4j langchain4j easy access to data dependency injection annotations live coding failover mockability observability
18. @holly_cummins #Quarkus #IBM #RedHat
19. @holly_cummins #Quarkus #IBM #RedHat
20. @holly_cummins #Quarkus #IBM #RedHat … but not live code more
21. @holly_cummins #Quarkus #IBM #RedHat bootstrapping <dependency> <groupId>io.quarkiverse.langchain4j</groupId> <artifactId>quarkus-langchain4j-openai</artifactId> </dependency> quarkus.langchain4j.openai.api-key=sk-... configure an api key
22. @holly_cummins #Quarkus #IBM #RedHat @RegisterAiService interface Assistant { String chat(String message); } define ai service use DI to instantiate assistant @Inject private final Assistant assistant;
23. @holly_cummins #Quarkus #IBM #RedHat @SystemMessage("You are a professional poet") @UserMessage(""" Write a poem about {topic}. The poem should be {lines} lines long. """) String writeAPoem(String topic, int lines); add context to the calls main message to send
24. @holly_cummins #Quarkus #IBM #RedHat interface TransactionExtractor { @UserMessage("Extract information about a transaction from {it}") TransactionInfo extractTransaction(String text); } class TransactionInfo { @Description("full name") public String name; @Description("IBAN value") public String iban; @Description("Date of the transaction") public LocalDate transactionDate; @Description("Amount in dollars of the transaction") public double amount; } unmarshalling objects
25. @holly_cummins #Quarkus #IBM #RedHat memory @RegisterAiService(chatMemoryProviderSupplier = BeanChatMemoryProviderSupplier.class) interface AiServiceWithMemory { String chat(@UserMessage String msg); } --------------------------------- @Inject private AiServiceWithMemory ai; String userMessage1 = "Can you give a brief explanation of Kubernetes?"; String answer1 = ai.chat(userMessage1); String userMessage2 = "Can you give me a YAML example to deploy an app for this?"; String answer2 = ai.chat(userMessage2); possibility to customize memory provider remember previous interactions
26. @holly_cummins #Quarkus #IBM #RedHat @RegisterAiService(/*chatMemoryProviderSupplier = BeanChatMemoryProviderSupplier.class*/) interface AiServiceWithMemory { String chat(@MemoryId Integer id, @UserMessage String msg); } --------------------------------- @Inject private AiServiceWithMemory ai; String answer1 = ai.chat(1,"I'm Frank"); String answer2 = ai.chat(2,"I'm Betty"); String answer3 = ai.chat(1,"Who Am I?"); default memory provider refers to conversation with id == 1, ie. Frank keep track of multiple parallel conversations
27. @holly_cummins #Quarkus #IBM #RedHat @RegisterAiService(tools = EmailService.class) public interface MyAiService { @SystemMessage("You are a professional poet") @UserMessage("Write a poem about {topic}. Then send this poem by email.") String writeAPoem(String topic); public class EmailService { @Inject Mailer mailer; @Tool("send the given content by email") public void sendAnEmail(String content) { mailer.send(Mail.withText("letter@quarkus.io", "A poem", content)); } } tools, functions, agents describe when to use the tool register the tool ties it back to the tool description
28. @holly_cummins #Quarkus #IBM #RedHat what could possibly go wrong? (why raw models aren’t enough)
29. @holly_cummins #Quarkus #IBM #RedHat the demo fails you didn’t get to see
30. @holly_cummins #Quarkus #IBM #RedHat
31. @holly_cummins #Quarkus #IBM #RedHat Guardrail messages: A Noodle is not a SNAKE.
32. @holly_cummins #Quarkus #IBM #RedHat
33. solution: problem: @holly_cummins #Quarkus #IBM #RedHat hallucinations guard rails RAG
34. solution: problem: @holly_cummins #Quarkus #IBM #RedHat latency fault tolerance fallbacks
35. solution: problem: @holly_cummins #Quarkus #IBM #RedHat attacks enterprise security
36. solution: problem: @holly_cummins #Quarkus #IBM #RedHat cost observability smaller models hybrid with rules engines
37. @holly_cummins #Quarkus #IBM #RedHat really: data is the new oil
38. @holly_cummins #Quarkus #IBM #RedHat - small models are cheaper + greener - consider hybrid with rules engines - keep an eye on memory build-up; use no more context than needed - limit how much retrieved content is included in RAG queries
39. @holly_cummins #Quarkus #IBM #RedHat this is not magic … but it is amazing
40. https://ibm.biz/BdnRNh Are you adding AI to existing Java applications? Are you developing new AI applications in Java? We would love to talk to you! Use the QR code to provide your contact details and we’ll be in touch.
41. slides + demo repo @hollycummins.com http://hollycummins.com/langchain4j-ljc-unconference/ survey https://ibm.biz/BdnRNh