KotlinConf'25 - LangChain4j with Quarkus

Demo: https://github.com/quarkiverse/quarkus-langchain4j/tree/1.0.0.CR2/samples/chatbot-easy-rag-kotlin

Links: https://github.com/quarkiverse/quarkus-langchain4j/blob/1.0.0.CR2/samples/chatbot-easy-rag-kotlin/LINKS.md

Links

- Quarkus: https://quarkus.io/ – A Kubernetes Native Java stack tailored for OpenJDK HotSpot and GraalVM, crafted from the best of breed Java libraries and standards.

- Quarkus LangChain4j Extension: https://github.com/quarkiverse/quarkus-langchain4j

- Quarkus MCP Extension: https://github.com/quarkiverse/quarkus-langchain4j

- LangChain4J: https://docs.langchain4j.dev/

- LangChain4J GitHub: https://github.com/langchain4j/langchain4j/

- AI Services: https://docs.langchain4j.dev/tutorials/ai-services

- RAG: https://docs.langchain4j.dev/tutorials/rag

- Kotlin support: https://docs.langchain4j.dev/tutorials/kotlin

Other Links

- Retrieval-augmented generation (RAG): https://en.wikipedia.org/wiki/Retrieval-augmented_generation

- Blog post: "Building effective agents" by Anthropic: https://www.anthropic.com/engineering/building-effective-agents

- "Agentic AI with Quarkus" by Mario Fusco: https://github.com/mariofusco/quarkus-agentic-ai

- Model Context Protocol (MCP): https://modelcontextprotocol.io/

- Awesome MCP Servers: https://mcpservers.org/ – A collection of servers for the Model Context Protocol.

- https://github.com/kpavlov/langchain4j-kotlin – Unofficial LangChain4j Kotlin extensions

- mokksy.dev: https://mokksy.dev/ – AI services simulators for Integration testing

Transcript

1. LangChain4j with Quarkus Max Rydahl Andersen, Konstantin Pavlov @maxandersen @k_pavlov
2. Speakers Konstantin Pavlov Staff Software Engineer, Twilio LangChain4J contributor mail@kpavlov.me kpavlov.me @ 🦋bluesky Max Rydahl Andersen Distinguished Engineer, Red Hat Quarkus co-lead manderse@redhat.com / @maxandersen.xam.dk
3. WHY QUARKUS? QUARK: elementary particle US: hardest thing in computer science
4. quarkus create app --kotlin
5. 5 A stack to write Java apps Cloud-Native Microservices Serverless AI-Infused
6. Architecture Style Job Batch Cloud / Container Native Messaging Integration Serverless Kube Operator Type of application CRUD Monoliths Network Tools Admin Tools 6 AI-Infused CLI EDA Micro services
7. Frictionless Development Experience Build-Time Principle Reactive Core Prod & Cloud Ready
8. World Simplest AI Infused Quarkus App
9. //DEPS io.quarkus.platform:quarkus-bom:3.21.0@pom //DEPS io.quarkus:quarkus-picocli //DEPS io.quarkiverse.langchain4j:quarkus-langchain4j-openai:1.0.0.CR1 import dev.langchain4j.model.chat.ChatModel import picocli.CommandLine.Command @Command(name = "jokes") class Jokes (val ai: ChatModel) : Runnable { override fun run() { println(ai.chat("tell me a joke")) } }
10. Let me introduce you: quarkus-langchain4j https://docs.quarkiverse.io/quarkus-langchain4j Langchain4j Quarkus Langchain4j Application LLMs Vector stores Embedding Models - Declarative clients - CDI integration - Observability (Otel, Prometheus) - Auditing - Resilience - RAG building blocks - Tool and Agent support
11. Let me introduce you: quarkus-langchain4j https://docs.quarkiverse.io/quarkus-langchain4j Model Providers and Models OpenAI (GPT 3.5, GPT 4.1) Hugging Face (text-generation models) IBM Watson.x and BAM Anthropic (Claude) Mistral.ai (Mistral, Mixtral) Vertex.ai (Gemma, Gemini) OpenShift.ai (Mistral, Llama, Granite…) OLLama (Mistral, Llama, …) Podman Desktop (Mistral, Merlinite, Granite…) Embedding Store (vector) Chroma, Cohere, Infinispan, Milvus, Neo4J, PGVector, Pinecone, QDrant, Redis… Local Embedding all-minilm-l6-v2, bge-small-en, e5-small-v2, …
12. Agent and Tools A tool is a function that the model can call: - Tools are parts of CDI bean - Tools are defined and described using the @Tool Prompt (Context) Extend the context with tool descriptions Invoke the model The model asks for a tool invocation (name + parameters) The tool is invoked and the result sent to the model The model computes the response using the tool result Response
13. A Declarative Model @RegisterAiService(tools = [EmailService::class]) interface MyAiService { @SystemMessage("You are a professional poet") @UserMessage(""" Write a single poem about {topic}. The poem should be {lines} lines long and your response should only include them poem itself, nothing else. Then send this poem by email. Your response should include the poem. """) fun writeAPoem(topic: String, lines: Int): String } @ApplicationScoped class EmailService { @Inject lateinit var mailer: Mailer @Tool("send the given content by email") fun sendAnEmail(content: String) { Log.info("Sending an email: $content") mailer.send(Mail.withText("sendMeALetter@quarkus.io", "A poem for you", content)) } }
14. <~~ My prompt <~~ Tool invocation request <~~ Tool invocation response <~~ Model Response
15. Quarkus MCP Server SDK supports Kotlin //KOTLIN //DEPS io.quarkus:quarkus-bom:${quarkus.version:3.20.0}@pom //DEPS io.quarkiverse.mcp.servers:mcp-server-shared:1.0.0.CR3 import io.quarkiverse.mcp.server.* class demo { @Tool(description = "Say hello") fun hello(@ToolArg(description = "Who you want to be greeted") who: String) = "Hello $who" } Use jbang demo.kt in Claude Desktop, Goose, vscode, cursor, etc. See mcp-java.github.io and github.com/quarkiverse/quarkus-mcp-server Streamable support released today!
16. Demo: Financial Assistant https://github.com/quarkiverse/quarkus-langchain samples/chatbot-easy-rag-kotlin
17. Mocking LLM Call val mockOpenAi = MockOpenai() mockOpenAi.completion { userMessageContains("Tell me a joke about LLM") } responds { assistantContent = "Why did LLM cross road? Hallucination." } val model: JokeService = OpenAiChatModel.builder() .baseUrl(mockOpenAi.baseUrl()) // other settings .build() mokksy.dev ⚡🆓 fast.free.offline
18. Why LangChain4j? • “Hibernate of AIˮ as in: ◦ Usable from any java based framework - not tied to Spring, Quarkus etc. ◦ Lively community with contributions from every major and minor model providers • Has all you need and more to come ◦ Tools ◦ Embeddings ◦ Audio, video, text ◦ MCP clients ◦ Agentic flows ◦ … • LangChain4J 1.0 released last week! • LangChain4J.next key area is native async
19. Why Quarkus LangChain4J? Prompt facilities, Hot reload, Dev UI Support many APIs, models and vector stores Fault-Tolerance, Ambassador pattern Metrics, Tracing, Auditing…
20. Metrics, Tracing, Auditing with Quarkus 1. Add MicroMeter + OpenTelemetry quarkus ext add micrometer-opentelemetry 2. Have OTEL consumer setup • Quarkus OTEL LGTM devservice quarkus ext add quarkus-observability-devservices-lgtm • OTEL LGTM standalone ◦ run-lgtm • Or otel-tui
21. Metrics, Tracing, Auditing with Quarkus
22. Metrics, Tracing, Auditing from Quarkus
23. Quarkus MCP Server SDK supports Kotlin //KOTLIN //DEPS io.quarkus:quarkus-bom:${quarkus.version:3.20.0}@pom //DEPS io.quarkiverse.mcp.servers:mcp-server-shared:1.0.0.CR3 import io.quarkiverse.mcp.server.* class demo { @Tool(description = "Say hello") fun hello(@ToolArg(description = "Who you want to be greeted") who: String) = "Hello $who" } Use jbang demo.kt in Claude Desktop, Goose, vscode, cursor, etc. See mcp-java.github.io and github.com/quarkiverse/quarkus-mcp-server Streamable support released today!
24. Where can you The Kotlin Experts help? • Kotlin extension methods for LangChain4J • Extend/Complete Quarkus LangChain4J Kotlin integrations • Kotlin Compiler hooks/tricks to make it go faster • LangChain4j.next 2?) to get proper async apis ◦ That works with standard Java AND Kotlin
25. Thank You, and Donʼt Forget to Vote @maxandersen @k_pavlov Links: