LangChain4j, Kotlin, Quarkus and Testable AI

As LLMs reshape software development, Java and Kotlin developers face unique challenges when integrating these powerful AI models into production applications. This talk demonstrates how to build AI-powered applications using LangChain4j, Quarkus and Kotlin. We'll explore how Kotlin's expressive syntax and coroutines transform complex asynchronous Java code intoclean, maintainable code on Kotlin. Through a practical example, you'll see LangChain4j's capabilities for RAG, function calling, Model Context Protocol, and request moderation. The heart of this presentation addresses the testing challenge: how do you verify AI integrations without unpredictable responses, high costs, or rate limits? I'll introduce Mokksy, a testing library that enables deterministic, fast, and reliable tests for AI integrations. You'll learn practical patterns for mocking and evaluating LLMs responses. Who is this talk for? Java and Kotlin developers looking to integrate LLMs into production systems, teams struggling with testing AI components, and engineers seeking practical strategies for maintaining reliability in AI applications. Attendees will leave with code examples and approaches they can immediately apply to their projects.

DevTalks Romania - 2025
https://www.devtalks.ro/agenda/12-day-2#future-of-engineering-stage

Transcript

1. LangChain4j, Kotlin, Quarkus and Testable AI Konstantin Pavlov, Twilio in/kpavlov mail@kpavlov.me kpavlov.me Bluesky
2. Agentic Some JVM AI Frameworks Eclipse LMOS LangChain4J JetBrains Koog* Spring AI Embabel Agent Framework*
3. «A Hibernate of AI» What is LangChain4J docs.langchain4j.dev 🛠 Tools 🔢 Embeddings 🤖 (multi*)Agentic Flows MCP Clients 🧠 Large Language Models 🗂 Vector DBs 📚➕🧠 RAG 🌅🎞🎙 Multimodality 🚫 Moderation 💬 Chat Memory
4. Why Kotlin?
5. Java Async Code final var result = callApi() .thenCompose(this::processData) .thenCompose(this::saveToDb) .handle((UUID id, Throwable th) -> { if (th != null) { System.out.println("❌ Error: " + th.getMessage()); return null; } else { return id; } }) .join();
6. Kotlin Async Code runBlocking { try { val apiResult = callApi() val processedData = processData(apiResult) val persistentId = saveToDb(processedData) println("✅ Result: $persistentId") } catch (e: Exception) { println("❌ Error: ${e.message}") } }
7. Blocking Call → Suspend Function suspend fun <T> callBlockingApi( dispatcher: CoroutineDispatcher = Dispatchers.IO, blockingCall: () -> T, ): T = withContext(dispatcher) { blockingCall.invoke() } // Virtual Threads Dispatcher Executors.newVirtualThreadPerTaskExecutor() .asCoroutineDispatcher()
8. Why Kotlin! 🛡 Built-in Null safety 🔄 Coroutines: asynchronous code that feels synchronous 📝 Concise and expressive syntax 🔧 Seamless Java interoperability 📱 Maybe Kotlin Multiplatform? 😉
9. Quarkus quarkus.io 🐳 Kubernetes-native Java/Kotlin framework ⚡ Build-time processing for fast startup 🪄 GraalVM native image support 🔄 Live reload (“Dev Modeˮ) for rapid feedback 🧩 Supports both imperative & reactive styles «Supersonic/Subatomic/Java»
10. ⚙ CDI, Observability, Auditing Resilience, RAG, Tools, ... ✅ Your code 🤖 AI Abstractions Quarkus-LangChain4j docs.quarkiverse.io/quarkus-langchain4j
11. Demo: Quarkus-LangChain4J + promptfoo github.com/kpavlov/quarkus-assistant-demo
12. RAG Ingestion
13. 1 2 3 RAG Ingestion
14. RAG Retrieval
15. RAG Retrieval 1 2 3
16. AI Systems Testing Challenges 🤯 Non-deterministic LLM responses 💸 API costs add up quickly 🧩 CI/CD complexity and fragility, rate limits 😓 Hard to simulate edge cases
17. AI Testing 🤖 Agent Testing 📝 Prompt Testing 🔌 Functional Testing
18. Testing Pyramid 🌐 System Integration 🔌 Integration ⚡ Unit
19. Testing Pyramid 🌐 System Integration 🔌 Integration ⚡ Unit
20. AIEnabled System
21. AI Testing 🤖 Agent Testing 📝 Prompt Testing 🔌 Functional Testing
22. Integration Testing
23. Mokksy 📦 Black-box 🏎 Fast & Deterministic 🆓 Zero API costs Works offline / on CI 💥 Simulates negative scenarios ⏫ Server-Sent Events SSE 🤖 OpenAI, Anthropic, Gemini, A2A Protocol mokksy.dev
24. Mocking LLM Call val mockOpenAi = MockOpenai() mockOpenAi.completion { userMessageContains("Tell me a joke about LLM") } responds { assistantContent = "Why did LLM cross road? Hallucination." } val model = OpenAiChatModel.builder() .baseUrl(mockOpenAi.baseUrl()) // other settings .build() mokksy.dev ⚡ 🆓 fast+free+offline
25. AI Testing 🤖 Agent Testing 📝 Prompt Testing 🔌 Functional Testing
26. Promptfoo - Open-source for tool evaluation testing
27. Thank You LINKS tinyurl.com/Quarkus-Assistant-Demo-Links in/kpavlov mail@kpavlov.me kpavlov.me Bluesky Konstantin Pavlov, Twilio