Beyond Code Sharing: Building Smarter Mobile Apps with KMP and Agents :: Devfest Gandhinagar 2025

For years we’ve talked about code sharing with Kotlin Multiplatform (KMP). This talk explores the next frontier: integrating AI agents directly into your KMP architecture to create intelligent, context-aware mobile applications.

We’ll walk through building a practical cross-platform mobile app (Android + iOS) from a single KMP codebase, and show how to embed an intelligent agent using Koog. The focus is on how agents become a first-class architectural element in mobile apps—bridging local intelligence, cloud reasoning, and structured workflows.

Key themes of the session include:

Koog fundamentals: defining tools, intents, and structured actions so an agent can operate safely and predictably inside your KMP app.

On-device intelligence: combining Gemini Nano, MediaPipe GenAI, and ML Kit/Vision APIs for OCR, classification, summarization, and lightweight inference—keeping apps fast, private, and responsive.

Cloud-based reasoning: leveraging Gemini models via Vertex AI to handle semantic understanding, summarization, and fallback cases when on-device heuristics aren’t enough.

Production pragmatism: lessons learned from shipping KMP at scale—covering performance, security, observability, and how to balance on-device vs. cloud intelligence.

By the end, you’ll have a clear blueprint for:

Embedding Koog agents into a Kotlin Multiplatform app.

Using Gemini Nano, MediaPipe, and ML Kit for fast, offline inference.

Scaling up with Vertex AI + Gemini for richer reasoning.

Designing apps that feel truly intelligent and multiplatform, not just code-shared.

This isn’t theory—we’ll ground it in a working, useful app that shows the stack in action, but the real takeaway will be the architecture and technologies you can apply to your own projects.

A practical deep dive into building production-grade Kotlin Multiplatform apps—this time with agents at the core. Learn how Koog enables structured agent workflows, how Gemini and Vertex AI extend reasoning in the cloud, and how on-device ML with Gemini Nano, MediaPipe, and ML Kit keeps apps responsive and privacy-first.

Transcript

1. Beyond Code Sharing: Building Smarter Mobile Apps with KMP and Agents (with Koog, Gemini and More) A Practical Guide to Native & Smart Apps for multiple platforms Rivu Chakraborty
2. WHO AM I? • GDE (Google Developer Expert) for Android • Previously India’s first GDE for Kotlin • More than 14 years in the Industry • Founder @ Mobrio Studio • Previously ◦ JioCinema/JioHotstar, Byju’s, Paytm, Gojek, Meesho • Author (wrote multiple Kotlin books) • Speaker • Mentor • Learning / Exploring ML/AI • Community Person (Started KotlinKolkata) • YouTuber (http://youtube.com/@RivuTalks)
3. • Led personally by me (Rivu), with my decades of experience of scaling 6+ unicorn startups, and many smaller ones • We do Mobile Dev tooling (products) as well as we consult with product based startups, helping them develop or scale their apps • We can help with anything to do with mobile, starting from code quality, migration, refactor to feature development • At Mobrio Studio, I have a team, who work under my direct super vision. • We don’t just develop for you, we train your team, so you’re independent in future https://mobrio.studio/
4. Beyond Code Sharing: Building Smarter Mobile Apps with KMP and Agents (with Koog, Gemini and More)
5. WHY THIS TALK? • GenAI & Agents are hot • Gemini API, Gemini Nano (Experimental) and Gemma models allow apps to use AI easily • KMP lets us build once for Android, iOS, Web & More • Koog let’s you build agents for multiple platforms • We'll walk through real code & gotchas
6. What’s KMP and Why? • A technology by JetBrains to share Kotlin code across platforms (Android, iOS, web, desktop, server). • Enables platform-specific UI while sharing core business logic (networking, database, state management). You control what you share and what you don’t • Write Once, Run Natively: Outputs native binaries (no VM or JS bridge).
7. What’s KMP and Why? • Incremental Adoption: Can be integrated into existing apps module by module, reducing migration risk. • Kotlin Ecosystem: Leverages the robust Kotlin ecosystem including Coroutines, Serialization, Ktor, SQLDelight, etc.
8. A Brief on KMP Kotlin Multiplatform Kotlin
9. A Brief on KMP
10. A Brief on KMP Common JVM JS Native Android Server Apple Linux mingw iOS tvOS watchOS macOS linux_x64 linux_arm64 mingx64 iosX64 iosArm64 iosSimulatorArm64 tvosX64 tvosArm64 tvosSimulatorArm6 4 macosX64 macosArm64 watchosX64 watchosArm32 watchosArm64 watchosSimulatorArm64 watchosDeviceArm64
11. A Brief on KMP expect fun doSomething() actual fun doSomething() { //Platform specific logic here }
12. A Brief on KMP
13. A Brief on KMP expect fun getRandomUUID(): String actual fun getRandomUUID(): String = UUID.randomUUID().toString() actual fun getRandomUUID(): String = NSUUID().UUIDString() import uuid actual fun getRandomUUID(): String = uuid.v4().toString() commonMain jvmMain iosMain jsMain
14. What is GenAI in Mobile Development? • GenAI brings creative intelligence to mobile apps by enabling them to generate rather than just respond. • Enables hyper-personalized, intelligent, and context-aware user experiences. • Enhances accessibility, productivity, and entertainment within apps. • Can run on-device (for privacy/speed) or via cloud APIs. • In mobile apps, GenAI powers features like: a. Text generation (e.g., storytelling, smart replies, chatbots) b. Image generation/editing c. Voice synthesis (TTS)
15. What’s AI? Algorithm Input Output Developers write explicit algorithms that take input and produce a desired output. 1. Train the model with large dataset of input and output 2. Model is deployed on cloud/on-device to process input data i.e. inference Traditional Programming Machine Learning ML Model Training Input ML Model Output Run ML Inference Input Output
16. What’s GenAI? • Generative AI introduces the capability to understand inputs such as text, images, audio and video and generate human-like responses. • This enables applications like chatbots, language translation, text summarization, image captioning, image or code generation, creative writing assistance, and much more. • At its core, an LLM is a neural network model trained on massive amounts of text data. It learns paerns, grammar, and semantic relationships between words and phrases, enabling it to predict and generate text that mimics human language.
17. Why Gemini (by Google)? • Multimodal: Understands text, image, code, audio, and more. • Optimized for Android, iOS & Web • Enhances accessibility, productivity, and entertainment within apps. • Developer Friendly a. Easy-to-use libraries / APIs b. SDKs support prompting, streaming, and low-latency generation
18. Different Ways To Integrate AI Directly in Mobile Apps 01 Gemini API 02 Mediapipe / LLMInterference Library and Offline Model Can be used with any tflite / LiteRT Models, not Gemma Specific 03 Gemini Nano Currently Experimental, available only on Pixel 9 Devices Either Directly with GeminiAPI or By Using The Third Party Library by Shreyas 04 Firebase Vertex AI You can use Gemini APIs and models with Firebase Vertex API, reducing the need for handling intricate details yourself Koog You can use Koog, along with various LLM providers, and custom/inbuilt tools to build AI agents and use them directly in Mobile Apps 05
19. Google Generative AI SDK for Kotlin Multiplatform by Shreyas Patil - hps://github.com/PatilShrey as/generative-ai-kmp API key stored in BuildKonfig Suspend function for story generation Works on Android & iOS GEMINI INTEGRATION (ONLINE)
20. GENERATIVEMODEL IMPLEMENTATION (GEMINI) class GenerativeModelGemini(private val apiKey: String) : GenerativeModel { private val model by lazy { GeminiApiGenerativeModel( ... ) } override suspend fun generate(prompt: String, awaitReadiness: Boolean): Result<String> { return runCatching { val input = content { text(prompt) } val response = model.generateContent(input) response.text ?: throw UnsupportedOperationException("No text returned from model") } } } commonMain.dependencies { implementation("dev.shreyaspatil.generativeai:generativeai-google:<version>") } hps://github.com/PatilShreyas/generative-ai-kmp
21. GENERATIVEMODEL IMPLEMENTATION (GEMINI) hps://github.com/PatilShreyas/generative-ai-kmp GeminiApiGenerativeModel( modelName = "gemini-2.5-flash", apiKey = apiKey, generationConfig = GenerationConfig.Builder().apply { topK = 40 } .build() )
22. Gemini Models
23. 01 USES MEDIAPIPE GENAI 02 TEXTGENERATOR EXPECT/ACTUAL for platform-specific code 03 LOCALGENERATIVEMODEL wraps the logic OFFLINE MODE WITH GEMMA
24. DOWNLOA D .TASK FILE FROM SERVER or Pack with App (not recommend ed) STORE IN INTERNAL APP DIRECTORY INIT MEDIAPIPE LLM AFTER DOWNLOAD COMPLETES MODEL DOWNLOAD & INITIALIZATION
25. Gemma Models hps://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference That are supported by LLMInterference API 550+ MB
26. Download .task file and Store it in App Directory (Android Code) https://huggingface.co/google/gemma-3-1b-it val request = DownloadManager.Request(modelUrl.toUri()) .setNotificationVisibility(DownloadManager.Request.VISIBILITY_VISIBLE) // Visibility of the download Notification .setDestinationUri(Uri.fromFile(modelFile)) // Uri of the destination file .setDescription("Downloading Gemma 3 Model") // Title of the Download Notification .setTitle("Downloading The Model") // Description of the Download Notification .setRequiresCharging(false) // Set if charging is required to begin the download .setAllowedOverMetered(true) // Set if download is allowed on Mobile network .setAllowedOverRoaming(true) // Set if download is allowed on roaming network val downloadManager = context.getSystemService(DOWNLOAD_SERVICE) as DownloadManager
27. Init MediaPipe LLM Interference private val llmInference: LlmInference by lazy { val options = LlmInference.LlmInferenceOptions.builder() .setModelPath(modelFile.absolutePath) .setMaxTokens(1024) .setMaxTopK(40) .build() LlmInference.createFromOptions(context, options) }
28. Use The Model actual suspend fun generate(prompt: String): String { val result = withContext(Dispatchers.IO) { llmInference.generateResponse(prompt) } return result ?: throw IllegalStateException("Model didn't generate") }
29. Gemma 3 1b Model Download
30. Generation Settings GeminiApiGenerativeModel( modelName = "gemini-2.0-flash", apiKey = apiKey, generationConfig = GenerationConfig.Builder().apply { topK = 40 } .build() ) private val llmInference: LlmInference by lazy { val options = LlmInference.LlmInferenceOptions.builder() .setModelPath(modelFile.absolutePath) .setMaxTokens(512) .setMaxTopK(40) .build() LlmInference.createFromOptions(context, options) }
31. TopK • Top-K filters tokens for output. • For example a Top-K of 3 keeps the three most probable tokens. • Increasing the Top-K value will increase the randomness of the model response.
32. TopK • Top-K filters tokens for output. • For example a Top-K of 3 keeps the three most probable tokens. • Increasing the Top-K value will increase the randomness of the model response.
33. TopK • Top-K filters tokens for output. • For example a Top-K of 3 keeps the three most probable tokens. • Increasing the Top-K value will increase the randomness of the model response.
34. maxTokens • Limits the maximum output length a model can generate • A token can be a whole word, part of a word (like "ing" or "est"), punctuation, or even a space. The exact way text is tokenized depends on the specific model’s tokenizer. • Whenever we call llmInference.generateResponse(prompt), the response generated by the local model will contain at most 512 tokens.
35. maxTokens • Limits the maximum output length a model can generate • A token can be a whole word, part of a word (like "ing" or "est"), punctuation, or even a space. The exact way text is tokenized depends on the specific model’s tokenizer. • Whenever we call llmInference.generateResponse(prompt), the response generated by the local model will contain at most 512 tokens.
36. Gemini Nano
37. Gemini Nano
38. Gemini Nano Source: hps://deepmind.google/technologies/gemini/nano/
39. Integrate Gemini Nano implementation("com.google.ai.edge.aicore:aicore:0.0.1-exp02")
40. Integrate Gemini Nano generativeModel = try { val generationConfig = generationConfig { context = getApplication<Application>().applicationContext temperature = 0.2f topK = 40 maxOutputTokens = 1024 } val downloadConfig = DownloadConfig( object : DownloadCallback { isReady.update { true } } ) GenerativeModel( generationConfig = generationConfig, downloadConfig = downloadConfig ) } catch (e: Exception) { Log.e("MainViewModel", "Failed to initialize AI Core: ${e.message}") null }
41. Integrate Gemini Nano val generationConfig = generationConfig { context = getApplication<Application>().applicationContext temperature = 0.2f topK = 40 maxOutputTokens = 1024 }
42. Integrate Gemini Nano val downloadConfig = DownloadConfig( object : DownloadCallback { isReady.update { true } } )
43. Integrate Gemini Nano GenerativeModel( generationConfig = generationConfig, downloadConfig = downloadConfig )
44. Vertex AI Google Recommends using Vertex AI in Firebase SDK for Android to access the Gemini API and the Gemini family of models directly from the app.
45. Vertex AI implementation("com.google.firebase:firebase-vertexai:$version") class GenerativeModelVertex() : GenerativeModel { val generativeModel = Firebase.vertexAI.generativeModel("gemini-2.5-flash") override suspend fun generate(prompt: String, awaitReadiness: Boolean): Result<String> { return runCatching { model.generateContent(prompt) } } }
46. Koog Koog is a Kotlin-based framework designed to build and run AI agents entirely in idiomatic Kotlin. It lets you create agents that can interact with tools, handle complex workflows, and communicate with users. https://docs.koog.ai/
47. Koog
48. Koog implementation("ai.koog:koog-agents:0.5.2") suspend fun runAgent(prompt: String): String { return try { val agent = AIAgent( promptExecutor = simpleGoogleAIExecutor(apiKey), llmModel = GoogleModels.Gemini2_5Flash, systemPrompt = systemPrompt ) val result = agent.run(prompt) result } catch (e: Exception) { "Koog Agent Error: ${e.message}\n${e.stackTraceToString()}" } }
49. Koog implementation("ai.koog:koog-agents:0.5.2") val agent = AIAgent( promptExecutor = MultiLLMPromptExecutor( GoogleLLMClient(apiKey) ), llmModel = GoogleModels.Gemini2_5Flash, systemPrompt = systemPrompt )
50. Koog
51. Koog = AI Agents in Kotlin
52. Image Source: https://cobusgreyling.medium.com/whats-your-definition-of-an-ai-agent-edb7d5e1c760
53. Koog - Tools Agents use tools to perform specific tasks or access external systems.
54. Koog - Tools Agents use tools to perform specific tasks or access external systems. expect class DatabaseOperationsToolSet( repository: YourRepository ) { suspend fun someDBOperation(): Result /** * Convert to Koog tools */ fun toTools(): List<Tool<*, *>> }
55. Koog - Tools Android/JVM @LLMDescription("Meaningfull description of the class/toolset") expect class DatabaseOperationsToolSet( repository: YourRepository ) : ToolSet { { @Tool @LLMDescription("Meaningfull description of the function/operation") suspend fun someDBOperation(): Result {...} fun toTools(): List<Tool<*, *>> {...} }
56. Koog - Tools Android/JVM actual fun toTools(): List<Tool<*, *>> { return this.asTools() }
57. Koog - Tools Android/JVM
58. Koog - Tools Common val dbToolRegistry = ToolRegistry { tools(databaseToolSet.toTools()) }
59. Koog - Tools Common private fun createAgent(): AIAgent<String, String> { return AIAgent( promptExecutor = MultiLLMPromptExecutor( GoogleLLMClient(apiKey) ), systemPrompt = """ ---Your System Prompt--- """.trimIndent(), llmModel = GoogleModels.Gemini2_5FlashLite, toolRegistry = dbToolRegistry ) }
60. Koog - Strategy Common val agentStrategy = strategy("bmi calculator strategy") { val firstNode by node<Input, Output> { // Transform input to output } val secondNode by node<Output, Output> { // Transform output to output } // Node will get the value of type Output as input from the previous node and path through it to the next node val setupContext by nodeAppendPrompt<Output>("setupContext") { system(systemPrompt) user(userPrompt) } edge(firstNode forwardTo setupContext) edge(setupContext forwardTo secondNode) }
61. COCOAPODS INTEGRATION FOR IOS Cocoapods integration for iOS is deprecated MEDIAPIPE GENAI MediaPipe GenAI supports Android, iOS and Web, however integrating it with KMP is challenging CHALLENGES FACED Koog Agents Using Koog makes building and using agents or even just calling any prominent LLM very easy in Mobile or any platform
62. It’s easy to integrate GenAI with your KMP apps LLM Interference / MediaPipe works but its’ not for most of the usecases Code reusability across platforms with KMP KEY TAKEAWAYS Gemini Nano can be a game changer Koog makes it even easier
63. hps://github.com/RivuChk/GolpoAI hps://github.com/PatilShreyas/generative-ai-kmp RESOURCES hps://docs.koog.ai https://speakerdeck.com/rivuchk/beyond-code-sharing-building-s marter-mobile-apps-with-kmp-and-agents-devfest-gandhinagar-2 025 hps://medium.com/@brianmwangi_dev/building-ai-agents-in-kotlin-a-d eep-dive-into-koog-d279915be5c1
64. PAUSE & THINK
65. THANK YOU 🌐 https://www.rivu.dev/ youtube.com/@rivutalks @rivuchakraborty