Bringing AI to Android With Gemini

Transcript

1. Bringing AI to Android With Gemini Yves Kalume Google Developer Expert | Android @Moneco
2. Yves Kalume Made in Congo DR ▪ Android Engineer @ Moneco (moneco.app) ▪ Google Developer Expert, Android ▪ www.linktr.ee/yveskalume
3. Finding the perfect Gemini fit on Android
5. Intelligent Apps Intelligent OS
7. Connect with your favorite apps Find me a recipe for Dan Dan noodles and add the ingredients to my shopping list Suggest a good playlist for running Remind me to pickup my package at 5PM today
8. Mobile TV Watch Large Screens Auto XR headsets and glasses
9. Intelligent Apps
11. Text, image GenAI Use case Modality Audio, Video input Image generation…
12. Summarize, proofread, image description… Text, image GenAI Use case Modality Complexity Audio, Video input Image generation… Requires domain knowledge…
13. Short input and output Summarize, proofread, image description… Text, image GenAI Use case Modality Complexity Context Window Audio, Video input Image generation… Requires domain knowledge… Extensive documents, hours of audio/video…
14. Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window
15. Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window
16. Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window
17. Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window
18. On-device GenAI (Gemini Nano) Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window
19. Offline availability No additional cost Local processing Benefits of on-device execution
21. GenAI APIs
22. GenAI APIs built for on-device tasks • Summarization • Proofreading • Rewrite • Image Description
23. Gemini Nano on Android: Building with on-device GenAI Caren Chang Developer Relations Engineer Taj Darra Product Manager Sa-ryong Kang Developer Relations Engineer
25. Play for On-device AI Helping Play developers seamlessly deliver ML models to power on-device AI features. Now Beta http://g.co/play/on-device-ai
26. Firebase AI SDKs (Gemini Flash, Pro, Imagen) On-device GenAI (Gemini Nano) Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window Audio, Video input Image generation… Requires domain knowledge… Extensive documents, hours of audio/video…
27. Firebase AI SDKs (Gemini Flash, Pro, Imagen) On-device GenAI (Gemini Nano) Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window Audio, Video input Image generation… Requires domain knowledge… Extensive documents, hours of audio/video…
28. Gemini Flash, Pro and Imagen for Android
29. Complexity Context window Modality
30. Firebase Gemini API in Vertex AI Gemini Developer API Firebase SDKs
31. Enhance your Android app with Gemini Pro and Flash, and Imagen Thomas Ezan Sr. Developer Relation Engineer Miguel Ramos Sr. Product Manager
32. Choose the right Gemini Fit
33. Context Window Device Reach Complexity Offline availability Cost Modality Local processing
34. Firebase AI SDKs (Gemini Flash, Pro, Imagen) Audio, Video input Image generation… Requires domain knowledge… Extensive documents, hours of audio/video… Audio, Video input Image generation On-device GenAI (Gemini Nano) Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window
35. Firebase AI SDKs (Gemini Flash, Pro, Imagen) On-device GenAI (Gemini Nano) Short input and output Summarize, proofread, rewrite… Text, image GenAI Use case Modality Complexity Context Window Audio, Video input Image generation… Requires domain knowledge… Extensive documents, hours of audio/video…
36. Enhance your app with Gemini Pro and Flash, and Imagen
37. Gemini 2.0 Live API capabilities Image generation with Imagen 3 Gemini Pro and Flash series models for Android
38. Gemini Flash and Pro series for Android
39. Text prompt + Code Image Audio Video Plain Text JSON Image Audio Gemini 2.0 Pro Gemini 2.0 Flash Gemini 2.0 Flash Lite Gemini 2.5 Pro (exp.) …
40. Text 750 000 words (4 Harry Potter books) Gemini 2.0 Flash: 1M tokens Gemini 2.0 Pro has 2M tokens! Video 1 hour Audio 11 hours (~1.5 Harry Potter audiobooks)
41. Vertex AI Gemini API in Vertex AI Google AI Studio Gemini Developer API
42. Firebase Firebase SDKs Gemini API in Vertex AI
43. Firebase Firebase SDKs Gemini API in Vertex AI Gemini Developer API
44. import com.google.firebase.Firebase import com.google.firebase.ai.ai @Composable fun MagicBackpackGenerator() { var output by remember { mutableStateOf("") } LaunchedEffect("") { val generativeModel = Firebase.ai.generativeModel("gemini-2.0-flash") val response = generativeModel. generateContent("Write a story about a magic backpack") output = response.text.toString() } MagicBackpackUI(output) } 01 02 03
45. Meal Planner by Jojo Apps Gemini 2.0 Flash is used to generate meal plans with ingredients and cooking instructions.
46. Right-click and select ‘Replace image’ Voice Recorder by Smart Mobi Tools Gemini analyzes transcripts, identifies key points, and delivers actionable summaries of users recordings.
47. Right-click and select ‘Replace image’ Sleep as Android by Urbandroid Gemini 2.0 Flash is used to identify trends in the user’s sleep patterns and suggests improvements.
48. Remote config Models Prompts Easy updates with Remote Config Remote Config enables you to override in-app variables via the Firebase console (or the Remote Config API). Gemini API in Vertex AI Gemini Developer API
49. Files Accelerate file uploads with Cloud Storage Use Cloud Storage for Firebase to efficiently upload files for your multimodal prompts. Firebase Security Rules provide granular control over file access, ensuring only authorized users can interact with uploaded content.
50. Bad request User request App check Backend Protect your assets with App Check App Check uses Play Integrity to help secure your backend resources by ensuring only authentic app requests from genuine devices are processed.
51. console.firebase.google.com/ Traffic monitoring Monitor app requests within the Firebase console, and keep a close eye on the traffic of your app.
52. Proprietary and Confidential Generative AI can be used for creative writing 6706 3540 1386 16481 798 614 1281 604 12354
53. val response = generativeModel .countTokens("Generative AI can be used for creative writing") println("Total Tokens: ${response.totalTokens}")
54. Gemini Live API
55. Live APIs in Firebase It’s Public Preview in Firebase SDKs Available on Gemini 2.0 Flash Live Stream text, voice, and video into Gemini and get stream voice response back Video Audio Text Speech
56. <Demo Live AI>
57. val model = Firebase.ai.vertexAI.liveModel( modelName = "gemini-2.0-flash-live-preview-04-09", generationConfig = liveGenerationConfig { responseModality = ResponseModality.AUDIO }, systemInstruction = "Your main job is to change the color...", tools = listOf(Tool.functionDeclarations(listOf(changeColorFunction))) ) val session = model.connect(::handler) session.startAudioConversation(::handler) session.stopAudioConversation() 01 02 03 04
58. Imagen 3 for Android
60. User profile avatar Contextual artwork Custom stickers
61. dependencies { [...] // BoM for the Firebase platform implementation(platform("com.google.firebase:firebase-bom:33.12.0")) // Dependency for the Vertex AI in Firebase library implementation("com.google.firebase:firebase-vertexai") }
62. suspend fun generateImage() { val imagenModel = Firebase.ai.vertexAI.imagenModel("imagen-3.0-generate-002") val imageResponse = imagenModel.generateImages("An astronaut riding a horse") val image = imageResponse.images.first() val bitmapImage = image.asBitmap() } 01 02 03
63. imagen-3.0-generate Generates realistic, high-quality images from natural language text prompts. imagen-3.0-fast-generate Generates images for prototyping or low-latency use cases.
64. console.cloud.google.com/vertex-ai/studio Vertex AI Studio Experiment and refine your image prompts.
65. A Small Recap
66. Imagen 3 through Vertex AI in Firebase is in public preview Gemini Live API is ready for prototyping conversational AI experiences Gemini Cloud models via Vertex AI Firebase are production ready
67. firebase.google.com/docs/vertex-ai firebase.google.com/docs/vertex-ai
68. developer.android.com/ai d.android.com/ai
70. d.android.com/ai
71. Conclusion
72. Thank you Yves Kalume Google Developer Expert | Android @Moneco