Tweet
Tweet

Slide 1

Slide 1 text

# Android Gemini In Android Studio ߓ೙઱ | GDG Korea Android

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Gemini API on Android • Gemini Models & AICore • Gemini Nano use cases on Android • Integration & SDKs Gemini in Android Studio • Code Suggestions • App Quality Insights • Multimodal inputs (Preview) Demo Gemini in Android Studioܳ ഝਊ೧ ࢲ Android Application ѐߊೞӝ ೠ ׀ী ࠁӝ Agenda

Slide 4

Slide 4 text

Gemini API on Android Android •Gemini Models & AICore •Gemini Nano use cases on Android •Integration & SDKs

Slide 5

Slide 5 text

Gemini Models On-device ੘সਸ ୊ܻ ೞחؘ ੓যࢲ о੢ ബҗ ੸ੋ ݽ؛ ׮নೠ ੘সী Ѧ୛ ࢎਊ ೞӝ જ਷ ୭࢚੄ ݽ؛ Gemini Nano Gemini Pro Ҋب۽ ࠂ੟ೠ ੘সਸ ୊ܻ ೞӝ ਤೠ о੢ ъ۱ೠ ݽ؛ Gemini Ultra

Slide 6

Slide 6 text

Gemini Models •Android 14+੄ AI Core ۽ ੸ਊ •Google AI Edge SDKܳ ా೧ API ੽Ӕ •Pixel 8 Pro৬ Samsung S24 दܻૉীࢲ ࢎਊ оמ •ੋఠ֔ োѾ হ੉ب ੘ز •ѐੋ ੿ࠁ ࠁഐ •ߔূ٘ ੋ೐ۄ ҙܻ ೙ਃࢿ ઁѢ On-device ੘সਸ ୊ܻ ೞחؘ ੓যࢲ о੢ ബҗ ੸ੋ ݽ؛ Gemini Nano

Slide 7

Slide 7 text

Gemini Nano ׮নೠ హਵ۽ ݫद૑ ਽׹ೞחؘ ب਑ • Magic Rewrite • Suggested Text ਺ࢿ ֣਺ ӝ۾੄ झ݃౟ ਃড ࢤࢿ • Highlight main topics of voice transcripts(English only) • Powered via Gemini Nano on eligible devices • Available offline + data processed on-device Messages Recorder ఃࠁ٘ীࢲ ޙ੗ ݫद૑ী ؀ೠ झ݃౟ ਽׹ ઁউ GBoard

Slide 8

Slide 8 text

AI Core Android on-device AI under the hood, I/O 24 Client app AI Edge SDK LoRA Gemini Nano Safety features AI Core TPU / NPU / APU High-level inference architecture •AICore ח System Imageী ನೣ •Google Playܳ ా೧ ૑ࣘ੸ਵ۽ সؘ੉౟ •জীࢲח AI Edge SDK۽ ੽Ӕ •Low-Rank Adaptation(LoRA) ۨ੉য۽ জ ѐߊ੗о ݽ؛ ݏ୺ ࢸ੿ оמ https://developer.android.com/ai/aicore

Slide 9

Slide 9 text

AI Core Android on-device AI under the hood, I/O 24 Why is Fine-tuning important? •೐܁೐౟ ূ૑פয݂݅ਵ۽ח on-device ݽ؛ীࢲ ୽࠙ೞ૑ ঋ਺ •ઁড ઑѤ : model size, context window •LoRA Fine-tuningਸ ా೧ ੘਷ ݽ؛੉ ౠ੿ ࢎਊ ࢎ۹ীࢲ ڪযդ ࢿמਸ ߊൃೡ ࣻ ੓ب۾ ೣ Client app AI Edge SDK Gemini Nano Safety features AI Core TPU / NPU / APU LoRA https://developer.android.com/ai/aicore

Slide 10

Slide 10 text

Gemini API Integrate Gemini API Google AI Client SDK Vertex AI SDK for Firebase (Preview) (Preview) Add Generative AI to your Android app with the Gemini API, I/O 24

Slide 11

Slide 11 text

Gemini API h tt ps://developer.android.com/ai/generativeai#reference-apis-sdks Google APIs and SDKs for Gen AI on Android

Slide 12

Slide 12 text

Gemini API Integration 4ױ҅ Add Generative AI to your Android app with the Gemini API, I/O 24 Google AI Studioীࢲ ೐܁೐౟ܳ ೐۽షఋ੉ೝ Google AI Studioীࢲ API ః ߊә 01 02 ೐۽ં౟ী Google AI Client SDKী ؀ೠ ઙࣘࢿ ୶о Gemini Proܳ Kotlin ௏٘ী ా೤ 03 04

Slide 13

Slide 13 text

Google AI Studio Gemini API

Slide 14

Slide 14 text

Kotlin Code Gemini API < > Get Code

Slide 15

Slide 15 text

val model = GenerativeModel( modelName = "gemini-1.5-pro", apiKey = BuildConfig.apiKey, generationConfig = generationConfig { temperature = 1f topK = 64 topP = 0.95f maxOutputTokens = 8192 }, safetySettings = listOf( SafetySetting(HarmCategory.DANGEROUS_CONTENT, BlockThreshold.MEDIUM_AND_ABOVE), ) ) https://ai.google.dev/gemini-api/docs/safety-settings https://github.com/google/secrets-gradle-plugin

Slide 16

Slide 16 text

val model = GenerativeModel( modelName = "gemini-1.5-pro", apiKey = BuildConfig.apiKey, generationConfig = generationConfig { temperature = 1f topK = 64 topP = 0.95f maxOutputTokens = 8192 }, safetySettings = listOf( SafetySetting(HarmCategory.DANGEROUS_CONTENT, BlockThreshold.MEDIUM_AND_ABOVE), ) ) https://ai.google.dev/gemini-api/docs/safety-settings https://github.com/google/secrets-gradle-plugin

Slide 17

Slide 17 text

val model = GenerativeModel( modelName = "gemini-1.5-pro", apiKey = BuildConfig.apiKey, generationConfig = generationConfig { temperature = 1f topK = 64 topP = 0.95f maxOutputTokens = 8192 }, safetySettings = listOf( SafetySetting(HarmCategory.DANGEROUS_CONTENT, BlockThreshold.MEDIUM_AND_ABOVE), ) ) https://ai.google.dev/gemini-api/docs/safety-settings https://github.com/google/secrets-gradle-plugin

Slide 18

Slide 18 text

val model = GenerativeModel( modelName = "gemini-1.5-pro", apiKey = BuildConfig.apiKey, generationConfig = generationConfig { temperature = 1f topK = 64 topP = 0.95f maxOutputTokens = 8192 }, safetySettings = listOf( SafetySetting(HarmCategory.DANGEROUS_CONTENT, BlockThreshold.MEDIUM_AND_ABOVE), ) ) https://ai.google.dev/gemini-api/docs/safety-settings https://github.com/google/secrets-gradle-plugin

Slide 19

Slide 19 text

val model = GenerativeModel( modelName = "gemini-1.5-pro", apiKey = BuildConfig.apiKey, generationConfig = generationConfig { temperature = 1f topK = 64 topP = 0.95f maxOutputTokens = 8192 }, safetySettings = listOf( SafetySetting( HarmCategory.DANGEROUS_CONTENT, BlockThreshold.MEDIUM_AND_ABOVE ) ) ) https://ai.google.dev/gemini-api/docs/safety-settings https://github.com/google/secrets-gradle-plugin

Slide 20

Slide 20 text

scope.launch { val response = model.generateContent( content { image(bitmap) text("What is the object in this picture?") } ) } Prompting https://developer.android.com/ai/google-ai-client-sdk

Slide 21

Slide 21 text

scope.launch { val response = model.generateContentStream( content { image(bitmap) text("What is the object in this picture?") } ).collect { response -> outputContent += response.text } } Prompting https://developer.android.com/ai/google-ai-client-sdk

Slide 22

Slide 22 text

Gemini API New Project Template File > New Project > Gemini API Starter

Slide 23

Slide 23 text

Gemini API Onboarding tool Prototyping SDKs Google AI Studio ীࢲ ೐܁೐౟ܳ ࢸ҅೤פ׮. Gemini API ܳ জী ୶о೤פ׮. Google AI Client SDK ܳ ా೤೤פ׮. Vertex AI SDK for Firebase ܳ दب೧ ࠇפ׮.

Slide 24

Slide 24 text

Gemini in Android Studio Android Studio •Code Suggestions •App Quality Insights •Multimodal inputs (Preview)

Slide 25

Slide 25 text

Android Studio 2024.1.3 Canary 1 Ladybug ੉ߣ ࣁ࣌ীࢲ ࢎਊػ Android Studio ߡ੹ What’s new in preview(link)

Slide 26

Slide 26 text

Gemini ৬ ҕਬೡ ೐۽ં౟ ࢶఖೞӝ Context Awareness Control what you share

Slide 27

Slide 27 text

Gemini৬ ੹୓ ҕਬ Context Awareness

Slide 28

Slide 28 text

അ੤ ೐۽ં౟ ҕਬೡ Ѥ૑ ೐۽ં౟݃׮ ޛযࠁӝ Context Awareness

Slide 29

Slide 29 text

೐۽ં౟ ௏٘ ҕਬೞ૑ ঋ਺ Context Awareness

Slide 30

Slide 30 text

Context Awareness .aiexclude ౵ੌ۽ ஶఫझ౟ ҕਬ ҳࢿ https://developer.android.com/studio/preview/gemini/aiexclude .gitignore ৬ زੌೠ ҳޙਵ۽ ੘ࢿ Intention actions ৬ code completion ನೣ

Slide 31

Slide 31 text

Context Awareness അ੤ ࠁҊ ੓ח ౵ੌ੉ AI ࢎਊ оמೠ ૑ ഛੋೞӝ

Slide 32

Slide 32 text

Context Awareness അ੤ ࠁҊ ੓ח ౵ੌ੉ AI ࢎਊ оמೠ ૑ ഛੋೞӝ

Slide 33

Slide 33 text

Context Awareness Gemini ࢎਊ Gemini ࢎਊ X

Slide 34

Slide 34 text

Gemini ହী ੑ۱ೠ ղਊ਷ ࢲߡ۽ ੹࣠ Context Awareness

Slide 35

Slide 35 text

Code Suggestion ௏٘ ੗ز ৮ࢿ https://developer.android.com/studio/preview/gemini/ai-code-completion Tab ఃܳ ׂ۞ ୶ୌਸ ࣻۅೞҊ Escܳ ׂ۞ ୶ୌਸ ૑਒פ׮. ౵ۄ޷ఠ۽ ߉਷ obj ੄ ࣘࢿ ч੉ String ژח Int ੌ ҃਋ ୹۱೧઱Ҋ, Ӓ ৻ীח “Unknown” ਵ۽ ୹۱

Slide 36

Slide 36 text

Code Suggestion Transform select code ࢶఖೠ ௏٘ ߸ജ Improve readability

Slide 37

Slide 37 text

Code Suggestion Suggest Improvements ѐࢶ ઁউ Make suggestions to improve this code: Code Block What would you like to do?

Slide 38

Slide 38 text

Code Suggestion • Copy • Insert at Cursor • Explore in Playground

Slide 39

Slide 39 text

Code Suggestion Explore in Playground

Slide 40

Slide 40 text

Code Suggestion Document Function

Slide 41

Slide 41 text

Code Suggestion ୶о۽ ѐࢶೞҊ र׮ݶ?

Slide 42

Slide 42 text

Code Suggestion Refine prompt “Describes receiving job types”

Slide 43

Slide 43 text

Code Suggestion Transformation Prompt History

Slide 44

Slide 44 text

Test Scenarios Test Scenarios ਬ׫ పझ౟ ੘ࢿ What's new in Android development tools, I/O 24

Slide 45

Slide 45 text

Test Scenarios Test Scenarios ਬ׫ పझ౟ ࢤࢿ Ѿҗ What's new in Android development tools, I/O 24

Slide 46

Slide 46 text

Test Scenarios // HomeViewModel.kt /** * Notify that the user updated the search query */ fun onSearchInputChanged(searchInput: String) { viewModelState.update { it.copy(searchInput = searchInput) } } ੘ࢿػ పझ౟ दաܻয় ࢎ۹ - ਗࠄ ೣࣻ What's new in Android development tools, I/O 24

Slide 47

Slide 47 text

Test Scenarios ੘ࢿػ పझ౟ दաܻয় ࢎ۹ - ࢤࢿػ ਬ׫ పझ౟ // Test that searchInput in viewModelState is updated with the provided searchInput @Test fun onSearchInputChanged_StateUpdate_EmptyString() { } @Test fun onSearchInputChanged_StateUpdate_SpecialCharacters() { } @Test fun onSearchInputChanged_StateUpdate_LongString() { } What's new in Android development tools, I/O 24

Slide 48

Slide 48 text

Test Scenarios ੘ࢿػ పझ౟ दաܻয় ࢎ۹ - ࢤࢿػ ਬ׫ పझ౟ // Test that searchInput in viewModelState is updated with the provided searchInput @Test fun onSearchInputChanged_StateUpdate_EmptyString() { } @Test fun onSearchInputChanged_StateUpdate_SpecialCharacters() { } @Test fun onSearchInputChanged_StateUpdate_LongString() { } What's new in Android development tools, I/O 24

Slide 49

Slide 49 text

Test Scenarios ੘ࢿػ పझ౟ दաܻয় ࢎ۹ - ࢤࢿػ ਬ׫ పझ౟ // Test that searchInput in viewModelState is updated with the provided searchInput @Test fun onSearchInputChanged_StateUpdate_EmptyString() { } @Test fun onSearchInputChanged_StateUpdate_SpecialCharacters() { } @Test fun onSearchInputChanged_StateUpdate_LongString() { } What's new in Android development tools, I/O 24

Slide 50

Slide 50 text

Commit Message Suggest Commit Message ழ޿ ݫद૑ ઁউ

Slide 51

Slide 51 text

Commit Message

Slide 52

Slide 52 text

Resource Translation Resource ߣ৉ೞӝ

Slide 53

Slide 53 text

Resource Translation Resource ߣ৉ೞӝ strings.xml > Gemini > Transform select code > “Translate to Korean” Gemini ؀ച ହী ֍যب Resource݅ ߣ৉ػ Ѿҗޛਸ ঳ਸ ࣻ ੓׮.

Slide 54

Slide 54 text

Resource Translation Resource ߣ৉ೞӝ

Slide 55

Slide 55 text

App Quality Insights Explain this exception from my app running on Google sdk_gphone64_arm64 with Android version 15: Exception: “Prompt” “Show insights”

Slide 56

Slide 56 text

Show insights Crash Analysis Report • The Exception • Location of the Exception • The Call Stack • How to Fix

Slide 57

Slide 57 text

@Composable fun Greeting(name: String, modifier: Modifier = Modifier) { Column { Text( modifier = modifier .clickable { onTextClicked() }, text = "Click", ) } } fun onTextClicked() { throw RuntimeException("Test exception") } Crash ਬߊ ௏٘

Slide 58

Slide 58 text

೧Ѿ ߑߨ • onTextClicked Ѩష: MainActivity.kt ౵ੌ੄ 55ߣ૩ ઴۽ ੉زೞৈ onTextClicked ೣࣻ ղ੄ ௏٘ܳ ઱੄ Өѱ Ѩష೤פ׮. RuntimeExceptionਸ ੌਵఆ ࣻ ੓ח ֤ܻо ੓ח૑ ഛੋ ೤פ׮. • ٣ߡӒ: Android Studio੄ ٣ߡѢܳ ࢎਊೞৈ ௏٘ प೯ਸ ױ҅߹۽ ૓೯ೞҊ ৘৻о ߊࢤೞ ח ੿ഛೠ ૑੼ਸ ध߹೤פ׮. ߸ࣻ чҗ प೯ ൒ܴਸ Ѩࢎೞৈ ࢚ടਸ ੉೧೤פ׮. • Ҋ੄੸ੋ ৘৻ ഛੋ: పझ౟ ݾ੸ਵ۽ ৘৻ܳ Ҋ੄۽ ߊࢤदఅ ҃਋, ೧׼ ௏٘ܳ ઁѢೞѢա ઑ ੿೤פ׮. • ৘৻ ୊ܻ: ৘৻о Ҋ੄੸ੋ Ѫ੉ ইצ ҃਋, try-catch ࠶۾җ э਷ ৘৻ ୊ܻܳ ୶оೞৈ ࢚ട ਸ ਋ইೞѱ ୊ܻೞҊ জ੉ ୽جೞ૑ ঋب۾ ೤פ׮. 😎

Slide 59

Slide 59 text

Coming soon Function Name ୶ୌ Shift + F6 Rename What's new in Android development tools, I/O 24 Gemini > Rethink variable names

Slide 60

Slide 60 text

Coming soon + = Design.png Compose code Gemini LongContext Multimodal Input preview

Slide 61

Slide 61 text

Geminiীѱ Compose code ࢤࢿ दఃӝ Demo Demo

Slide 62

Slide 62 text

Jetnews Sample app https://github.com/android/compose-samples [рױೠ ಕ੉૑] • Ѩ࢝যܳ ੑ۱ೞח Textfield • Ӓ ইې ୶ୌ ఃਕ٘ ࢜۽਍ Ѩ࢝ ಕ੉૑ܳ ٜ݅যࠁ੗ Demo

Slide 63

Slide 63 text

Demo ਃ୒ࢎ೦җ ಁః૑ ૑੿ Generate a Jetpack Compose `SearchScreen` composable in package `com.example.jetnews.ui.search`.

Slide 64

Slide 64 text

Demo Layout ࢸݺ [Layout] - A Material 3 `TopAppBar` (all padding 10.dp) containing an `OutlinedTextField` with a search icon. - Below, a `LazyColumn` (padding 10.dp) displaying clickable hardcoded search recommendations from 1 to 10.

Slide 65

Slide 65 text

Demo ӝמ ࢸݺ [Functionality] - The search icon should trigger `submitSearch(query: String)` in `SearchScreen` using the text from the `OutlinedTextField`. - Clicking a recommendation should also trigger `submitSearch` in `SearchScreen`.

Slide 66

Slide 66 text

Demo ઁೠ ࢎ೦ ࢸݺ Include all necessary impo rt s with preview composable. Don’t omit code, provide the full solution. Do not include any previous response as context.

Slide 67

Slide 67 text

Demo Prompt Generate a Jetpack Compose `SearchScreen` composable in package `com.example.jetnews.ui.search`. [Layout] - A Material 3 `TopAppBar` (all padding 10.dp) containing an `OutlinedTextField` with a search icon. - Below, a `LazyColumn` (padding 10.dp) displaying clickable hardcoded search recommendations from 1 to 10. [Functionality] - The search icon should trigger `submitSearch(query: String)` in `SearchScreen` using the text from the `OutlinedTextField`. - Clicking a recommendation should also trigger `submitSearch` in `SearchScreen`. Include all necessary impo rt s with preview composable. Don’t omit code, provide the full solution. Do not include any previous response as context.

Slide 68

Slide 68 text

Demo ࢤࢿ Ѿҗ

Slide 69

Slide 69 text

хࢎ೤פ׮. ߓ೙઱ veronikapj GDG Korea Android