Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Get started with the Gemini API in Android apps...

Love
May 20, 2024

Get started with the Gemini API in Android apps (client SDK)

Learn how to build mobile applications using Google's Gemini API in Kotlin.

Love

May 20, 2024
Tweet

More Decks by Love

Other Decks in Technology

Transcript

  1. Love Otudor Get started with the Gemini API in Android

    apps (client SDK) Android Developer, FrontendLabs @lamoureSparus
  2. Google Cloud Proprietary & Confidential Set up your project and

    your API key Generate text from text-only input 01 02 03 04 In this codelab, you'll learn how to: 2 Build multi-turn conversations (chat) Generate text from text-and-image input (multimodal) Build with AI
  3. Prerequisites Build with AI 1. Familiar with using Android Studio

    to develop Android apps. 2. Android Studio (latest version) 3. Your Android app must target API level 21 or higher. The client SDK for Android described in this tutorial lets you access the Gemini Pro models which run on Google's servers. For use cases that involve processing sensitive data, offline availability, or for cost savings for frequently used user flows, you may want to consider accessing Gemini Nano which runs on-device. For more details, refer to the Android (on-device) tutorial.
  4. Google Cloud Proprietary & Confidential Prerequisites 6 Build with AI

    1. Familiar with using Android Studio to develop Android apps. 2. Android Studio (latest version) 3. Your Android app must target API level 21 or higher.
  5. Google Cloud Proprietary & Confidential Set up your project and

    your API key 8 Build with AI 1. Create a key in Google AI Studio. bit.ly/GeminiApiKey
  6. Google Cloud Proprietary & Confidential Set up your project and

    your API key 9 Build with AI 2. Click on create API key.
  7. Google Cloud Proprietary & Confidential Set up your project and

    your API key 10 Build with AI 3. Create an API key in a new project if you don't have one already Or Select a project from your existing write-access Google Cloud projects. 4. Copy your API key for use.
  8. Google Cloud Proprietary & Confidential Set up your project and

    your API key 11 Build with AI 5. Scan the starter project. bit.ly/GeminiAndroid
  9. Google Cloud Proprietary & Confidential Set up your project and

    your API key 13 Build with AI 6a. Add Secrets Gradle plugin for Android. In your project's root build.gradle file: We are using Gradle version catalogs to maintain dependencies and plugins in a scalable way.
  10. Google Cloud Proprietary & Confidential Set up your project and

    your API key 14 Build with AI 6b. Add Secrets Gradle plugin for Android. In your app-level build.gradle file:
  11. Google Cloud Proprietary & Confidential Set up your project and

    your API key 15 Build with AI 7. Enable build config in your module's build.gradle:
  12. Google Cloud Proprietary & Confidential Set up your project and

    your API key 16 Build with AI 8. Add and Secure your API key. bit.ly/SecureApiKey
  13. Google Cloud Proprietary & Confidential Generate text from text-only input

    18 Build with AI In TextViewModel: 1. Initialize the Generative Model using gemini-1.0-pro as modelName. 2. Access your API key as a Build Configuration variable, using BuildConfig.apiKey
  14. Google Cloud Proprietary & Confidential Generate text from text-only input

    19 Build with AI 3. Create a function summarize, that gets the inputText to be summarized as String: a. Create a String value prompt of "Summarize the following text for me: " and append the inputted string to it. b. In a coroutine, try to get the generated response by calling generativeModel.generateContent(prompt). c. Update the UiState with the response.
  15. Google Cloud Proprietary & Confidential 21 Build with AI In

    TextScreen() composable: 4. On click of the Submit button, call Gemini to summarize the text from prompt. 5. Update the UI accordingly. 6. Check out Solution_1_Generate-text-from-text-only-input branch for the full code. Generate text from text-only input
  16. Prerequisites Build with AI 1. Familiar with using Android Studio

    to develop Android apps. 2. Android Studio (latest version) 3. Your Android app must target API level 21 or higher. Prompts using the Gemini API cannot exceed 20MB in size. The Gemini API provides a File API for temporarily storing media files for use in prompting, which lets you provide prompt data beyond the 20MB limit. For more information on using the Files API and file formats supported for prompting, see Prompting with media files.
  17. Google Cloud Proprietary & Confidential Generate text from text-and-image input

    (multimodal) 25 Build with AI In PhotoReasoningViewModel: 1. Initialize the Generative Model using gemini-1.0-pro-vision-latest as modelName. 2. Access your API key as a Build Configuration variable, using BuildConfig.apiKey
  18. Google Cloud Proprietary & Confidential 26 Build with AI 3.

    Create a function reason, that gets the userInput text and selectedImages as a list of bitmap: a. Create a string value prompt of "Look at the image(s), and then answer the following question: " and append the inputted string to it. b. Create an inputContent value for the model, adding each image and the prompt, using content { }. Generate text from text-and-image input (multimodal)
  19. Google Cloud Proprietary & Confidential 27 Build with AI 3.

    Cont. c. In a coroutine, try to get the generated response by calling generativeModel.generateContent(inputContent). d. Update the UiState with the response. Generate text from text-and-image input (multimodal)
  20. Google Cloud Proprietary & Confidential 29 Build with AI In

    PhotoScreen composable: 4. Call the reason function in the onReasonClicked lambda of PhotoReasoningScreen and pass in the selected images and input text. 5. Update the UI accordingly. 6. Check out Solution_2_Multimodal branch for the full code. Generate text from text-and-image input (multimodal)
  21. Google Cloud Proprietary & Confidential Build multi-turn conversations (chat) 31

    Build with AI In ChatViewModel: 1. Initialize the Generative Model using gemini-1.0-pro as modelName. 2. Access your API key as a Build Configuration variable, using BuildConfig.apiKey
  22. Google Cloud Proprietary & Confidential 32 Build with AI 3.

    (Optional) Create a default chat history, which is a list of Content like so: Build multi-turn conversations (chat)
  23. Google Cloud Proprietary & Confidential 33 Build with AI 4.

    Create a chat instance value called chat, to track the ongoing conversation by calling generativeModel.startChat(defaultHistory). You can pass in the defaultHistory if you created one. Build multi-turn conversations (chat)
  24. Google Cloud Proprietary & Confidential 34 Build with AI 5.

    Create a function sendMessage, that has a String parameter called userMessage: a. Update the UiState by adding a pending User ChatMessage with userMessage as its text. By calling _uiState.value.addMessage(ChatMessage(...)). Build multi-turn conversations (chat)
  25. Google Cloud Proprietary & Confidential 35 Build with AI 5.

    Cont. b. In a coroutine, try to get the generated response by calling chat.sendMessage(userMessage). c. Call the replaceLastPendingMessage() to update the last users message as not pending. Build multi-turn conversations (chat)
  26. Google Cloud Proprietary & Confidential 36 Build with AI 5.

    Cont. d. If the generated response text is not null, update the UiState with a Model ChatMessage having the generated response as its text. _uiState.value.addMessage(ChatMessage(...)). Build multi-turn conversations (chat)
  27. Google Cloud Proprietary & Confidential 37 Build with AI 5.

    Cont. e. In your catch block, call: replaceLastPendingMessage() and update the Uistate by adding an Error ChatMessage with the exception’s localizedMessage as its text. _uiState.value.addMessage(ChatMessage(...)). Build multi-turn conversations (chat)
  28. Google Cloud Proprietary & Confidential 39 Build with AI In

    ChatScreen composable: 6. In the ChatList composable, replace the dummyChatHistory with the messages from the UiState. 7. Call the chatViewmodel’s sendMessage function in the onSendMessage lambda of the MessageInput composable and pass in the input text. Build multi-turn conversations (chat)
  29. Google Cloud Proprietary & Confidential 40 Build with AI In

    ChatScreen composable: 8. Check out Solution_3_Chat branch for the full code. Build multi-turn conversations (chat)