Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Creating Custom Models with AutoML and MLKit

Creating Custom Models with AutoML and MLKit

This is a presentation that I gave at DroidCon Greece in 2019


Peter-John Welcome

September 24, 2019

More Decks by Peter-John Welcome

Other Decks in Programming


  1. Creating Custom Models with AutoML and MLKit Peter-John Welcome @pjapplez

    Mobile Engineering Lead
  2. About me

  3. AutoML with MLKit • Create Custom Model • Generate a

    Rest API • Offline Models • Remote serving • Remote Config • User experience • Performance AutoML MLKit Android
  4. AutoML

  5. This is me

  6. What is AutoML?

  7. AutoML

  8. AutoML Rest API

  9. None
  10. AutoML Pros & Cons • Create Custom machine learning models

    without any coding • Able to serve via Rest API • Can export Models for Mobile Devices • Easy to use • No way to do do A/B Testing • No way to serve mobile model dynamically • No easy way to track events through analytics
  11. AutoML with MLKit

  12. MLKit

  13. MLKit: Using the Model

  14. private fun callAutoMLModelLocally() { val localModel = FirebaseLocalModel.Builder("my_local_model") .setAssetFilePath("manifest.json") .build()

    FirebaseModelManager.getInstance().registerLocalModel(localModel) } MLKit: Local model
  15. dependencies { implementation 'com.google.firebase:firebase-ml-vision:23.0.0' implementation 'com.google.firebase:firebase-ml-vision-automl:18.0.1' } apply plugin: 'com.google.gms.google-services'

    MLKit: Dependencies
  16. private fun callAutoMLModelRemotely() { val conditions = FirebaseModelDownloadConditions.Builder() .requireWifi() .build()

    val remoteModel = FirebaseRemoteModel.Builder("Pavonia_Leafshapes") .enableModelUpdates(true) .setInitialDownloadConditions(conditions) .setUpdatesDownloadConditions(conditions) .build() FirebaseModelManager.getInstance().registerRemoteModel(remoteModel) downloadRemoteModel(remoteModel) } MLKit: Remote model
  17. val labelerOptions = FirebaseVisionOnDeviceAutoMLImageLabelerOptions.Builder() .setLocalModelName("my_local_model") .setRemoteModelName("Pavonia_Leafshapes") .setConfidenceThreshold(0F) .build() val labeler

    = FirebaseVision.getInstance() .getOnDeviceAutoMLImageLabeler(labelerOptions) labeler.processImage(image) .addOnSuccessListener { labels -> //Do something with the label array (label.text & label.confidence } MLKit: Using our Model
  18. MLKit: Offline vs Serving • Offline is fast and is

    bundle with the app • Offline models does make your app bigger in size • Offline requires app update to update model • Serving lets you serve new updates of the model • Makes your app smaller to download from the PlayStore
  19. MLKit: Advanced • Remote Config • Firebase Analytics val remoteConfig

    = FirebaseRemoteConfig.getInstance() val remoteConfigDefaults = HashMap<String, Any>() remoteConfigDefaults["plant_labeler_model"] = "plant_labeler_v1" Tasks.await(remoteConfig.setDefaultsAsync(remoteConfigDefaults)) remoteConfig.fetchAndActivate().addOnSuccessListener { success -> if (success) { // Okay to get remote values. // ... } }
  20. Android

  21. MLKit Offline AutoML Rest API AutoML: Performance & UX

  22. AutoML: Pricing

  23. AutoML: Use case

  24. Thanks! @pjapplez