Creating Custom Models with AutoML and MLKit

Slide 1

Slide 1 text

Creating Custom Models with AutoML and MLKit Peter-John Welcome @pjapplez Mobile Engineering Lead

Slide 2

Slide 2 text

About me

Slide 3

Slide 3 text

AutoML with MLKit ● Create Custom Model ● Generate a Rest API ● Offline Models ● Remote serving ● Remote Config ● User experience ● Performance AutoML MLKit Android

Slide 4

Slide 4 text

AutoML

Slide 5

Slide 5 text

This is me

Slide 6

Slide 6 text

What is AutoML?

Slide 7

Slide 7 text

AutoML

Slide 8

Slide 8 text

AutoML Rest API

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

AutoML Pros & Cons ● Create Custom machine learning models without any coding ● Able to serve via Rest API ● Can export Models for Mobile Devices ● Easy to use ● No way to do do A/B Testing ● No way to serve mobile model dynamically ● No easy way to track events through analytics

Slide 11

Slide 11 text

AutoML with MLKit

Slide 12

Slide 12 text

MLKit

Slide 13

Slide 13 text

MLKit: Using the Model

Slide 14

Slide 14 text

private fun callAutoMLModelLocally() { val localModel = FirebaseLocalModel.Builder("my_local_model") .setAssetFilePath("manifest.json") .build() FirebaseModelManager.getInstance().registerLocalModel(localModel) } MLKit: Local model

Slide 15

Slide 15 text

dependencies { implementation 'com.google.firebase:firebase-ml-vision:23.0.0' implementation 'com.google.firebase:firebase-ml-vision-automl:18.0.1' } apply plugin: 'com.google.gms.google-services' MLKit: Dependencies

Slide 16

Slide 16 text

private fun callAutoMLModelRemotely() { val conditions = FirebaseModelDownloadConditions.Builder() .requireWifi() .build() val remoteModel = FirebaseRemoteModel.Builder("Pavonia_Leafshapes") .enableModelUpdates(true) .setInitialDownloadConditions(conditions) .setUpdatesDownloadConditions(conditions) .build() FirebaseModelManager.getInstance().registerRemoteModel(remoteModel) downloadRemoteModel(remoteModel) } MLKit: Remote model

Slide 17

Slide 17 text

val labelerOptions = FirebaseVisionOnDeviceAutoMLImageLabelerOptions.Builder() .setLocalModelName("my_local_model") .setRemoteModelName("Pavonia_Leafshapes") .setConfidenceThreshold(0F) .build() val labeler = FirebaseVision.getInstance() .getOnDeviceAutoMLImageLabeler(labelerOptions) labeler.processImage(image) .addOnSuccessListener { labels -> //Do something with the label array (label.text & label.confidence } MLKit: Using our Model

Slide 18

Slide 18 text

MLKit: Oﬄine vs Serving ● Offline is fast and is bundle with the app ● Offline models does make your app bigger in size ● Offline requires app update to update model ● Serving lets you serve new updates of the model ● Makes your app smaller to download from the PlayStore

Slide 19

Slide 19 text

MLKit: Advanced ● Remote Config ● Firebase Analytics val remoteConfig = FirebaseRemoteConfig.getInstance() val remoteConfigDefaults = HashMap() remoteConfigDefaults["plant_labeler_model"] = "plant_labeler_v1" Tasks.await(remoteConfig.setDefaultsAsync(remoteConfigDefaults)) remoteConfig.fetchAndActivate().addOnSuccessListener { success -> if (success) { // Okay to get remote values. // ... } }