Creating Custom Models with AutoML and MLKit

This is a presentation that I gave at DroidCon Greece in 2019

Transcript

1. Creating Custom Models with AutoML and MLKit Peter-John Welcome @pjapplez Mobile Engineering Lead
2. About me
3. AutoML with MLKit • Create Custom Model • Generate a Rest API • Offline Models • Remote serving • Remote Config • User experience • Performance AutoML MLKit Android
4. AutoML
5. This is me
6. What is AutoML?
7. AutoML
8. AutoML Rest API
10. AutoML Pros & Cons • Create Custom machine learning models without any coding • Able to serve via Rest API • Can export Models for Mobile Devices • Easy to use • No way to do do A/B Testing • No way to serve mobile model dynamically • No easy way to track events through analytics
11. AutoML with MLKit
12. MLKit
13. MLKit: Using the Model
14. private fun callAutoMLModelLocally() { val localModel = FirebaseLocalModel.Builder("my_local_model") .setAssetFilePath("manifest.json") .build() FirebaseModelManager.getInstance().registerLocalModel(localModel) } MLKit: Local model
15. dependencies { implementation 'com.google.firebase:firebase-ml-vision:23.0.0' implementation 'com.google.firebase:firebase-ml-vision-automl:18.0.1' } apply plugin: 'com.google.gms.google-services' MLKit: Dependencies
16. private fun callAutoMLModelRemotely() { val conditions = FirebaseModelDownloadConditions.Builder() .requireWifi() .build() val remoteModel = FirebaseRemoteModel.Builder("Pavonia_Leafshapes") .enableModelUpdates(true) .setInitialDownloadConditions(conditions) .setUpdatesDownloadConditions(conditions) .build() FirebaseModelManager.getInstance().registerRemoteModel(remoteModel) downloadRemoteModel(remoteModel) } MLKit: Remote model
17. val labelerOptions = FirebaseVisionOnDeviceAutoMLImageLabelerOptions.Builder() .setLocalModelName("my_local_model") .setRemoteModelName("Pavonia_Leafshapes") .setConfidenceThreshold(0F) .build() val labeler = FirebaseVision.getInstance() .getOnDeviceAutoMLImageLabeler(labelerOptions) labeler.processImage(image) .addOnSuccessListener { labels -> //Do something with the label array (label.text & label.confidence } MLKit: Using our Model
18. MLKit: Offline vs Serving • Offline is fast and is bundle with the app • Offline models does make your app bigger in size • Offline requires app update to update model • Serving lets you serve new updates of the model • Makes your app smaller to download from the PlayStore
19. MLKit: Advanced • Remote Config • Firebase Analytics val remoteConfig = FirebaseRemoteConfig.getInstance() val remoteConfigDefaults = HashMap<String, Any>() remoteConfigDefaults["plant_labeler_model"] = "plant_labeler_v1" Tasks.await(remoteConfig.setDefaultsAsync(remoteConfigDefaults)) remoteConfig.fetchAndActivate().addOnSuccessListener { success -> if (success) { // Okay to get remote values. // ... } }
20. Android
21. MLKit Offline AutoML Rest API AutoML: Performance & UX
22. AutoML: Pricing
23. AutoML: Use case
24. Thanks! @pjapplez