Life Cycle of building ML enabled features on Mobile

As Mobile engineers we have a very common feature life cycle. 90% of the time it's implementing some kind of rest service into our apps to display some information that was calculated in a backend. When it comes to building ML enabled features, the life cycle works a bit differently. In this talk we will go through this ML life cycle, from what a ML enabled feature looks like, to building ML models for mobile, different ways to deploy these models to your app and how to monitor these models to know that it's giving users the correct information.

Transcript

1. Peter-John Welcome
   GDE Firebase
   @pjapplez
   The life cycle of building
   Machine Learning enabled
   features on mobile
   An Introduction to ML life cycle on mobile
   

   View Slide

2. Half screen photo slide if
   text is necessary
   Who I am
   Freelance Mobile Engineer
   Google Developer Expert for Firebase
   

   View Slide

3. Data-driven vs
   ML-driven
   

   View Slide

4. Data-driven vs ML-driven
   

   View Slide

5. Data-driven vs ML-driven
   

   View Slide

6. Will ML add unique
   value?
   

   View Slide

7. Will ML add value
   Yes
   Developer could write a bunch of if statements or doing word searches
   but maintain that will become unmaintainable.
   

   View Slide

8. ● Recognize class entities
   ● Recommend content
   ● Natural language understanding
   ● Predicting future events (forecasting)
   ● Etc.
   https://pair.withgoogle.com/guidebook/
   Examples of ML adding value
   

   View Slide

9. ● Automating high value tasks
   ● Providing static information to user.
   ● Providing transparency.
   ● Etc.
   https://pair.withgoogle.com/guidebook/
   Examples of ML !adding value
   

   View Slide

10. The ML life cycle
    

    View Slide

11. ML Life Cycle
    

    View Slide

12. ML Decision Pyramid
    https://www.youtube.com/watch?v=pm_-pVPvZ-4
    

    View Slide

13. ● Data wrangling.
    ● Making it accessible in some kind of database/warehouse/data lake.
    ● Trying to use the data to solve the problem.
    Data collection and labelling
    

    View Slide

14. ● Tensorflow ( Google’s open source ML platform)
    ● CoreML (Apples ML platform)
    ● Cloud-based approach.
    Training our model
    

    View Slide

15. Training our model
    Tensorflow lite
    An open source machine learning platform for on-device inference
    Cloud , Google colab , on personal machine
    Python and Swift
    Has support for Android, iOS, IOT device etc
    https://medium.com/dvt-engineering/will-swift-take-over-python-as-the-go-to-language-for-ml-ff9f8d9d7141
    https://www.tensorflow.org/lite
    

    View Slide

16. Training our model
    CoreML
    Machine learning platform by Apple thats optimized for on device ML
    and leverages Apple hardware
    Supports only iOS devices
    CoreML does have a converter to convert Tensorflow and Pytorch
    models to coreML
    CreateML is an Xcode bundled ML tool to create coreml models
    https://developer.apple.com/machine-learning/core-ml/
    

    View Slide

17. Training our model
    Cloud based - API
    Google Cloud, Amazon, Azure all provide us the ability to train ML
    models in the cloud and deploy them.
    We could create a REST API or there are products that can generate
    REST API’s
    

    View Slide

18. ● Better accuracy
    ● Preventing Bias
    ● Feature importance
    Optimize
    

    View Slide

19. ● Tensorflow Lite Converter
    ● CoreML Converter
    import tensorflow as tf
    converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
    tflite_model = converter.convert()
    open("converted_model.tflite", "wb").write(tflite_model)
    https://www.tensorflow.org/lite/guide/get_started#2_convert_the_model_format
    https://coremltools.readme.io/docs
    Converting
    

    View Slide

20. ● Models are intellectual property
    ● Once the model is deployed its in the users hands.
    ● Obfuscation
    ● Model encryption ( weights, architecture, format scrambling)
    ● Hybrid processing ( processing data before inputting it into the model)
    Encrypt and Protect
    

    View Slide

21. ● On-Device models helps with privacy
    ● Faster inference/predictions
    ● Bloat app ( App size )
    ● Updating model requires updating the app as its bundled.
    ● REST API can be used by then it requires network and violates user privacy
    ● Remote deployment of models with Firebase or ML deployment (iOS)
    Deployment
    

    View Slide

22. ● This is the hardest part on the process
    ● Analytics
    ● User Testing, A/B Testing
    ● UX is really important at this point in how the user interacts with the model
    ● Being aware of privacy when collecting data
    ● Making sure model is not Bias
    ● Explained really well in the People and AI guide book
    https://pair.withgoogle.com/guidebook/
    Monitoring
    

    View Slide

23. On-device ML
    Using transfer learning
    Being able to update/retraining parts a model
    This helps with personalization
    CoreML on iOS and Tensorflow lite on Android both support this.
    Great for privacy
    

    View Slide

24. References
    ● https://www.tensorflow.org/lite
    ● https://developer.apple.com/machine-learning/core-ml/
    ● https://pair.withgoogle.com/guidebook/
    ● https://firebase.google.com/docs/ml/manage-hosted-models
    ● https://www.fritz.ai/
    

    View Slide

25. Peter-John Welcome
    GDE Firebase
    @pjapplez
    Thank You!
    

    View Slide