Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ML Kit: Hype or Har Gow?

ML Kit: Hype or Har Gow?

Introduction to ML Kit For Firebase focusing on how to use custom machine learning models on Android with TensorFlow Lite. I demonstrate how I followed Google's codelabs, retrained a MobileNet with my own images, and modified a sample to be able to classify different kinds of dim sum.

NOTE: This deck doesn't include the live demo I did. Please watch the video below:

Video: https://www.youtube.com/watch?v=Y5LMNA9drr0


Eric Fung

July 16, 2018

More Decks by Eric Fung

Other Decks in Programming


  1. ML Kit: Hype or Har Gow? Eric Fung <efung@acm.org> GDG

    Toronto Android • 2018-07-16
  2. What is Machine Learning?

  3. Field of study that gives computers the ability to learn

    without being explicitly programmed — Paraphrased from Arthur Samuel (1959)
  4. Machine learning is concerned with the ques3on of how to

    construct computer programs that automa&cally improve (their performance at some task) with experience — Paraphrased from Tom Mitchell (1998)
  5. Example Task: Image Classifica3on

  6. None
  7. What is ML Kit?

  8. What is ML Kit? • Set of machine learning SDKs

    for mobile • Applica7on layer of Google's mobile ML stack • ML Kit • TensorFlow Lite • Android Neural Networks API / iOS Metal • Solves hard challenges in deploying ML on mobile
  9. ML Kit Base APIs • Recognizing text • Labeling images

    • Detec4ng faces • Scanning barcodes • Iden4fying landmarks • More to come: Smart Reply, face contours
  10. Other ML Kit Features • Runs on-device (faster, no network)

    or via cloud (greater accuracy, more func;onality) • Models can be downloaded at app install ;me, or on first-use • Supports custom ML models
  11. Let's Play with Custom Models

  12. Custom Models • Upload your own TensorFlow Lite models •

    A/B test different models • Decouple model updates from app updates
  13. Codelabs • TensorFlow For Poets • Use Python to retrain

    an exis4ng network • Classify images on command-line • Output is a TF model
  14. Codelabs • TensorFlow For Poets 2: TFLite Android • Convert

    TF model to TensorFlow Lite model • Include model with Android app • Run inference using embedded TFLite interpreter
  15. Codelabs • Iden&fy objects in images using custom machine learning

    models with ML Kit for Firebase • Upload a TF Lite model into Firebase • Use ML Kit SDK in Android app to download model • Run inference using SDK
  16. ! Idea • What's in that steamer basket? • Train

    an exis3ng vision model to classify dim sum • Deploy model using Firebase ML Kit • Build an app to capture images and iden3fy them via model
  17. ! Womp womp • Unfortunately, it didn't work out •

    TFLite model incompa:bility with ML Kit Android sample • float32 vs int? • Live camera feed • ML Kit sample uses bundled sta:c images • Need more inves:ga:on
  18. However…

  19. Demo

  20. (Non ML Kit) Demo

  21. End • Email efung@acm.org • Blog code.gnufmuffin.com • Code github.com/efung

    • Social @gnufmuffin • Slides speakerdeck.com/efung