Deep Learning for Android with TensorFlow

Talk at TensorFlow World.

Transcript

1. Deep Learning for Android with TensorFlow 1 @margaretmz
2. @margaretmz | #ML | #GDE Topics • Overview of Android ML options • React Native • ML Kit • tf.Keras to TFLite to Android 2
3. @margaretmz | #ML | #GDE TensorFlow on mobile 3 2015 TF open sourced 2016 TF Mobile 2017 TF Lite developer preview 2018 ML Kit 2019 - ML Kit new features - TF Mobile deprecated - TFLite new announcements!!!
4. @margaretmz | #ML | #GDE On-device ML for mobile 4 How Who Where Native Android (iOS) apps • Direct deploy to Android • With ML Kit Android (or iOS) developers React Native Web developers
5. 5 React Native
6. @margaretmz | #ML | #GDE React Native Support • Use TF.js ML directly inside React Native with WebGL acceleration • Load models from the web, or compile into your application Link to demo video | Link to github 6
7. 7 ML Kit for Android Google’s Machine Learning SDK for Mobile
8. @margaretmz | #ML | #GDE ML Kit • Brings Google’s ML expertise to mobile developers in a powerful and easy-to-use package. • Powered by TensorFlow Lite & hosted on Firebase + 8 Lite
9. @margaretmz | #ML | #GDE ML Kit - base APIs Here are the base APIs you can use for ML Kit 9 9 Image labelling OCR Face detection Barcode scanning Landmark detection Smart reply Object detection & Tracking Translation (56 languages) AutoML g.co/mlkit
10. @margaretmz | #ML | #GDE ML Kit - custom models For custom models, ML kits offers - • Dynamic model downloads • A/B testing (via Firebase remote Configuration) • Model compression & conversion (from TensorFlow to TF Lite) 10
11. @margaretmz | #ML | #GDE ML Kit 11 Convert to Bytebuffer/ bitmap Calibration Java Native Frame Scheduler (Image Timestamp) Convert to byte array Output Results Pipeline config Convert to Grayscale Resize/Rotate Tracker Frame Selection Convert to RGB/Resize/ Rotate Detector (TF Lite model) Object Manager Image Validation Resize Pipeline Classifier ( TF Lite model) Source: ML Kit team
12. @margaretmz | #ML | #GDE • Firebase console • AutoML - train model • Download TFLite • Mobile https://firebase.google.com/docs/ml-kit/automl-image-labeling ML Kit - AutoML 12
13. 13 Native Android Apps Direct deploy to Android End to end tf.Keras to TFLite to Android
14. @margaretmz | #ML | #GDE Models How to get a model for your mobile app: • Download a pre-trained model (here): Inception-v3, mobilenet etc. • Transfer learning with a pre-trained model ◦ Feature extraction or fine tuning on pre-trained model ◦ TensorFlow hub (https://www.tensorflow.org/hub/) • Train your own model from scratch (example in this talk) 14
15. 15 Training a model With tf.Keras
16. @margaretmz | #ML | #GDE End to End: tf.Keras to TFLite to Android 16
17. @margaretmz | #ML | #GDE MNIST dataset • 60,000 train set and 10,000 test set • 28x28x1 grayscale images • 10 classes: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 • Popular for computer vision ◦ “hello world” tutorial or ◦ benchmarking ML algorithms 17
18. @margaretmz | #ML | #GDE Training model Launch sample code on Colab → mnist_tfkeras_to_tflite.ipynb 1. Import data 2. Define a model 3. Train a model 4. Save a Keras model (save as SavedModel) & convert to tflite 18
19. @margaretmz | #ML | #GDE A simple CNN architecture MNIST example: • Convolutional layer (definition) • Pooling layer (definition) • Dense (fully-connected layer) definition 19 input conv pool conv pool conv pool Dense 0 1 2 3 4 5 6 7 8 9
20. @margaretmz | #ML | #GDE Inspect the model After defining the model architecture, use model.summary() to show the model architecture 20
21. @margaretmz | #ML | #GDE Visualize model architecture Use a visualization tool: • (during training) TensorBoard • (after conversion) Netron - Drop the .tflite model into Netron and see the model visually Note: model metadata in a new TFLite tool (to be launched) will allow you to inspect the model & modify the metadata 21
22. @margaretmz | #ML | #GDE TensorFlow 2.0 22 • tf.Keras (TensorFlow) • Python libraries: Numpy, Matplotlib etc SavedModel • Cloud • Web • Mobile • IoT • Micro controllers • Edge TPU Training Inference (Or a tf.Keras model)
23. @margaretmz | #ML | #GDE Model saving When to save as SavedModel or a Keras model? Note: In TensorFlow 2.0 , tf.keras.Model.save() and tf.keras.models.save_model() default to the SavedModel format (not HDF5). (link to doc) 23 SavedModel Keras Model Share pre-trained models and model pieces on TensorFlow Hub Train with tf.Keras and you know your deploy your target When you don’t know the deploy target
24. 24 Conversion With TFLite converter
25. @margaretmz | #ML | #GDE TensorFlow Lite • Converter - convert to TFLite file format • Interpreter - execute inference & optimized for small devices • Ops/Kernel - limited ops • Interface to hardware acceleration ◦ NN API ◦ Edge TPU 25
26. @margaretmz | #ML | #GDE Changes to TFLite converter 26 TensorFlow 1.X TensorFlow 2.0 from_session() Removed N/A from_frozen_graph() Removed N/A from_saved_model() Same from_saved_model() from_keras_model_file(string) Changed Changed to from_keras_model(model) N/A Added from_concrete_function() Details of changes between TF 1.X and 2.0
27. @margaretmz | #ML | #GDE TFLite converter 27 Command line Python code (recommended) SavedModel tflite_convert \ --saved_model_dir=/tmp/my_saved_model \ --output_file=/tmp/my_model.tflite Keras Model --keras_model_file=/tmp/my_keras_model.h5 \ --output_file=/tmp/my_model.tflite # Create a converter converter = tf.contrib.lite.TFLiteConverter.from_keras_model_file(keras_model) from_keras_model(model) # Set quantize to true (optional) converter.post_training_quantize=True # Convert the model tflite_model = converter.convert() # Create the tflite model file tflite_model_name = "my_model.tflite" open(tflite_model_name, "wb").write(tflite_model)
28. @margaretmz | #ML | #GDE Validate TFLite model after conversion 28 Protip: validate the tflite model in python after conversion - 28 TensorFlow result TFLite result Compare results # Test the TensorFlow model on random Input data. tf_result = model(tf.constant(input_data)) # Load TFLite model and allocate tensors. interpreter = tf.lite.Interpreter(model_path="converted_model.tflite") interpreter.allocate_tensors() # Get input and output tensors. input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() # Test model on random input data. input_shape = input_details[0]['shape'] input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) interpreter.set_tensor(input_details[0]['index'], input_data) interpreter.invoke() tflite_result = interpreter.get_tensor(output_details[0]['index']) # Compare the result. for tf_result, tflite_result in zip(tf_result, tflite_result): np.testing.assert_almost_equal(tf_result, tflite_result, decimal=5)
29. 29 Deploy to Android In Android Studio
30. @margaretmz | #ML | #GDE TFLite on Android Android sample code DigitRecognizer, step by step: • Place tf.lite model under assets folder • Update build.gradle dependencies • Input image - custom view, gallery or camera • Data preprocessing • Classify with the model • Post processing • Display result in UI 30
31. @margaretmz | #ML | #GDE Gradle dependency Update build.gradle to include tensorflow lite android { // Make sure model doesn't get compressed when app is compiled aaptOptions { noCompress "tflite" } } dependencies { …. // Add dependency for TensorFlow Lite compile 'org.tensorflow:tensorflow-lite:[version-number]’ } Place the mnist.tflite model file under /assets folder 31
32. @margaretmz | #ML | #GDE Input image data Input to the classifier is an image, your options: 1. Draw on canvas from a custom View 2. Get image from Gallery or a 3rd party camera 3. Live frames from Camera2 API Make sure the image dimensions (shape) matches what your classifier expects • 28x28x1- MNIST or FASHION_MNIST gray scale image • 299x299x3 - Inception V3 • 256x256x3 - MobileNet 32
33. @margaretmz | #ML | #GDE Image preprocessing • Convert Bitmap to ByteBuffer • Normalize pixel values to be a certain range • Convert from color to grayscale, if needed Note: with the TFLite Android Support library announced today, this step is simplified 33
34. @margaretmz | #ML | #GDE Run inference Load the model file located under the assets folder Use the TensorFlow Lite interpreter to run inference on the input image 34
35. @margaretmz | #ML | #GDE Post processing & UI The output is an array of probabilities, each correspond to a category Find the category with the highest probability and output result to UI 35
36. @margaretmz | #ML | #GDE Summary • Training with tf.Keras is easy • Model conversion to TFLite is easier • Android implementation is still challenging & error-prone: (getting improved with the support library) ◦ Validate tflite model before deploy to Android ◦ Image pre-processing ◦ Input tensor shape? ◦ Color or grayscale? ◦ Post processing My blog post: E2E tf.Keras to TFLite to Android 36
37. @margaretmz | #ML | #GDE More TFLite examples 37
38. @margaretmz | #ML | #GDE What’s next? • Why the future of machine learning is tiny? - Pete Warden • Deploying to mobile and IoT will get much easier • TFLite new features • Federated learning • On device training 38
39. @margaretmz | #ML | #GDE Thank you! Follow me on Twitter, Medium or GitHub to learn more about Deep learning, TensorFlow and Android ML. @margaretmz @margaretmz margaretmz 39