Tensor Flow for Android Devs

Tensor Flow for Android Devs

Introduction to the Tensorflow from an Android developers Point of view. We will go through a simple example, explaining the steps, hurdles of integration of a TensorFlow model into a mobile application. In the end, you will understand the process of working with TensorFlow on Android devices. Hopefully, this will motivate you to implement your crazy AI ideas in the mobile world.

6ad75d0512b7da8f9d3076e722dcc23d?s=128

Blénesi Attila

September 09, 2019
Tweet

Transcript

  1. @ablenessy | @droidconLisbon | #DCLISBON19 for android devs

  2. @ablenessy | @droidconLisbon | #DCLISBON19 What can AI do?

  3. @ablenessy | @droidconLisbon | #DCLISBON19 What can AI do? Image

    classification Object detection Gesture recognition Speech recognition Text generation Translate Generate music Generate images
  4. @ablenessy | @droidconLisbon | #DCLISBON19 Artificial Intelligence Machine Learning Deep

    learning: Computer vision, NLP, GAN …
  5. @ablenessy | @droidconLisbon | #DCLISBON19 Source: tensorflow.com/about

  6. @ablenessy | @droidconLisbon | #DCLISBON19 Source: tensorflow.com/about Neural Network Neural

    Network
  7. @ablenessy | @droidconLisbon | #DCLISBON19 Source: tensorflow.com/about Training

  8. @ablenessy | @droidconLisbon | #DCLISBON19 Caffe PyTorch

  9. @ablenessy | @droidconLisbon | #DCLISBON19 TensorFlow https://www.tensorflow.org/beta You can write

    code in: C++, Python, Swift, JavaScript Deploy to: CPU, GPU, TPU Mobile iOS and Android Raspberry Py, Coral (edge TPU), Microcontrollers TensorFlow 2.0 is now in beta Premade Estimators tf.Keras tf.* Low level API High level API Ready made models
  10. @ablenessy | @droidconLisbon | #DCLISBON19 Keras High level Neural Networks

    API, written in Python Integrated in TensorFlow : tf.keras Sequential - best to start with Functional - more flexible Model subclassing - extend a Model class Deep learning with Python By Francois Chollet
  11. @ablenessy | @droidconLisbon | #DCLISBON19 import tensorflow as tf mnist

    = tf.keras.datasets.mnist (x_train, y_train),(x_test, y_test) = mnist.load_data() x_train, x_test = x_train / 255.0, x_test / 255.0 model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(512, activation=tf.nn.relu), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10, activation=tf.nn.softmax) ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.fit(x_train, y_train, epochs=5) model.evaluate(x_test, y_test) Digit classifier
  12. @ablenessy | @droidconLisbon | #DCLISBON19 On device ML Low latency,

    no server calls Works offline, no connection needed Privacy, data stays on device
  13. @ablenessy | @droidconLisbon | #DCLISBON19 Tensor Flow Open sourced Tensor

    Flow Mobile TF Lite Developer preview ML Kit Tensor Flow 2.0 TF Lite - final TF Mobile deprecated 2015 2016 2017 2018 2019 Source: https://speakerdeck.com/margaretmz
  14. @ablenessy | @droidconLisbon | #DCLISBON19 Application Developer ML Practitioner Data

    Scientist Firebase ML Kit TensorFlow Lite Neural Networks API
  15. @ablenessy | @droidconLisbon | #DCLISBON19 Firebase ML Kit Source: firebase.google.com/products/ml-kit

    Image labelling Text Recognition Face Detection Barcode scanning Landmark detection Supports Custom models Dynamic downloads New: Auto ML, Smart reply, Object detection, Translation
  16. @ablenessy | @droidconLisbon | #DCLISBON19 Text Recognition

  17. @ablenessy | @droidconLisbon | #DCLISBON19 val image = FirebaseVisionImage.fromBitmap(selectedImage) val

    detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) } .addOnFailureListener(...)
  18. @ablenessy | @droidconLisbon | #DCLISBON19 ML workflow Get Data >

    Train > Convert > Integrate
  19. @ablenessy | @droidconLisbon | #DCLISBON19 TensorFlow #pix2pix Source: https://affinelayer.com/pixsrv/

  20. @ablenessy | @droidconLisbon | #DCLISBON19 #pix2pix TensorFlow

  21. @ablenessy | @droidconLisbon | #DCLISBON19 #pix2pix TensorFlow

  22. @ablenessy | @droidconLisbon | #DCLISBON19 How does pix2pix work? Source:

    https://affinelayer.com/pix2pix/
  23. @ablenessy | @droidconLisbon | #DCLISBON19 How does pix2pix work? Source:

    https://affinelayer.com/pix2pix/
  24. @ablenessy | @droidconLisbon | #DCLISBON19 Source: https://dzone.com/articles/working-principles-of-generative-adversarial-netwo

  25. @ablenessy | @droidconLisbon | #DCLISBON19 How does pix2pix work? generator

    INPUT OUTPUT Source: https://affinelayer.com/pix2pix/
  26. @ablenessy | @droidconLisbon | #DCLISBON19 How does pix2pix work? Source:

    https://affinelayer.com/pix2pix/
  27. @ablenessy | @droidconLisbon | #DCLISBON19

  28. @ablenessy | @droidconLisbon | #DCLISBON19

  29. Source: https://affinelayer.com/pix2pix/

  30. @ablenessy | @droidconLisbon | #DCLISBON19 TensorBoard

  31. @ablenessy | @droidconLisbon | #DCLISBON19 Train > Convert > Integrate

  32. ~680 mb ~2600 operations Train > Convert > Integrate

  33. ~680 mb ~2600 operations Train > Convert > Integrate

  34. ~200 mb ~400 operations Train > Convert > Integrate

  35. @ablenessy | @droidconLisbon | #DCLISBON19 Train > Convert > Integrate

  36. @ablenessy | @droidconLisbon | #DCLISBON19 Interpreter Core Converter TensorFlow Lite

    Format Operation Kernels Hardware acceleration ~75 KB vs 1.1 MB in TF ~2.4 MB vs 22.3 MB in Deprecate TF Mobile
  37. @ablenessy | @droidconLisbon | #DCLISBON19 Converter TensorFlow Lite Format 


    Command line Python API Android App (Java /C++ API) iOS App (C++ API) Linux (e.g. Raspberry Pi) (C++ API) Trained TensorFlow Model SavedModel HDF5
  38. Convert saved_model_dir = PATH + 'saved_model' tf.saved_model.save(generator, saved_model_dir) converter =

    tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) tf_lite_model = converter.convert() tf_lite_model_file.write_bytes(tf_lite_model)
  39. Convert NEW saved_model_dir = PATH + 'saved_model' tf.saved_model.save(generator, saved_model_dir) converter

    = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) tf_lite_model = converter.convert() tf_lite_model_file.write_bytes(tf_lite_model) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.float16] tf_lite_fp16_model = converter.convert() tf_lite_f16_model_file.write_bytes(tf_lite_fp16_model)
  40. Optimisation options 103 MB 207 MB 154 MB 61 MB

    - requires custom ops Fails
  41. Test test test!

  42. Model > Training > Deployment android { aaptOptions { noCompress

    "tflite" } } implementation 'org.tensorflow:tensorflow-lite:1.14.0'
  43. @ablenessy | @droidconLisbon | #DCLISBON19 Model > Training > Deployment

  44. @ablenessy | @droidconLisbon | #DCLISBON19 Attila Blénesi Android Engineer @

    Babylon Health @ablenessy @ablenessy ablenesi