Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tensor Flow for Android Devs

Tensor Flow for Android Devs

Introduction to the Tensorflow from an Android developers Point of view. We will go through a simple example, explaining the steps, hurdles of integration of a TensorFlow model into a mobile application. In the end, you will understand the process of working with TensorFlow on Android devices. Hopefully, this will motivate you to implement your crazy AI ideas in the mobile world.

Blénesi Attila

September 09, 2019
Tweet

More Decks by Blénesi Attila

Other Decks in Programming

Transcript

  1. @ablenessy | @droidconLisbon | #DCLISBON19 What can AI do? Image

    classification Object detection Gesture recognition Speech recognition Text generation Translate Generate music Generate images
  2. @ablenessy | @droidconLisbon | #DCLISBON19 TensorFlow https://www.tensorflow.org/beta You can write

    code in: C++, Python, Swift, JavaScript Deploy to: CPU, GPU, TPU Mobile iOS and Android Raspberry Py, Coral (edge TPU), Microcontrollers TensorFlow 2.0 is now in beta Premade Estimators tf.Keras tf.* Low level API High level API Ready made models
  3. @ablenessy | @droidconLisbon | #DCLISBON19 Keras High level Neural Networks

    API, written in Python Integrated in TensorFlow : tf.keras Sequential - best to start with Functional - more flexible Model subclassing - extend a Model class Deep learning with Python By Francois Chollet
  4. @ablenessy | @droidconLisbon | #DCLISBON19 import tensorflow as tf mnist

    = tf.keras.datasets.mnist (x_train, y_train),(x_test, y_test) = mnist.load_data() x_train, x_test = x_train / 255.0, x_test / 255.0 model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(512, activation=tf.nn.relu), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10, activation=tf.nn.softmax) ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.fit(x_train, y_train, epochs=5) model.evaluate(x_test, y_test) Digit classifier
  5. @ablenessy | @droidconLisbon | #DCLISBON19 On device ML Low latency,

    no server calls Works offline, no connection needed Privacy, data stays on device
  6. @ablenessy | @droidconLisbon | #DCLISBON19 Tensor Flow Open sourced Tensor

    Flow Mobile TF Lite Developer preview ML Kit Tensor Flow 2.0 TF Lite - final TF Mobile deprecated 2015 2016 2017 2018 2019 Source: https://speakerdeck.com/margaretmz
  7. @ablenessy | @droidconLisbon | #DCLISBON19 Application Developer ML Practitioner Data

    Scientist Firebase ML Kit TensorFlow Lite Neural Networks API
  8. @ablenessy | @droidconLisbon | #DCLISBON19 Firebase ML Kit Source: firebase.google.com/products/ml-kit

    Image labelling Text Recognition Face Detection Barcode scanning Landmark detection Supports Custom models Dynamic downloads New: Auto ML, Smart reply, Object detection, Translation
  9. @ablenessy | @droidconLisbon | #DCLISBON19 val image = FirebaseVisionImage.fromBitmap(selectedImage) val

    detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) } .addOnFailureListener(...)
  10. @ablenessy | @droidconLisbon | #DCLISBON19 How does pix2pix work? generator

    INPUT OUTPUT Source: https://affinelayer.com/pix2pix/
  11. @ablenessy | @droidconLisbon | #DCLISBON19 Interpreter Core Converter TensorFlow Lite

    Format Operation Kernels Hardware acceleration ~75 KB vs 1.1 MB in TF ~2.4 MB vs 22.3 MB in Deprecate TF Mobile
  12. @ablenessy | @droidconLisbon | #DCLISBON19 Converter TensorFlow Lite Format 


    Command line Python API Android App (Java /C++ API) iOS App (C++ API) Linux (e.g. Raspberry Pi) (C++ API) Trained TensorFlow Model SavedModel HDF5
  13. Convert saved_model_dir = PATH + 'saved_model' tf.saved_model.save(generator, saved_model_dir) converter =

    tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) tf_lite_model = converter.convert() tf_lite_model_file.write_bytes(tf_lite_model)
  14. Convert NEW saved_model_dir = PATH + 'saved_model' tf.saved_model.save(generator, saved_model_dir) converter

    = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) tf_lite_model = converter.convert() tf_lite_model_file.write_bytes(tf_lite_model) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.float16] tf_lite_fp16_model = converter.convert() tf_lite_f16_model_file.write_bytes(tf_lite_fp16_model)
  15. Model > Training > Deployment android { aaptOptions { noCompress

    "tflite" } } implementation 'org.tensorflow:tensorflow-lite:1.14.0'