Heat the Neurons of Your Smartphone with Deep Learning

7b5a07956eb0b62be7214d043821a987?s=47 jinqian
April 10, 2017

Heat the Neurons of Your Smartphone with Deep Learning

[AndroidMakers 2017] TensorFlow on Android, the project Magritte is an educative application that allows people to learn a language with deep learning on-device!

Video demo here: https://www.youtube.com/watch?v=FCzrWMmrbYQ
Video for the talk: https://www.youtube.com/watch?v=znjGEW6pESQ

7b5a07956eb0b62be7214d043821a987?s=128

jinqian

April 10, 2017
Tweet

Transcript

  1. 1.

    Heat the Neurons of Your Smartphone with Deep Learning Qian

    Jin @bonbonking Yoann Benoit @yoannbenoit AndroidMakers Paris | 10th April 2017
  2. 4.

    4

  3. 5.

    “In my 34 years in the semiconductor industry, I have

    witnessed the advertised death of Moore’s Law no less than four times. As we progress from 14 nanometer technology to 10 nanometer and plan for 7 nanometer and 5 nanometer and even beyond, our plans are proof that Moore’s Law is alive and well.” 5
  4. 8.

    The ultimate goal of the on-device intelligence is to improve

    mobile devices’ ability to understand the world. 8
  5. 10.

    10

  6. 12.

    12

  7. 14.

    14

  8. 25.

    25

  9. 34.

    Transfer Learning • Use a pre-trained Deep Neural Network •

    Keep all operations but the last one • Re-train only the last operation to specialize your network to your classes Keep all weights identical except these ones 34
  10. 35.

    Save the model • 2 things to save • Execution

    graph • Weights for each operation • 2 outputs • Model as protobuf file • Labels in text file 35 model.pb label.txt
  11. 37.

    Unsupported Operation • Only keep the operations dedicated to the

    inference step • Remove decoding, training, loss and evaluation operations 37
  12. 41.

    Standalone App • Use nightly build • Library .so •

    Java API jar android { //… sourceSets { main { jniLibs.srcDirs = ['libs'] } } } 41
  13. 48.

    Android SDK (Java) Android NDK (C++) Classifier Implementation TensorFlow JNI

    wrapper Image (Bitmap) Trained Model top_results Classifications + Confidence input_tensor 1 2 3 4 Camera Preview Ref: https://jalammar.github.io/Supercharging-android-apps-using-tensorflow/ Overlay Display 48
  14. 50.

    Converts YUV420 to ARGB8888 public static native void convertYUV420ToARGB8888( byte[]

    y, byte[] u, byte[] v, int[] output, int width, int height, int yRowStride, int uvRowStride, int uvPixelStride, boolean halfSize ); 50
  15. 51.

    Create Input Tensor From RGB values // Preprocess the image

    data from 0-255 int to normalized float based // on the provided parameters. bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight()); for (int i = 0; i < intValues.length; ++i) { final int val = intValues[i]; floatValues[i * 3 + 0] = (((val >> 16) & 0xFF) - imageMean) / imageStd; floatValues[i * 3 + 1] = (((val >> 8) & 0xFF) - imageMean) / imageStd; floatValues[i * 3 + 2] = ((val & 0xFF) - imageMean) / imageStd; } inferenceInterface.feed(inputName, floatValues, 1, inputSize, inputSize, 3); 51
  16. 54.

    Model Stacking 54 • Start from previous model to keep

    all specific operations in the graph • Specify all operations to keep when optimizing for inference graph_util.convert_variables_to_constants(sess, graph.as_graph_def(), [“final_result_fruits”, “final_result_vegetables”]
  17. 55.
  18. 56.

    56

  19. 59.

    Next: Federate Learning Collaborative Machine Learning without Centralized Training Data

    59 Ref: https://research.googleblog.com/2017/04/federated-learning-collaborative.html