Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Heat the Neurons of Your Smartphone with Deep Learning

jinqian
April 10, 2017

Heat the Neurons of Your Smartphone with Deep Learning

[AndroidMakers 2017] TensorFlow on Android, the project Magritte is an educative application that allows people to learn a language with deep learning on-device!

Video demo here: https://www.youtube.com/watch?v=FCzrWMmrbYQ
Video for the talk: https://www.youtube.com/watch?v=znjGEW6pESQ

jinqian

April 10, 2017
Tweet

More Decks by jinqian

Other Decks in Technology

Transcript

  1. Heat the Neurons of Your
    Smartphone with Deep Learning
    Qian Jin @bonbonking
    Yoann Benoit @yoannbenoit
    AndroidMakers Paris | 10th April 2017

    View Slide

  2. On-Device Intelligence

    View Slide

  3. 3
    Image Credit: Google Research Blog

    View Slide

  4. 4

    View Slide

  5. “In my 34 years in the semiconductor industry, I have witnessed the advertised
    death of Moore’s Law no less than four times. As we progress from 14 nanometer
    technology to 10 nanometer and plan for 7 nanometer and 5 nanometer and even
    beyond, our plans are proof that Moore’s Law is alive and well.”
    5

    View Slide

  6. 6
    Ref: https://www.qualcomm.com/news/snapdragon/2017/01/09/tensorflow-machine-learning-now-optimized-snapdragon-835-and-hexagon-682

    View Slide

  7. 7
    Ref: https://9to5google.com/2017/01/10/qualcomm-snapdragon-835-machine-learning-tensorflow/

    View Slide

  8. The ultimate goal of the on-device
    intelligence is to improve mobile
    devices’ ability to understand the world.
    8

    View Slide

  9. Magritte
    Ceci n’est pas une pomme.

    View Slide

  10. 10

    View Slide

  11. #datamobile
    Chat History of the Slack channel
    11

    View Slide

  12. 12

    View Slide

  13. Build TensorFlow
    Android Example With Bazel
    13

    View Slide

  14. 14

    View Slide

  15. Android Developer
    Deep Learning Noob

    View Slide

  16. NEURONS
    NEURONS EVERYWHERE
    16

    View Slide

  17. WE CAN RECOGNIZE ALL
    THE THINGS!
    17

    View Slide

  18. I THOUGHT THERE WERE
    MODELS FOR EVERYTHING...
    18

    View Slide

  19. Neural Networks in a Nutshell

    View Slide

  20. Here’s a Neural Network
    20

    View Slide

  21. Prediction on an image - Inference
    21

    View Slide

  22. Prediction on an image - Inference
    22

    View Slide

  23. Prediction on an image - Inference
    Apple: 0.98
    Banana: 0.02
    23

    View Slide

  24. Training a model

    View Slide

  25. 25

    View Slide

  26. Training a model - Back Propagation
    26

    View Slide

  27. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    27

    View Slide

  28. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    Prediction
    error
    28

    View Slide

  29. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    Prediction
    error
    29

    View Slide

  30. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    Prediction
    error
    30

    View Slide

  31. Training a model - Back Propagation
    Apple: 0.27
    Banana: 0.73
    31

    View Slide

  32. Transfer Learning

    View Slide

  33. Deep Convolutional Neural Network
    33

    View Slide

  34. Transfer Learning
    • Use a pre-trained Deep Neural Network
    • Keep all operations but the last one
    • Re-train only the last operation to specialize your network to your classes
    Keep all weights identical
    except these ones
    34

    View Slide

  35. Save the model
    • 2 things to save
    • Execution graph
    • Weights for each operation
    • 2 outputs
    • Model as protobuf file
    • Labels in text file
    35
    model.pb label.txt

    View Slide

  36. java.lang.UnsupportedOperationException:
    Op BatchNormWithGlobalNormalization is not
    available in GraphDef version 21.
    36

    View Slide

  37. Unsupported Operation
    • Only keep the operations dedicated to the inference step
    • Remove decoding, training, loss and evaluation operations
    37

    View Slide

  38. Data Scientist
    Android Development Noob

    View Slide

  39. CLICK 7 TIMES ON BUILD NUMBER
    39

    View Slide

  40. Build Standalone App

    View Slide

  41. Standalone App
    • Use nightly build
    • Library .so
    • Java API jar
    android {
    //…
    sourceSets {
    main {
    jniLibs.srcDirs = ['libs']
    }
    }
    }
    41

    View Slide

  42. App size ~80MB
    42

    View Slide

  43. Reducing model size

    View Slide

  44. WHO CARES?
    MODEL SIZE
    44

    View Slide

  45. Model Size
    All weights are stored as they are (64-bit floats) => 80MB
    45

    View Slide

  46. 80 MB => 20 MB
    46
    Weights quantization
    6.372638493746383 => 6.4

    View Slide

  47. Architecture Underneath

    View Slide

  48. Android SDK (Java) Android NDK (C++)
    Classifier
    Implementation
    TensorFlow
    JNI wrapper
    Image (Bitmap)
    Trained Model
    top_results
    Classifications + Confidence
    input_tensor
    1 2
    3
    4
    Camera
    Preview
    Ref: https://jalammar.github.io/Supercharging-android-apps-using-tensorflow/
    Overlay
    Display
    48

    View Slide

  49. Image Sampling
    Get Image from
    Camera Preview
    Crop the center square Resize Sample Image
    49

    View Slide

  50. Converts YUV420 to ARGB8888
    public static native void
    convertYUV420ToARGB8888(
    byte[] y,
    byte[] u,
    byte[] v,
    int[] output,
    int width,
    int height,
    int yRowStride,
    int uvRowStride,
    int uvPixelStride,
    boolean halfSize
    );
    50

    View Slide

  51. Create Input Tensor From RGB values
    // Preprocess the image data from 0-255 int to normalized float based
    // on the provided parameters.
    bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(),
    bitmap.getHeight());
    for (int i = 0; i < intValues.length; ++i) {
    final int val = intValues[i];
    floatValues[i * 3 + 0] = (((val >> 16) & 0xFF) - imageMean) / imageStd;
    floatValues[i * 3 + 1] = (((val >> 8) & 0xFF) - imageMean) / imageStd;
    floatValues[i * 3 + 2] = ((val & 0xFF) - imageMean) / imageStd;
    }
    inferenceInterface.feed(inputName, floatValues, 1, inputSize, inputSize, 3);
    51

    View Slide

  52. Adding new models

    View Slide

  53. Adding a new model
    53
    2 * 20 MB = 40 MB

    View Slide

  54. Model Stacking
    54
    • Start from previous model to keep all specific operations in the graph
    • Specify all operations to keep when optimizing for inference
    graph_util.convert_variables_to_constants(sess, graph.as_graph_def(),
    [“final_result_fruits”, “final_result_vegetables”]

    View Slide

  55. Demo Time

    View Slide

  56. 56

    View Slide

  57. What’s next?
    57

    View Slide

  58. Next: Pulling Model From the Cloud
    58
    Ref: https://www.youtube.com/watch?v=EnFyneRScQ8

    View Slide

  59. Next: Federate Learning
    Collaborative Machine Learning without Centralized Training Data
    59
    Ref: https://research.googleblog.com/2017/04/federated-learning-collaborative.html

    View Slide

  60. Thank you! Questions?
    Github: https://github.com/xebia-france/magritte

    View Slide