Heat the Neurons of Your Smartphone with Deep Learning

[AndroidMakers 2017] TensorFlow on Android, the project Magritte is an educative application that allows people to learn a language with deep learning on-device!

Video demo here: https://www.youtube.com/watch?v=FCzrWMmrbYQ
Video for the talk: https://www.youtube.com/watch?v=znjGEW6pESQ

Transcript

1. Heat the Neurons of Your
   Smartphone with Deep Learning
   Qian Jin @bonbonking
   Yoann Benoit @yoannbenoit
   AndroidMakers Paris | 10th April 2017
   

   View Slide

2. On-Device Intelligence
   

   View Slide

3. 3
   Image Credit: Google Research Blog
   

   View Slide

4. View Slide

5. “In my 34 years in the semiconductor industry, I have witnessed the advertised
   death of Moore’s Law no less than four times. As we progress from 14 nanometer
   technology to 10 nanometer and plan for 7 nanometer and 5 nanometer and even
   beyond, our plans are proof that Moore’s Law is alive and well.”
   5
   

   View Slide

6. 6
   Ref: https://www.qualcomm.com/news/snapdragon/2017/01/09/tensorflow-machine-learning-now-optimized-snapdragon-835-and-hexagon-682
   

   View Slide

7. 7
   Ref: https://9to5google.com/2017/01/10/qualcomm-snapdragon-835-machine-learning-tensorflow/
   

   View Slide

8. The ultimate goal of the on-device
   intelligence is to improve mobile
   devices’ ability to understand the world.
   8
   

   View Slide

9. Magritte
   Ceci n’est pas une pomme.
   

   View Slide

10. View Slide

11. #datamobile
    Chat History of the Slack channel
    11
    

    View Slide

12. View Slide

13. Build TensorFlow
    Android Example With Bazel
    13
    

    View Slide

14. View Slide

15. Android Developer
    Deep Learning Noob
    

    View Slide

16. NEURONS
    NEURONS EVERYWHERE
    16
    

    View Slide

17. WE CAN RECOGNIZE ALL
    THE THINGS!
    17
    

    View Slide

18. I THOUGHT THERE WERE
    MODELS FOR EVERYTHING...
    18
    

    View Slide

19. Neural Networks in a Nutshell
    

    View Slide

20. Here’s a Neural Network
    20
    

    View Slide

21. Prediction on an image - Inference
    21
    

    View Slide

22. Prediction on an image - Inference
    22
    

    View Slide

23. Prediction on an image - Inference
    Apple: 0.98
    Banana: 0.02
    23
    

    View Slide

24. Training a model
    

    View Slide

25. View Slide

26. Training a model - Back Propagation
    26
    

    View Slide

27. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    27
    

    View Slide

28. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    Prediction
    error
    28
    

    View Slide

29. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    Prediction
    error
    29
    

    View Slide

30. Training a model - Back Propagation
    Apple: 0.34
    Banana: 0.66
    Prediction
    error
    30
    

    View Slide

31. Training a model - Back Propagation
    Apple: 0.27
    Banana: 0.73
    31
    

    View Slide

32. Transfer Learning
    

    View Slide

33. Deep Convolutional Neural Network
    33
    

    View Slide

34. Transfer Learning
    • Use a pre-trained Deep Neural Network
    • Keep all operations but the last one
    • Re-train only the last operation to specialize your network to your classes
    Keep all weights identical
    except these ones
    34
    

    View Slide

35. Save the model
    • 2 things to save
    • Execution graph
    • Weights for each operation
    • 2 outputs
    • Model as protobuf file
    • Labels in text file
    35
    model.pb label.txt
    

    View Slide

36. java.lang.UnsupportedOperationException:
    Op BatchNormWithGlobalNormalization is not
    available in GraphDef version 21.
    36
    

    View Slide

37. Unsupported Operation
    • Only keep the operations dedicated to the inference step
    • Remove decoding, training, loss and evaluation operations
    37
    

    View Slide

38. Data Scientist
    Android Development Noob
    

    View Slide

39. CLICK 7 TIMES ON BUILD NUMBER
    39
    

    View Slide

40. Build Standalone App
    

    View Slide

41. Standalone App
    • Use nightly build
    • Library .so
    • Java API jar
    android {
    //…
    sourceSets {
    main {
    jniLibs.srcDirs = ['libs']
    }
    }
    }
    41
    

    View Slide

42. App size ~80MB
    42
    

    View Slide

43. Reducing model size
    

    View Slide

44. WHO CARES?
    MODEL SIZE
    44
    

    View Slide

45. Model Size
    All weights are stored as they are (64-bit floats) => 80MB
    45
    

    View Slide

46. 80 MB => 20 MB
    46
    Weights quantization
    6.372638493746383 => 6.4
    

    View Slide

47. Architecture Underneath
    

    View Slide

48. Android SDK (Java) Android NDK (C++)
    Classifier
    Implementation
    TensorFlow
    JNI wrapper
    Image (Bitmap)
    Trained Model
    top_results
    Classifications + Confidence
    input_tensor
    1 2
    3
    4
    Camera
    Preview
    Ref: https://jalammar.github.io/Supercharging-android-apps-using-tensorflow/
    Overlay
    Display
    48
    

    View Slide

49. Image Sampling
    Get Image from
    Camera Preview
    Crop the center square Resize Sample Image
    49
    

    View Slide

50. Converts YUV420 to ARGB8888
    public static native void
    convertYUV420ToARGB8888(
    byte[] y,
    byte[] u,
    byte[] v,
    int[] output,
    int width,
    int height,
    int yRowStride,
    int uvRowStride,
    int uvPixelStride,
    boolean halfSize
    );
    50
    

    View Slide

51. Create Input Tensor From RGB values
    // Preprocess the image data from 0-255 int to normalized float based
    // on the provided parameters.
    bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(),
    bitmap.getHeight());
    for (int i = 0; i < intValues.length; ++i) {
    final int val = intValues[i];
    floatValues[i * 3 + 0] = (((val >> 16) & 0xFF) - imageMean) / imageStd;
    floatValues[i * 3 + 1] = (((val >> 8) & 0xFF) - imageMean) / imageStd;
    floatValues[i * 3 + 2] = ((val & 0xFF) - imageMean) / imageStd;
    }
    inferenceInterface.feed(inputName, floatValues, 1, inputSize, inputSize, 3);
    51
    

    View Slide

52. Adding new models
    

    View Slide

53. Adding a new model
    53
    2 * 20 MB = 40 MB
    

    View Slide

54. Model Stacking
    54
    • Start from previous model to keep all specific operations in the graph
    • Specify all operations to keep when optimizing for inference
    graph_util.convert_variables_to_constants(sess, graph.as_graph_def(),
    [“final_result_fruits”, “final_result_vegetables”]
    

    View Slide

55. Demo Time
    

    View Slide

56. View Slide

57. What’s next?
    57
    

    View Slide

58. Next: Pulling Model From the Cloud
    58
    Ref: https://www.youtube.com/watch?v=EnFyneRScQ8
    

    View Slide

59. Next: Federate Learning
    Collaborative Machine Learning without Centralized Training Data
    59
    Ref: https://research.googleblog.com/2017/04/federated-learning-collaborative.html
    

    View Slide

60. Thank you! Questions?
    Github: https://github.com/xebia-france/magritte
    

    View Slide