Who's Afraid Of Machine Learning? & first steps with TensorFlow

Chicago Roboto & Android Makers 2018

Transcript

1. Who’s afraid of
   Machine Learning?
   Britt Barak
   

   View Slide

2. Britt Barak
   Google Developer Expert - Android
   Women Techmakers Israel
   Britt Barak @brittBarak
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. In a machine...
    

    View Slide

12. View Slide

13. Strawberry
    Not
    Strawberry
    

    View Slide

14. Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.64
    0.75
    0.4
    

    View Slide

15. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

16. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

17. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

18. 0.64
    0.75
    0.4
    Input
    0.5
    0.8
    0.3
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

19. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    

    View Slide

20. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    

    View Slide

21. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    + 0.7
    

    View Slide

22. 0.64
    0.75
    0.4
    1.74
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    + 0.7
    ___________
    1.74
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    

    View Slide

23. 0.64
    0.75
    0.4
    1.02
    1.74
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.97
    

    View Slide

24. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    1.02
    1.74
    0.97
    

    View Slide

25. 0.64
    0.75
    0.4
    Output
    Strawberry
    Not
    Strawberry
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    1.02
    1.74
    0.97
    0.87
    0.13
    

    View Slide

26. 0.64
    0.75
    0.4
    0.87
    0.13
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    1.02
    1.74
    0.97
    

    View Slide

27. View Slide

28. 0.7
    0.03
    0.01
    3.72
    0.89
    1.92
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.2
    0.8
    

    View Slide

29. 0.7
    0.03
    0.01
    3.72
    0.89
    1.92
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.2
    0.8
    

    View Slide

30. 0.7
    0.03
    0.01
    3.72
    0.89
    1.92
    0.2
    0.8
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

31. 0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    + 0.7
    ___________
    1.74
    Strawberry Not Not
    Strawberry
    Not Not
    Strawberry Not
    Not
    

    View Slide

32. Training
    TRAINING
    

    View Slide

33. 0.64
    0.75
    0.4
    1.02
    1.74
    0.97
    0.89
    0.11
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

34. Strawberry
    Not
    Strawberry
    Output
    Input Hidden
    Red
    Seeds
    pattern
    Top
    leaves
    

    View Slide

35. View Slide

36. Data science
    

    View Slide

37. We get a trained model !
    

    View Slide

38. TensorFlow
    - Open source
    - Widely used
    - Flexible for scale:
    - 1 or more CPUs / GPUs
    - desktop, server, mobile device
    

    View Slide

39. Strawberry
    

    View Slide

40. Strawberry
    

    View Slide

41. Strawberry
    ● Bandwidth
    ● Performance
    ● Latency
    ● Network
    ● Security
    ● Privacy
    ● …
    

    View Slide

42. TensorFlow
    Mobile
    - Speech Recognition
    - Image Recognition
    - Object Localization
    - Gesture Recognition
    - Translation
    - Text Classification
    - Voice Synthesis
    

    View Slide

43. Lightweight Fast Cross platform
    

    View Slide

44. MobileNet Inception-V3 SmartReply
    Models
    

    View Slide

45. View Slide

46. Image
    Classifier
    classifier
    .classify(bitmap)
    label
    

    View Slide

47. 1. Add Assets
    

    View Slide

48. View Slide

49. labels.txt
    strawberry
    orange
    lemon
    fig
    pineapple
    banana
    jackfruit
    custard apple
    pomegranate
    hay
    carbonara
    chocolate sauce
    dough
    meat loaf
    

    View Slide

50. 2. Add TensorFlow Lite
    

    View Slide

51. repositories {

    maven {

    url 'https://google.bintray.com/tensorflow'

    }

    }

    

    dependencies {

    // ...

    implementation 'org.tensorflow:tensorflow-lite:+'

    }
    build.gradle
    

    View Slide

52. android {

    aaptOptions {

    noCompress "tflite"

    }

    }
    build.gradle
    

    View Slide

53. 3. Create ImageClassifier.java
    

    View Slide

54. Image
    Classifier
    

    View Slide

55. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter();
    

    View Slide

56. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    

    View Slide

57. MappedByteBuffer loadModelFile() {

    AssetFileDescriptor descriptor=
    getAssets().openFd(MODEL_PATH);
    

    

    View Slide

58. MappedByteBuffer loadModelFile() {

    AssetFileDescriptor descriptor=
    getAssets().openFd(MODEL_PATH);

    FileInputStream inputStream =
    new FileInputStream(descriptor.getFileDescriptor());
    FileChannel channel = inputStream.getChannel();

    

    View Slide

59. MappedByteBuffer loadModelFile() {

    AssetFileDescriptor descriptor=
    getAssets().openFd(MODEL_PATH);

    FileInputStream inputStream =
    new FileInputStream(descriptor.getFileDescriptor());
    FileChannel channel = inputStream.getChannel();
    long start = descriptor.getStartOffset();

    long length = descriptor.getDeclaredLength();

    return
    channel.map(FileChannel.MapMode.READ_ONLY, start, length);
    }
    

    View Slide

60. Image
    Classifier
    [strawberry, apple, ...
    ]
    labels.txt
    

    View Slide

61. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    labelList = loadLabelList();
    

    View Slide

62. labels.txt
    strawberry
    orange
    lemon
    fig
    pineapple
    banana
    jackfruit
    custard apple
    pomegranate
    hay
    carbonara
    chocolate sauce
    dough
    meat loaf
    

    View Slide

63. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    

    }
    

    View Slide

64. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    BufferedReader reader = new BufferedReader(inputStream);

    

    }
    

    View Slide

65. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    BufferedReader reader = new BufferedReader(inputStream);

    List labelList = new ArrayList<>();
    String line;

    while ((line = reader.readLine()) != null) {

    labelList.add(line);

    }

    

    }
    

    View Slide

66. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    BufferedReader reader = new BufferedReader(inputStream);

    List labelList = new ArrayList<>();
    String line;

    while ((line = reader.readLine()) != null) {

    labelList.add(line);

    }

    reader.close();

    return labelList;
    

    }
    

    View Slide

67. Image
    Classifier
    [ [0..6] , [ 0.1 ] , ...
    ]
    [strawberry, apple, ...
    ]
    probArray
    labels.txt
    

    View Slide

68. probArray =
    {
    [0.7],
    [0.3],
    [0],
    [0],
    }
    labelList =
    {
    strawberry,
    apple,
    pineapple,
    banana,
    }
    0.3
    

    View Slide

69. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    labelList = loadLabelList();
    probArray = new float[1][labelList.size()];
    

    View Slide

70. Image
    Classifier
    [......] [ [0..6] , [ 0.1 ] , ...
    ]
    [strawberry, apple, ...
    ]
    ByteBuffer probArray
    labels.txt
    

    View Slide

71. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    labelList = loadLabelList();
    probArray = new float[1][labelList.size()];
    imgData =

    ByteBuffer.allocateDirect(
    DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE);
    imgData.order(ByteOrder.nativeOrder());
    

    View Slide

72. 4. Run the model / classify
    

    View Slide

73. classifier
    .classify(bitmap)
    Image
    Classifier
    [......] [ [0..6] , [ 0.1 ] , ...
    ]
    [strawberry, apple, ...
    ]
    ByteBuffer probArray
    labels.txt
    

    View Slide

74. ImageClassifier.java
    String classify(Bitmap bitmap) {

    

    convertBitmapToByteBuffer(imgData, bitmap);

    

    

    }
    

    View Slide

75. void convertBitmapToByteBuffer(Bitmap bitmap) {
    //...
    bitmap.getPixels(intValues, 0, bitmap.getWidth(),
    0, 0,bitmap.getWidth(), bitmap.getHeight());
    }
    

    View Slide

76. void convertBitmapToByteBuffer(Bitmap bitmap) {
    //...
    bitmap.getPixels(intValues, 0, bitmap.getWidth(),
    0, 0,bitmap.getWidth(), bitmap.getHeight());
    int pixel = 0;
    for (int i = 0; i < DIM_IMG_SIZE_X; ++i) {
    for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) {
    final int val = intValues[pixel++];
    imgData.put((byte) ((val >> 16) & 0xFF));
    imgData.put((byte) ((val >> 8) & 0xFF));
    imgData.put((byte) (val & 0xFF));
    }
    }
    }
    

    View Slide

77. void convertBitmapToByteBuffer(Bitmap bitmap) {
    //...
    bitmap.getPixels(intValues, 0, bitmap.getWidth(),
    0, 0,bitmap.getWidth(), bitmap.getHeight());
    int pixel = 0;
    for (int i = 0; i < DIM_IMG_SIZE_X; ++i) {
    for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) {
    final int val = intValues[pixel++];
    imgData.put((byte) ((val >> 16) & 0xFF));
    imgData.put((byte) ((val >> 8) & 0xFF));
    imgData.put((byte) (val & 0xFF));
    }
    }
    }
    

    View Slide

78. ImageClassifier.java
    String classify(Bitmap bitmap) {

    

    convertBitmapToByteBuffer(imgData, bitmap);

    

    tflite.run(imgData, probArray);

    

    

    }
    

    View Slide

79. ImageClassifier.java
    String classify(Bitmap bitmap) {

    

    convertBitmapToByteBuffer(imgData, bitmap);

    

    tflite.run(imgData, probArray);

    

    String textToShow = getTopLabels();

    return textToShow;

    }
    

    View Slide

80. Strawberry - 0.87
    Apple - 0.13
    Tomato - 0.01
    

    View Slide

81. Machine
    Learning
    is a
    new world
    

    View Slide

82. Links
    - Tensorflow
    - https://www.tensorflow.org/
    - Tensorflow lite
    - https://www.tensorflow.org/mobile/tflite/
    - Codes labs
    - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/
    - Google’s Machine Learning Crash Course
    - developers.google.com/machine-learning/crash-course/
    - [Dr. Joe Dispenza]
    

    View Slide

83. Thank you!
    Keep in touch!
    Britt Barak
    @brittBarak
    

    View Slide