Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Who's Afraid Of Machine Learning? & first steps with TensorFlow

Who's Afraid Of Machine Learning? & first steps with TensorFlow

Chicago Roboto & Android Makers 2018

Britt Barak

April 23, 2018
Tweet

More Decks by Britt Barak

Other Decks in Technology

Transcript

  1. Who’s afraid of
    Machine Learning?
    Britt Barak

    View Slide

  2. Britt Barak
    Google Developer Expert - Android
    Women Techmakers Israel
    Britt Barak @brittBarak

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. In a machine...

    View Slide

  12. View Slide

  13. Strawberry
    Not
    Strawberry

    View Slide

  14. Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.64
    0.75
    0.4

    View Slide

  15. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  16. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  17. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  18. 0.64
    0.75
    0.4
    Input
    0.5
    0.8
    0.3
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  19. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4

    View Slide

  20. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04

    View Slide

  21. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    + 0.7

    View Slide

  22. 0.64
    0.75
    0.4
    1.74
    0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    + 0.7
    ___________
    1.74
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.5
    0.8
    0.3

    View Slide

  23. 0.64
    0.75
    0.4
    1.02
    1.74
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.97

    View Slide

  24. 0.64
    0.75
    0.4
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    1.02
    1.74
    0.97

    View Slide

  25. 0.64
    0.75
    0.4
    Output
    Strawberry
    Not
    Strawberry
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    1.02
    1.74
    0.97
    0.87
    0.13

    View Slide

  26. 0.64
    0.75
    0.4
    0.87
    0.13
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    1.02
    1.74
    0.97

    View Slide

  27. View Slide

  28. 0.7
    0.03
    0.01
    3.72
    0.89
    1.92
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.2
    0.8

    View Slide

  29. 0.7
    0.03
    0.01
    3.72
    0.89
    1.92
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves
    0.2
    0.8

    View Slide

  30. 0.7
    0.03
    0.01
    3.72
    0.89
    1.92
    0.2
    0.8
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  31. 0.5 * 0.64
    + 0.8 * 0.75
    + 0.3 * 0.4
    ___________
    1.04
    + 0.7
    ___________
    1.74
    Strawberry Not Not
    Strawberry
    Not Not
    Strawberry Not
    Not

    View Slide

  32. Training
    TRAINING

    View Slide

  33. 0.64
    0.75
    0.4
    1.02
    1.74
    0.97
    0.89
    0.11
    Strawberry
    Not
    Strawberry
    Output
    Input
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  34. Strawberry
    Not
    Strawberry
    Output
    Input Hidden
    Red
    Seeds
    pattern
    Top
    leaves

    View Slide

  35. View Slide

  36. Data science

    View Slide

  37. We get a trained model !

    View Slide

  38. TensorFlow
    - Open source
    - Widely used
    - Flexible for scale:
    - 1 or more CPUs / GPUs
    - desktop, server, mobile device

    View Slide

  39. Strawberry

    View Slide

  40. Strawberry

    View Slide

  41. Strawberry
    ● Bandwidth
    ● Performance
    ● Latency
    ● Network
    ● Security
    ● Privacy
    ● …

    View Slide

  42. TensorFlow
    Mobile
    - Speech Recognition
    - Image Recognition
    - Object Localization
    - Gesture Recognition
    - Translation
    - Text Classification
    - Voice Synthesis

    View Slide

  43. Lightweight Fast Cross platform

    View Slide

  44. MobileNet Inception-V3 SmartReply
    Models

    View Slide

  45. View Slide

  46. Image
    Classifier
    classifier
    .classify(bitmap)
    label

    View Slide

  47. 1. Add Assets

    View Slide

  48. View Slide

  49. labels.txt
    strawberry
    orange
    lemon
    fig
    pineapple
    banana
    jackfruit
    custard apple
    pomegranate
    hay
    carbonara
    chocolate sauce
    dough
    meat loaf

    View Slide

  50. 2. Add TensorFlow Lite

    View Slide

  51. repositories {

    maven {

    url 'https://google.bintray.com/tensorflow'

    }

    }


    dependencies {

    // ...

    implementation 'org.tensorflow:tensorflow-lite:+'

    }
    build.gradle

    View Slide

  52. android {

    aaptOptions {

    noCompress "tflite"

    }

    }
    build.gradle

    View Slide

  53. 3. Create ImageClassifier.java

    View Slide

  54. Image
    Classifier

    View Slide

  55. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter();

    View Slide

  56. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);

    View Slide

  57. MappedByteBuffer loadModelFile() {

    AssetFileDescriptor descriptor=
    getAssets().openFd(MODEL_PATH);

    View Slide

  58. MappedByteBuffer loadModelFile() {

    AssetFileDescriptor descriptor=
    getAssets().openFd(MODEL_PATH);

    FileInputStream inputStream =
    new FileInputStream(descriptor.getFileDescriptor());
    FileChannel channel = inputStream.getChannel();


    View Slide

  59. MappedByteBuffer loadModelFile() {

    AssetFileDescriptor descriptor=
    getAssets().openFd(MODEL_PATH);

    FileInputStream inputStream =
    new FileInputStream(descriptor.getFileDescriptor());
    FileChannel channel = inputStream.getChannel();
    long start = descriptor.getStartOffset();

    long length = descriptor.getDeclaredLength();

    return
    channel.map(FileChannel.MapMode.READ_ONLY, start, length);
    }

    View Slide

  60. Image
    Classifier
    [strawberry, apple, ...
    ]
    labels.txt

    View Slide

  61. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    labelList = loadLabelList();

    View Slide

  62. labels.txt
    strawberry
    orange
    lemon
    fig
    pineapple
    banana
    jackfruit
    custard apple
    pomegranate
    hay
    carbonara
    chocolate sauce
    dough
    meat loaf

    View Slide

  63. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));

    }

    View Slide

  64. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    BufferedReader reader = new BufferedReader(inputStream);


    }

    View Slide

  65. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    BufferedReader reader = new BufferedReader(inputStream);

    List labelList = new ArrayList<>();
    String line;

    while ((line = reader.readLine()) != null) {

    labelList.add(line);

    }


    }

    View Slide

  66. List loadLabelList() throws IOException {

    InputStreamReader inputStream =
    new InputStreamReader(getAssets().open(LABEL_PATH));
    BufferedReader reader = new BufferedReader(inputStream);

    List labelList = new ArrayList<>();
    String line;

    while ((line = reader.readLine()) != null) {

    labelList.add(line);

    }

    reader.close();

    return labelList;

    }

    View Slide

  67. Image
    Classifier
    [ [0..6] , [ 0.1 ] , ...
    ]
    [strawberry, apple, ...
    ]
    probArray
    labels.txt

    View Slide

  68. probArray =
    {
    [0.7],
    [0.3],
    [0],
    [0],
    }
    labelList =
    {
    strawberry,
    apple,
    pineapple,
    banana,
    }
    0.3

    View Slide

  69. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    labelList = loadLabelList();
    probArray = new float[1][labelList.size()];

    View Slide

  70. Image
    Classifier
    [......] [ [0..6] , [ 0.1 ] , ...
    ]
    [strawberry, apple, ...
    ]
    ByteBuffer probArray
    labels.txt

    View Slide

  71. ImageClassifier.java
    model = loadModelFile();

    tflite = new Interpreter(model);
    labelList = loadLabelList();
    probArray = new float[1][labelList.size()];
    imgData =

    ByteBuffer.allocateDirect(
    DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE);
    imgData.order(ByteOrder.nativeOrder());

    View Slide

  72. 4. Run the model / classify

    View Slide

  73. classifier
    .classify(bitmap)
    Image
    Classifier
    [......] [ [0..6] , [ 0.1 ] , ...
    ]
    [strawberry, apple, ...
    ]
    ByteBuffer probArray
    labels.txt

    View Slide

  74. ImageClassifier.java
    String classify(Bitmap bitmap) {


    convertBitmapToByteBuffer(imgData, bitmap);



    }

    View Slide

  75. void convertBitmapToByteBuffer(Bitmap bitmap) {
    //...
    bitmap.getPixels(intValues, 0, bitmap.getWidth(),
    0, 0,bitmap.getWidth(), bitmap.getHeight());
    }

    View Slide

  76. void convertBitmapToByteBuffer(Bitmap bitmap) {
    //...
    bitmap.getPixels(intValues, 0, bitmap.getWidth(),
    0, 0,bitmap.getWidth(), bitmap.getHeight());
    int pixel = 0;
    for (int i = 0; i < DIM_IMG_SIZE_X; ++i) {
    for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) {
    final int val = intValues[pixel++];
    imgData.put((byte) ((val >> 16) & 0xFF));
    imgData.put((byte) ((val >> 8) & 0xFF));
    imgData.put((byte) (val & 0xFF));
    }
    }
    }

    View Slide

  77. void convertBitmapToByteBuffer(Bitmap bitmap) {
    //...
    bitmap.getPixels(intValues, 0, bitmap.getWidth(),
    0, 0,bitmap.getWidth(), bitmap.getHeight());
    int pixel = 0;
    for (int i = 0; i < DIM_IMG_SIZE_X; ++i) {
    for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) {
    final int val = intValues[pixel++];
    imgData.put((byte) ((val >> 16) & 0xFF));
    imgData.put((byte) ((val >> 8) & 0xFF));
    imgData.put((byte) (val & 0xFF));
    }
    }
    }

    View Slide

  78. ImageClassifier.java
    String classify(Bitmap bitmap) {


    convertBitmapToByteBuffer(imgData, bitmap);


    tflite.run(imgData, probArray);



    }

    View Slide

  79. ImageClassifier.java
    String classify(Bitmap bitmap) {


    convertBitmapToByteBuffer(imgData, bitmap);


    tflite.run(imgData, probArray);


    String textToShow = getTopLabels();

    return textToShow;

    }

    View Slide

  80. Strawberry - 0.87
    Apple - 0.13
    Tomato - 0.01

    View Slide

  81. Machine
    Learning
    is a
    new world

    View Slide

  82. Links
    - Tensorflow
    - https://www.tensorflow.org/
    - Tensorflow lite
    - https://www.tensorflow.org/mobile/tflite/
    - Codes labs
    - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/
    - Google’s Machine Learning Crash Course
    - developers.google.com/machine-learning/crash-course/
    - [Dr. Joe Dispenza]

    View Slide

  83. Thank you!
    Keep in touch!
    Britt Barak
    @brittBarak

    View Slide