Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Who's Afraid Of Machine Learning?

Who's Afraid Of Machine Learning?

Devfest Pisa 2018

Intro to Machine Learning and the crazy stuff everyone's now taking about.. and a bit of TensorFlow Lite for Android.

3142db3adb711e247e371153b5777e04?s=128

Britt Barak

March 10, 2018
Tweet

Transcript

  1. Who’s afraid of Machine Learning? Britt Barak

  2. Britt Barak Google Developer Expert - Android Women Techmakers Israel

    Britt Barak @brittBarak
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. In a machine...

  13. None
  14. Strawberry Not Strawberry

  15. Input Red Seeds pattern Top leaves

  16. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

  17. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

  18. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

  19. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3
  20. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4
  21. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04
  22. 0.64 0.75 0.4 Red Seeds pattern Top leaves 0.5 0.8

    0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ Input
  23. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.74 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Input
  24. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.74 Input

  25. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74

    0.97 Input
  26. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74

    0.97 Output Strawberry Not Strawberry Input
  27. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74

    0.97 0.87 0.13 Strawberry Not Strawberry Output Input
  28. None
  29. 0.7 0.03 0.01 Red Seeds pattern Top leaves Strawberry Not

    Strawberry Output Input
  30. 0.7 0.03 0.01 Red Seeds pattern Top leaves 3.72 0.89

    1.92 Strawberry Not Strawberry Output Input
  31. 0.7 0.03 0.01 Red Seeds pattern Top leaves 3.72 0.89

    1.92 0.2 0.8 Strawberry Not Strawberry Output Input
  32. Strawberry Not Not Strawberry Not Not Strawberry Not Not 0.5

    * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74
  33. Training

  34. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74

    0.97 0.89 0.11 Strawberry Not Strawberry Output Input
  35. Red Seeds pattern Top leaves Strawberry Not Strawberry Output Input

    Hidden
  36. None
  37. Data science

  38. TensorFlow - Open source - Widely used - Flexible for

    scale: - 1 or more CPUs / GPUs - desktop, server, mobile device
  39. Strawberry

  40. Strawberry

  41. TensorFlow Mobile - Speech Recognition - Image Recognition - Object

    Localization - Gesture Recognition - Translation - Text Classification - Voice Synthesis
  42. Lightweight Fast Cross platform

  43. MobileNet Inception-V3 SmartReply Models

  44. Example implementation

  45. Image Classifier classifier .classify(bitmap) label

  46. 1. Add assets

  47. None
  48. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple

    pomegranate hay carbonara chocolate sauce dough meat loaf pizza
  49. 2. Add TensorFlow Lite

  50. repositories { maven { url 'https://google.bintray.com/tensorflow' } } dependencies {

    // ... implementation 'org.tensorflow:tensorflow-lite:+' } build.gradle
  51. android { aaptOptions { noCompress "tflite" noCompress "lite" } }

    build.gradle
  52. 3. Create ImageClassifier.java

  53. Image Classifier

  54. ImageClassifier.java tflite = new Interpreter();

  55. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model);

  56. MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); }

  57. MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); FileInputStream inputStream = new

    FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); }
  58. MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); FileInputStream inputStream = new

    FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); long start = descriptor.getStartOffset(); long length = descriptor.getDeclaredLength(); return channel.map(FileChannel.MapMode.READ_ONLY, start, length); }
  59. Image Classifier [strawberry, apple, ... ] labels.txt

  60. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList =

    loadLabelList();
  61. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple

    pomegranate hay carbonara chocolate sauce dough meat loaf pizza
  62. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new

    InputStreamReader(getAssets().open(LABEL_PATH)); }
  63. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new

    InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream); }
  64. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new

    InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream); List<String> labelList = new ArrayList<String>(); String line; while ((line = reader.readLine()) != null) { labelList.add(line); } }
  65. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new

    InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream); List<String> labelList = new ArrayList<String>(); String line; while ((line = reader.readLine()) != null) { labelList.add(line); } reader.close(); return labelList; }
  66. Image Classifier [ [0..6] , [ 0.1 ] , ...

    ] [strawberry, apple, ... ] probArray labels.txt
  67. probArray = { [0.7], [0.3], [0], [0], } labelList =

    { strawberry, apple, pineapple, banana } 0.3
  68. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList =

    loadLabelList(); probArray = new float[1][labelList.size()];
  69. Image Classifier [......] [ [0..6] , [ 0.1 ] ,

    ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
  70. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList =

    loadLabelList(); probArray = new float[1][labelList.size()]; imgData = ByteBuffer.allocateDirect( DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE); imgData.order(ByteOrder.nativeOrder());
  71. 4. Run the model / classify

  72. classifier .classify(bitmap) Image Classifier [......] [ [0..6] , [ 0.1

    ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
  73. ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(bitmap); }

  74. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); } }
  75. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
  76. ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(bitmap); tflite.run(imgData, probArray);

  77. ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(bitmap); tflite.run(imgData, probArray); String textToShow

    = getTopLabels(); return textToShow; }
  78. private String getTopKLabels() { for (int i = 0; i

    < labelList.size(); ++i) { sortedLabels.add( new AbstractMap.SimpleEntry<>(labelList.get(i), (labelProbArray[0][i] & 0xff) / 255.0f)); if (sortedLabels.size() > RESULTS_TO_SHOW) { sortedLabels.poll(); } } String textToShow = ""; final int size = sortedLabels.size(); for (int i = 0; i < size; ++i) { Map.Entry<String, Float> label = sortedLabels.poll(); textToShow = String.format("\n%s: %4.2f", label.getKey(), label.getValue()) + textToShow; } return textToShow; }
  79. Strawberry - 0.87 Apple - 0.13 Tomato - 0.01

  80. Machine Learning is a new world

  81. Links - Tensorflow - https://www.tensorflow.org/ - Tensorflow lite - https://www.tensorflow.org/mobile/tflite/

    - Codes labs - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/ - Google’s Machine Learning Crash Course - developers.google.com/machine-learning/crash-course/ - [Dr. Joe Dispenza]
  82. Thank you! Keep in touch! Britt Barak @brittBarak