Who's Afraid Of Machine Learning?

Devfest Pisa 2018

Intro to Machine Learning and the crazy stuff everyone's now taking about.. and a bit of TensorFlow Lite for Android.

Transcript

1. Who’s afraid of Machine Learning? Britt Barak
2. Britt Barak Google Developer Expert - Android Women Techmakers Israel Britt Barak @brittBarak
12. In a machine...
14. Strawberry Not Strawberry
15. Input Red Seeds pattern Top leaves
16. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves
17. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves
18. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves
19. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5 0.8 0.3
20. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4
21. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04
22. 0.64 0.75 0.4 Red Seeds pattern Top leaves 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ Input
23. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.74 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Input
24. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.74 Input
25. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74 0.97 Input
26. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74 0.97 Output Strawberry Not Strawberry Input
27. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74 0.97 0.87 0.13 Strawberry Not Strawberry Output Input
29. 0.7 0.03 0.01 Red Seeds pattern Top leaves Strawberry Not Strawberry Output Input
30. 0.7 0.03 0.01 Red Seeds pattern Top leaves 3.72 0.89 1.92 Strawberry Not Strawberry Output Input
31. 0.7 0.03 0.01 Red Seeds pattern Top leaves 3.72 0.89 1.92 0.2 0.8 Strawberry Not Strawberry Output Input
32. Strawberry Not Not Strawberry Not Not Strawberry Not Not 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74
33. Training
34. 0.64 0.75 0.4 Red Seeds pattern Top leaves 1.02 1.74 0.97 0.89 0.11 Strawberry Not Strawberry Output Input
35. Red Seeds pattern Top leaves Strawberry Not Strawberry Output Input Hidden
37. Data science
38. TensorFlow - Open source - Widely used - Flexible for scale: - 1 or more CPUs / GPUs - desktop, server, mobile device
39. Strawberry
40. Strawberry
41. TensorFlow Mobile - Speech Recognition - Image Recognition - Object Localization - Gesture Recognition - Translation - Text Classification - Voice Synthesis
42. Lightweight Fast Cross platform
43. MobileNet Inception-V3 SmartReply Models
44. Example implementation
45. Image Classifier classifier .classify(bitmap) label
46. 1. Add assets
48. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple pomegranate hay carbonara chocolate sauce dough meat loaf pizza
49. 2. Add TensorFlow Lite
50. repositories { maven { url 'https://google.bintray.com/tensorflow' } } dependencies { // ... implementation 'org.tensorflow:tensorflow-lite:+' } build.gradle
51. android { aaptOptions { noCompress "tflite" noCompress "lite" } } build.gradle
52. 3. Create ImageClassifier.java
53. Image Classifier
54. ImageClassifier.java tflite = new Interpreter();
55. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model);
56. MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); }
57. MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); FileInputStream inputStream = new FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); }
58. MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); FileInputStream inputStream = new FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); long start = descriptor.getStartOffset(); long length = descriptor.getDeclaredLength(); return channel.map(FileChannel.MapMode.READ_ONLY, start, length); }
59. Image Classifier [strawberry, apple, ... ] labels.txt
60. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList = loadLabelList();
61. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple pomegranate hay carbonara chocolate sauce dough meat loaf pizza
62. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); }
63. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream); }
64. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream); List<String> labelList = new ArrayList<String>(); String line; while ((line = reader.readLine()) != null) { labelList.add(line); } }
65. List<String> loadLabelList() throws IOException { ` InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream); List<String> labelList = new ArrayList<String>(); String line; while ((line = reader.readLine()) != null) { labelList.add(line); } reader.close(); return labelList; }
66. Image Classifier [ [0..6] , [ 0.1 ] , ... ] [strawberry, apple, ... ] probArray labels.txt
67. probArray = { [0.7], [0.3], [0], [0], } labelList = { strawberry, apple, pineapple, banana } 0.3
68. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList = loadLabelList(); probArray = new float[1][labelList.size()];
69. Image Classifier [......] [ [0..6] , [ 0.1 ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
70. ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList = loadLabelList(); probArray = new float[1][labelList.size()]; imgData = ByteBuffer.allocateDirect( DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE); imgData.order(ByteOrder.nativeOrder());
71. 4. Run the model / classify
72. classifier .classify(bitmap) Image Classifier [......] [ [0..6] , [ 0.1 ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
73. ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(bitmap); }
74. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(), bitmap.getHeight()); } }
75. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(), bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
76. ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(bitmap); tflite.run(imgData, probArray);
77. ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(bitmap); tflite.run(imgData, probArray); String textToShow = getTopLabels(); return textToShow; }
78. private String getTopKLabels() { for (int i = 0; i < labelList.size(); ++i) { sortedLabels.add( new AbstractMap.SimpleEntry<>(labelList.get(i), (labelProbArray[0][i] & 0xff) / 255.0f)); if (sortedLabels.size() > RESULTS_TO_SHOW) { sortedLabels.poll(); } } String textToShow = ""; final int size = sortedLabels.size(); for (int i = 0; i < size; ++i) { Map.Entry<String, Float> label = sortedLabels.poll(); textToShow = String.format("\n%s: %4.2f", label.getKey(), label.getValue()) + textToShow; } return textToShow; }
79. Strawberry - 0.87 Apple - 0.13 Tomato - 0.01
80. Machine Learning is a new world
81. Links - Tensorflow - https://www.tensorflow.org/ - Tensorflow lite - https://www.tensorflow.org/mobile/tflite/ - Codes labs - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/ - Google’s Machine Learning Crash Course - developers.google.com/machine-learning/crash-course/ - [Dr. Joe Dispenza]
82. Thank you! Keep in touch! Britt Barak @brittBarak