Who's Afraid Of Machine Learning? & first steps with TensorFlow

Chicago Roboto & Android Makers 2018

Transcript

1. Who’s afraid of Machine Learning? Britt Barak
2. Britt Barak Google Developer Expert - Android Women Techmakers Israel Britt Barak @brittBarak
11. In a machine...
13. Strawberry Not Strawberry
14. Input Red Seeds pattern Top leaves 0.64 0.75 0.4
15. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves
16. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves
17. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves
18. 0.64 0.75 0.4 Input 0.5 0.8 0.3 Red Seeds pattern Top leaves
19. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4
20. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04
21. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5 0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7
22. 0.64 0.75 0.4 1.74 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Input Red Seeds pattern Top leaves 0.5 0.8 0.3
23. 0.64 0.75 0.4 1.02 1.74 Input Red Seeds pattern Top leaves 0.97
24. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 1.02 1.74 0.97
25. 0.64 0.75 0.4 Output Strawberry Not Strawberry Input Red Seeds pattern Top leaves 1.02 1.74 0.97 0.87 0.13
26. 0.64 0.75 0.4 0.87 0.13 Strawberry Not Strawberry Output Input Red Seeds pattern Top leaves 1.02 1.74 0.97
28. 0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output Input Red Seeds pattern Top leaves 0.2 0.8
29. 0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output Input Red Seeds pattern Top leaves 0.2 0.8
30. 0.7 0.03 0.01 3.72 0.89 1.92 0.2 0.8 Strawberry Not Strawberry Output Input Red Seeds pattern Top leaves
31. 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Strawberry Not Not Strawberry Not Not Strawberry Not Not
32. Training TRAINING
33. 0.64 0.75 0.4 1.02 1.74 0.97 0.89 0.11 Strawberry Not Strawberry Output Input Red Seeds pattern Top leaves
34. Strawberry Not Strawberry Output Input Hidden Red Seeds pattern Top leaves
36. Data science
37. We get a trained model !
38. TensorFlow - Open source - Widely used - Flexible for scale: - 1 or more CPUs / GPUs - desktop, server, mobile device
39. Strawberry
40. Strawberry
41. Strawberry • Bandwidth • Performance • Latency • Network • Security • Privacy • …
42. TensorFlow Mobile - Speech Recognition - Image Recognition - Object Localization - Gesture Recognition - Translation - Text Classification - Voice Synthesis
43. Lightweight Fast Cross platform
44. MobileNet Inception-V3 SmartReply Models
46. Image Classifier classifier .classify(bitmap) label
47. 1. Add Assets
49. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple pomegranate hay carbonara chocolate sauce dough meat loaf
50. 2. Add TensorFlow Lite
51. repositories {
 maven {
 url 'https://google.bintray.com/tensorflow'
 }
 }
 
 dependencies {
 // ...
 implementation 'org.tensorflow:tensorflow-lite:+'
 } build.gradle
52. android {
 aaptOptions {
 noCompress "tflite"
 }
 } build.gradle
53. 3. Create ImageClassifier.java
54. Image Classifier
55. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter();
56. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model);
57. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); 

58. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
 FileInputStream inputStream = new FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel();

59. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
 FileInputStream inputStream = new FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); long start = descriptor.getStartOffset();
 long length = descriptor.getDeclaredLength();
 return channel.map(FileChannel.MapMode.READ_ONLY, start, length); }
60. Image Classifier [strawberry, apple, ... ] labels.txt
61. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList = loadLabelList();
62. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple pomegranate hay carbonara chocolate sauce dough meat loaf
63. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); 
 }
64. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream);
 
 }
65. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream);
 List<String> labelList = new ArrayList<>(); String line;
 while ((line = reader.readLine()) != null) {
 labelList.add(line);
 }
 
 }
66. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH)); BufferedReader reader = new BufferedReader(inputStream);
 List<String> labelList = new ArrayList<>(); String line;
 while ((line = reader.readLine()) != null) {
 labelList.add(line);
 }
 reader.close();
 return labelList; 
 }
67. Image Classifier [ [0..6] , [ 0.1 ] , ... ] [strawberry, apple, ... ] probArray labels.txt
68. probArray = { [0.7], [0.3], [0], [0], } labelList = { strawberry, apple, pineapple, banana, } 0.3
69. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList = loadLabelList(); probArray = new float[1][labelList.size()];
70. Image Classifier [......] [ [0..6] , [ 0.1 ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
71. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList = loadLabelList(); probArray = new float[1][labelList.size()]; imgData =
 ByteBuffer.allocateDirect( DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE); imgData.order(ByteOrder.nativeOrder());
72. 4. Run the model / classify
73. classifier .classify(bitmap) Image Classifier [......] [ [0..6] , [ 0.1 ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
74. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 
 }
75. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(), bitmap.getHeight()); }
76. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(), bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
77. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(), bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
78. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 tflite.run(imgData, probArray);
 
 
 }
79. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 tflite.run(imgData, probArray);
 
 String textToShow = getTopLabels();
 return textToShow;
 }
80. Strawberry - 0.87 Apple - 0.13 Tomato - 0.01
81. Machine Learning is a new world
82. Links - Tensorflow - https://www.tensorflow.org/ - Tensorflow lite - https://www.tensorflow.org/mobile/tflite/ - Codes labs - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/ - Google’s Machine Learning Crash Course - developers.google.com/machine-learning/crash-course/ - [Dr. Joe Dispenza]
83. Thank you! Keep in touch! Britt Barak @brittBarak