Who's Afraid Of Machine Learning? & first steps with TensorFlow

Transcript

1. Who’s afraid of Machine Learning? Britt Barak

2. Britt Barak Google Developer Expert - Android Women Techmakers Israel

   Britt Barak @brittBarak
3. None
4. None
5. None
6. None
7. None
8. None
9. None
10. None

11. In a machine...

12. None

13. Strawberry Not Strawberry

14. Input Red Seeds pattern Top leaves 0.64 0.75 0.4

15. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

16. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

17. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

18. 0.64 0.75 0.4 Input 0.5 0.8 0.3 Red Seeds pattern

    Top leaves

19. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4

20. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04

21. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7

22. 0.64 0.75 0.4 1.74 0.5 * 0.64 + 0.8 *

    0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Input Red Seeds pattern Top leaves 0.5 0.8 0.3

23. 0.64 0.75 0.4 1.02 1.74 Input Red Seeds pattern Top

    leaves 0.97

24. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 1.02

    1.74 0.97

25. 0.64 0.75 0.4 Output Strawberry Not Strawberry Input Red Seeds

    pattern Top leaves 1.02 1.74 0.97 0.87 0.13

26. 0.64 0.75 0.4 0.87 0.13 Strawberry Not Strawberry Output Input

    Red Seeds pattern Top leaves 1.02 1.74 0.97
27. None

28. 0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output

    Input Red Seeds pattern Top leaves 0.2 0.8

29. 0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output

    Input Red Seeds pattern Top leaves 0.2 0.8

30. 0.7 0.03 0.01 3.72 0.89 1.92 0.2 0.8 Strawberry Not

    Strawberry Output Input Red Seeds pattern Top leaves

31. 0.5 * 0.64 + 0.8 * 0.75 + 0.3 *

    0.4 ___________ 1.04 + 0.7 ___________ 1.74 Strawberry Not Not Strawberry Not Not Strawberry Not Not

32. Training TRAINING

33. 0.64 0.75 0.4 1.02 1.74 0.97 0.89 0.11 Strawberry Not

    Strawberry Output Input Red Seeds pattern Top leaves

34. Strawberry Not Strawberry Output Input Hidden Red Seeds pattern Top

    leaves
35. None

36. Data science

37. We get a trained model !

38. TensorFlow - Open source - Widely used - Flexible for

    scale: - 1 or more CPUs / GPUs - desktop, server, mobile device

39. Strawberry

40. Strawberry

41. Strawberry • Bandwidth • Performance • Latency • Network •

    Security • Privacy • …

42. TensorFlow Mobile - Speech Recognition - Image Recognition - Object

    Localization - Gesture Recognition - Translation - Text Classification - Voice Synthesis

43. Lightweight Fast Cross platform

44. MobileNet Inception-V3 SmartReply Models

45. None

46. Image Classifier classifier .classify(bitmap) label

47. 1. Add Assets

48. None

49. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple

    pomegranate hay carbonara chocolate sauce dough meat loaf

50. 2. Add TensorFlow Lite

51. repositories {
 maven {
 url 'https://google.bintray.com/tensorflow'
 }
 }
 
 dependencies

    {
 // ...
 implementation 'org.tensorflow:tensorflow-lite:+'
 } build.gradle

52. android {
 aaptOptions {
 noCompress "tflite"
 }
 } build.gradle

53. 3. Create ImageClassifier.java

54. Image Classifier

55. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter();

56. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model);

57. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); 


58. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
 FileInputStream inputStream = new

    FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel();


59. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
 FileInputStream inputStream = new

    FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); long start = descriptor.getStartOffset();
 long length = descriptor.getDeclaredLength();
 return channel.map(FileChannel.MapMode.READ_ONLY, start, length); }

60. Image Classifier [strawberry, apple, ... ] labels.txt

61. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList =

    loadLabelList();

62. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple

    pomegranate hay carbonara chocolate sauce dough meat loaf

63. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    
 }

64. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    BufferedReader reader = new BufferedReader(inputStream);
 
 }

65. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    BufferedReader reader = new BufferedReader(inputStream);
 List<String> labelList = new ArrayList<>(); String line;
 while ((line = reader.readLine()) != null) {
 labelList.add(line);
 }
 
 }

66. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    BufferedReader reader = new BufferedReader(inputStream);
 List<String> labelList = new ArrayList<>(); String line;
 while ((line = reader.readLine()) != null) {
 labelList.add(line);
 }
 reader.close();
 return labelList; 
 }

67. Image Classifier [ [0..6] , [ 0.1 ] , ...

    ] [strawberry, apple, ... ] probArray labels.txt

68. probArray = { [0.7], [0.3], [0], [0], } labelList =

    { strawberry, apple, pineapple, banana, } 0.3

69. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList =

    loadLabelList(); probArray = new float[1][labelList.size()];

70. Image Classifier [......] [ [0..6] , [ 0.1 ] ,

    ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt

71. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList =

    loadLabelList(); probArray = new float[1][labelList.size()]; imgData =
 ByteBuffer.allocateDirect( DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE); imgData.order(ByteOrder.nativeOrder());

72. 4. Run the model / classify

73. classifier .classify(bitmap) Image Classifier [......] [ [0..6] , [ 0.1

    ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt

74. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 


    }

75. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); }

76. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }

77. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }

78. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 tflite.run(imgData,

    probArray);
 
 
 }

79. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 tflite.run(imgData,

    probArray);
 
 String textToShow = getTopLabels();
 return textToShow;
 }

80. Strawberry - 0.87 Apple - 0.13 Tomato - 0.01

81. Machine Learning is a new world

82. Links - Tensorflow - https://www.tensorflow.org/ - Tensorflow lite - https://www.tensorflow.org/mobile/tflite/

    - Codes labs - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/ - Google’s Machine Learning Crash Course - developers.google.com/machine-learning/crash-course/ - [Dr. Joe Dispenza]

83. Thank you! Keep in touch! Britt Barak @brittBarak