Who's Afraid Of Machine Learning? & first steps with TensorFlow

Who's Afraid Of Machine Learning? & first steps with TensorFlow

Chicago Roboto & Android Makers 2018

3142db3adb711e247e371153b5777e04?s=128

Britt Barak

April 23, 2018
Tweet

Transcript

  1. Who’s afraid of Machine Learning? Britt Barak

  2. Britt Barak Google Developer Expert - Android Women Techmakers Israel

    Britt Barak @brittBarak
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. In a machine...

  12. None
  13. Strawberry Not Strawberry

  14. Input Red Seeds pattern Top leaves 0.64 0.75 0.4

  15. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

  16. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

  17. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves

  18. 0.64 0.75 0.4 Input 0.5 0.8 0.3 Red Seeds pattern

    Top leaves
  19. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4
  20. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04
  21. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5

    0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7
  22. 0.64 0.75 0.4 1.74 0.5 * 0.64 + 0.8 *

    0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Input Red Seeds pattern Top leaves 0.5 0.8 0.3
  23. 0.64 0.75 0.4 1.02 1.74 Input Red Seeds pattern Top

    leaves 0.97
  24. 0.64 0.75 0.4 Input Red Seeds pattern Top leaves 1.02

    1.74 0.97
  25. 0.64 0.75 0.4 Output Strawberry Not Strawberry Input Red Seeds

    pattern Top leaves 1.02 1.74 0.97 0.87 0.13
  26. 0.64 0.75 0.4 0.87 0.13 Strawberry Not Strawberry Output Input

    Red Seeds pattern Top leaves 1.02 1.74 0.97
  27. None
  28. 0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output

    Input Red Seeds pattern Top leaves 0.2 0.8
  29. 0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output

    Input Red Seeds pattern Top leaves 0.2 0.8
  30. 0.7 0.03 0.01 3.72 0.89 1.92 0.2 0.8 Strawberry Not

    Strawberry Output Input Red Seeds pattern Top leaves
  31. 0.5 * 0.64 + 0.8 * 0.75 + 0.3 *

    0.4 ___________ 1.04 + 0.7 ___________ 1.74 Strawberry Not Not Strawberry Not Not Strawberry Not Not
  32. Training TRAINING

  33. 0.64 0.75 0.4 1.02 1.74 0.97 0.89 0.11 Strawberry Not

    Strawberry Output Input Red Seeds pattern Top leaves
  34. Strawberry Not Strawberry Output Input Hidden Red Seeds pattern Top

    leaves
  35. None
  36. Data science

  37. We get a trained model !

  38. TensorFlow - Open source - Widely used - Flexible for

    scale: - 1 or more CPUs / GPUs - desktop, server, mobile device
  39. Strawberry

  40. Strawberry

  41. Strawberry • Bandwidth • Performance • Latency • Network •

    Security • Privacy • …
  42. TensorFlow Mobile - Speech Recognition - Image Recognition - Object

    Localization - Gesture Recognition - Translation - Text Classification - Voice Synthesis
  43. Lightweight Fast Cross platform

  44. MobileNet Inception-V3 SmartReply Models

  45. None
  46. Image Classifier classifier .classify(bitmap) label

  47. 1. Add Assets

  48. None
  49. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple

    pomegranate hay carbonara chocolate sauce dough meat loaf
  50. 2. Add TensorFlow Lite

  51. repositories {
 maven {
 url 'https://google.bintray.com/tensorflow'
 }
 }
 
 dependencies

    {
 // ...
 implementation 'org.tensorflow:tensorflow-lite:+'
 } build.gradle
  52. android {
 aaptOptions {
 noCompress "tflite"
 }
 } build.gradle

  53. 3. Create ImageClassifier.java

  54. Image Classifier

  55. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter();

  56. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model);

  57. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); 


  58. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
 FileInputStream inputStream = new

    FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel();

  59. MappedByteBuffer loadModelFile() {
 AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
 FileInputStream inputStream = new

    FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); long start = descriptor.getStartOffset();
 long length = descriptor.getDeclaredLength();
 return channel.map(FileChannel.MapMode.READ_ONLY, start, length); }
  60. Image Classifier [strawberry, apple, ... ] labels.txt

  61. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList =

    loadLabelList();
  62. labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple

    pomegranate hay carbonara chocolate sauce dough meat loaf
  63. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    
 }
  64. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    BufferedReader reader = new BufferedReader(inputStream);
 
 }
  65. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    BufferedReader reader = new BufferedReader(inputStream);
 List<String> labelList = new ArrayList<>(); String line;
 while ((line = reader.readLine()) != null) {
 labelList.add(line);
 }
 
 }
  66. List<String> loadLabelList() throws IOException {
 InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));

    BufferedReader reader = new BufferedReader(inputStream);
 List<String> labelList = new ArrayList<>(); String line;
 while ((line = reader.readLine()) != null) {
 labelList.add(line);
 }
 reader.close();
 return labelList; 
 }
  67. Image Classifier [ [0..6] , [ 0.1 ] , ...

    ] [strawberry, apple, ... ] probArray labels.txt
  68. probArray = { [0.7], [0.3], [0], [0], } labelList =

    { strawberry, apple, pineapple, banana, } 0.3
  69. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList =

    loadLabelList(); probArray = new float[1][labelList.size()];
  70. Image Classifier [......] [ [0..6] , [ 0.1 ] ,

    ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
  71. ImageClassifier.java model = loadModelFile();
 tflite = new Interpreter(model); labelList =

    loadLabelList(); probArray = new float[1][labelList.size()]; imgData =
 ByteBuffer.allocateDirect( DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE); imgData.order(ByteOrder.nativeOrder());
  72. 4. Run the model / classify

  73. classifier .classify(bitmap) Image Classifier [......] [ [0..6] , [ 0.1

    ] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
  74. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 


    }
  75. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); }
  76. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
  77. void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),

    bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
  78. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 tflite.run(imgData,

    probArray);
 
 
 }
  79. ImageClassifier.java String classify(Bitmap bitmap) {
 
 convertBitmapToByteBuffer(imgData, bitmap);
 
 tflite.run(imgData,

    probArray);
 
 String textToShow = getTopLabels();
 return textToShow;
 }
  80. Strawberry - 0.87 Apple - 0.13 Tomato - 0.01

  81. Machine Learning is a new world

  82. Links - Tensorflow - https://www.tensorflow.org/ - Tensorflow lite - https://www.tensorflow.org/mobile/tflite/

    - Codes labs - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/ - Google’s Machine Learning Crash Course - developers.google.com/machine-learning/crash-course/ - [Dr. Joe Dispenza]
  83. Thank you! Keep in touch! Britt Barak @brittBarak