Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Who's Afraid Of Machine Learning? & first steps with TensorFlow
Britt Barak
April 23, 2018
Technology
5
490
Who's Afraid Of Machine Learning? & first steps with TensorFlow
Chicago Roboto & Android Makers 2018
Britt Barak
April 23, 2018
Tweet
Share
More Decks by Britt Barak
See All by Britt Barak
brittbarak
1
91
brittbarak
2
170
brittbarak
2
1.2k
brittbarak
2
650
brittbarak
1
57
brittbarak
0
110
brittbarak
8
930
brittbarak
1
310
brittbarak
0
330
Other Decks in Technology
See All in Technology
con_mame
4
2k
hirosys
0
140
kakka
0
3.6k
kawaguti
0
120
suzukiry
0
210
y0hgi
1
390
hsano
0
130
line_developers
PRO
3
510
kraj
0
1.6k
smzksts
0
240
ama_ch
0
3.6k
satoshirobatofujimoto
0
110
Featured
See All Featured
hatefulcrawdad
257
17k
jnunemaker
PRO
40
4.5k
rasmusluckow
318
18k
reverentgeek
168
7.1k
zenorocha
297
40k
eitanlees
111
9.9k
andyhume
62
3.4k
caitiem20
308
17k
malarkey
119
16k
addyosmani
494
110k
scottboms
251
11k
swwweet
206
6.8k
Transcript
Who’s afraid of Machine Learning? Britt Barak
Britt Barak Google Developer Expert - Android Women Techmakers Israel
Britt Barak @brittBarak
None
None
None
None
None
None
None
None
In a machine...
None
Strawberry Not Strawberry
Input Red Seeds pattern Top leaves 0.64 0.75 0.4
0.64 0.75 0.4 Input Red Seeds pattern Top leaves
0.64 0.75 0.4 Input Red Seeds pattern Top leaves
0.64 0.75 0.4 Input Red Seeds pattern Top leaves
0.64 0.75 0.4 Input 0.5 0.8 0.3 Red Seeds pattern
Top leaves
0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5
0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4
0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5
0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04
0.64 0.75 0.4 Input Red Seeds pattern Top leaves 0.5
0.8 0.3 0.5 * 0.64 + 0.8 * 0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7
0.64 0.75 0.4 1.74 0.5 * 0.64 + 0.8 *
0.75 + 0.3 * 0.4 ___________ 1.04 + 0.7 ___________ 1.74 Input Red Seeds pattern Top leaves 0.5 0.8 0.3
0.64 0.75 0.4 1.02 1.74 Input Red Seeds pattern Top
leaves 0.97
0.64 0.75 0.4 Input Red Seeds pattern Top leaves 1.02
1.74 0.97
0.64 0.75 0.4 Output Strawberry Not Strawberry Input Red Seeds
pattern Top leaves 1.02 1.74 0.97 0.87 0.13
0.64 0.75 0.4 0.87 0.13 Strawberry Not Strawberry Output Input
Red Seeds pattern Top leaves 1.02 1.74 0.97
None
0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output
Input Red Seeds pattern Top leaves 0.2 0.8
0.7 0.03 0.01 3.72 0.89 1.92 Strawberry Not Strawberry Output
Input Red Seeds pattern Top leaves 0.2 0.8
0.7 0.03 0.01 3.72 0.89 1.92 0.2 0.8 Strawberry Not
Strawberry Output Input Red Seeds pattern Top leaves
0.5 * 0.64 + 0.8 * 0.75 + 0.3 *
0.4 ___________ 1.04 + 0.7 ___________ 1.74 Strawberry Not Not Strawberry Not Not Strawberry Not Not
Training TRAINING
0.64 0.75 0.4 1.02 1.74 0.97 0.89 0.11 Strawberry Not
Strawberry Output Input Red Seeds pattern Top leaves
Strawberry Not Strawberry Output Input Hidden Red Seeds pattern Top
leaves
None
Data science
We get a trained model !
TensorFlow - Open source - Widely used - Flexible for
scale: - 1 or more CPUs / GPUs - desktop, server, mobile device
Strawberry
Strawberry
Strawberry • Bandwidth • Performance • Latency • Network •
Security • Privacy • …
TensorFlow Mobile - Speech Recognition - Image Recognition - Object
Localization - Gesture Recognition - Translation - Text Classification - Voice Synthesis
Lightweight Fast Cross platform
MobileNet Inception-V3 SmartReply Models
None
Image Classifier classifier .classify(bitmap) label
1. Add Assets
None
labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple
pomegranate hay carbonara chocolate sauce dough meat loaf
2. Add TensorFlow Lite
repositories { maven { url 'https://google.bintray.com/tensorflow' } } dependencies
{ // ... implementation 'org.tensorflow:tensorflow-lite:+' } build.gradle
android { aaptOptions { noCompress "tflite" } } build.gradle
3. Create ImageClassifier.java
Image Classifier
ImageClassifier.java model = loadModelFile(); tflite = new Interpreter();
ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model);
MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); FileInputStream inputStream = new
FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel();
MappedByteBuffer loadModelFile() { AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH); FileInputStream inputStream = new
FileInputStream(descriptor.getFileDescriptor()); FileChannel channel = inputStream.getChannel(); long start = descriptor.getStartOffset(); long length = descriptor.getDeclaredLength(); return channel.map(FileChannel.MapMode.READ_ONLY, start, length); }
Image Classifier [strawberry, apple, ... ] labels.txt
ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList =
loadLabelList();
labels.txt strawberry orange lemon fig pineapple banana jackfruit custard apple
pomegranate hay carbonara chocolate sauce dough meat loaf
List<String> loadLabelList() throws IOException { InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));
}
List<String> loadLabelList() throws IOException { InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));
BufferedReader reader = new BufferedReader(inputStream); }
List<String> loadLabelList() throws IOException { InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));
BufferedReader reader = new BufferedReader(inputStream); List<String> labelList = new ArrayList<>(); String line; while ((line = reader.readLine()) != null) { labelList.add(line); } }
List<String> loadLabelList() throws IOException { InputStreamReader inputStream = new InputStreamReader(getAssets().open(LABEL_PATH));
BufferedReader reader = new BufferedReader(inputStream); List<String> labelList = new ArrayList<>(); String line; while ((line = reader.readLine()) != null) { labelList.add(line); } reader.close(); return labelList; }
Image Classifier [ [0..6] , [ 0.1 ] , ...
] [strawberry, apple, ... ] probArray labels.txt
probArray = { [0.7], [0.3], [0], [0], } labelList =
{ strawberry, apple, pineapple, banana, } 0.3
ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList =
loadLabelList(); probArray = new float[1][labelList.size()];
Image Classifier [......] [ [0..6] , [ 0.1 ] ,
... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
ImageClassifier.java model = loadModelFile(); tflite = new Interpreter(model); labelList =
loadLabelList(); probArray = new float[1][labelList.size()]; imgData = ByteBuffer.allocateDirect( DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE); imgData.order(ByteOrder.nativeOrder());
4. Run the model / classify
classifier .classify(bitmap) Image Classifier [......] [ [0..6] , [ 0.1
] , ... ] [strawberry, apple, ... ] ByteBuffer probArray labels.txt
ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(imgData, bitmap);
}
void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),
bitmap.getHeight()); }
void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),
bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
void convertBitmapToByteBuffer(Bitmap bitmap) { //... bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0,bitmap.getWidth(),
bitmap.getHeight()); int pixel = 0; for (int i = 0; i < DIM_IMG_SIZE_X; ++i) { for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) { final int val = intValues[pixel++]; imgData.put((byte) ((val >> 16) & 0xFF)); imgData.put((byte) ((val >> 8) & 0xFF)); imgData.put((byte) (val & 0xFF)); } } }
ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(imgData, bitmap); tflite.run(imgData,
probArray); }
ImageClassifier.java String classify(Bitmap bitmap) { convertBitmapToByteBuffer(imgData, bitmap); tflite.run(imgData,
probArray); String textToShow = getTopLabels(); return textToShow; }
Strawberry - 0.87 Apple - 0.13 Tomato - 0.01
Machine Learning is a new world
Links - Tensorflow - https://www.tensorflow.org/ - Tensorflow lite - https://www.tensorflow.org/mobile/tflite/
- Codes labs - codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/ - Google’s Machine Learning Crash Course - developers.google.com/machine-learning/crash-course/ - [Dr. Joe Dispenza]
Thank you! Keep in touch! Britt Barak @brittBarak