Slide 1
Slide 1 text
TensorFlow for Mobile Machine Learning
Supriya Srivatsa, Software Engineer, Xome
Slide 2
Slide 2 text
Overview
• AI and Mobile – the Convergence
• Inference – Today and Tomorrow
• TensorFlow Primer
• TensorFlow in your Pocket
– TensorFlow Mobile
– TensorFlow Lite
• PokéDemo
• Applications and Case Studies
• Q & A
Slide 3
Slide 3 text
AI AND MOBILE – THE CONVERGENCE
Slide 4
Slide 4 text
INFERENCE - TODAY AND TOMORROW
Slide 5
Slide 5 text
The “Transfer to Infer” Approach
Slide 6
Slide 6 text
Why On Device Prediction
• Data Privacy
• Poor Internet Connection
• Questionable User Experience
Slide 7
Slide 7 text
Why On Device Prediction
Case Study: Portrait Mode
Slide 8
Slide 8 text
TENSORFLOW PRIMER
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
TensorFlow – Deferred Execution Model
(Building the Computational Graph)
import tensorflow as tf
num1 = tf.constant(5)
num2 = tf.constant(10)
sum = num1 + num2
print(sum)
#O/P: Tensor("add:0", shape=(), dtype=int32)
Slide 11
Slide 11 text
TensorFlow – Deferred Execution Model
(Running the Computational Graph)
import tensorflow as tf
num1 = tf.constant(5)
num2 = tf.constant(10)
sum = num1 + num2
with tf.Session() as sess:
print(sess.run(sum))
#O/P: 15
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
TENSORFLOW
IN YOUR POCKET
Slide 15
Slide 15 text
Pick Your Weapon
• Choose a pre-trained TF Model
– Inception V3 Model
– MNIST
– Smart Reply
– Deep Speech
• Build a TF Model
Slide 16
Slide 16 text
Sharpen your Sword
• Retrain Model as required.
Slide 17
Slide 17 text
Neural Network and Transfer Learning
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
TENSORFLOW MOBILE
VS
TENSORFLOW LITE
Slide 20
Slide 20 text
TensorFlow Lite
• Smaller binary size, better
performance.
• Ability to leverage hardware
acceleration.
• Only supports a limited set
of operators.
Slide 21
Slide 21 text
TensorFlow Mobile and TensorFlow Lite
Slide 22
Slide 22 text
TensorFlow Mobile and TensorFlow Lite
Slide 23
Slide 23 text
TensorFlow Mobile and TensorFlow Lite
Slide 24
Slide 24 text
Optimization
• optimize_for_inference
• Quantization
Slide 25
Slide 25 text
Quantization
• Round it up
• Transform: round_weights
• Compression rates: ~8% => ~70%
• Shrink down node names
• Transform: obfuscate_names
• Eight bit calculations
Slide 26
Slide 26 text
Quantization – Eight Bit Calculations
Slide 27
Slide 27 text
Optimization – Before and After
Slide 28
Slide 28 text
TensorFlow Mobile and TensorFlow Lite
Slide 29
Slide 29 text
TensorFlow Mobile and TensorFlow Lite
Slide 30
Slide 30 text
TensorFlow Lite
• TOCO – TensorFlow Lite
Optimizing Converter
– Pruning unused nodes.
– Performance Improvements.
– Convert to tflite format.
(Generate FlatBuffer file.)
Slide 31
Slide 31 text
ü Frozen
ü Optimized,
Quantized
ü .tflite / FlatBuffer
Slide 32
Slide 32 text
How does it work?
Slide 33
Slide 33 text
Packaging App and Model
Slide 34
Slide 34 text
CODE AWAY
J
Slide 35
Slide 35 text
Code Away – Gradle Files
Slide 36
Slide 36 text
Code Away :)
Tflite = new Interpreter()
tflite.run(giveInput, outputObject)
• Create Interpreter
• Run model with input, fetch output.
Slide 37
Slide 37 text
POKÉDEMO!
Slide 38
Slide 38 text
PokéDemo
Slide 39
Slide 39 text
APPLICATIONS AND
CASE STUDIES
Slide 40
Slide 40 text
Coca Cola
Slide 41
Slide 41 text
Google Assistant
Slide 42
Slide 42 text
Smart Reply
Slide 43
Slide 43 text
Q & A
Slide 44
Slide 44 text
Thank you