GIDS18_SupriyaSrivatsa.pdf

1. TensorFlow for Mobile Machine Learning Supriya Srivatsa, Software Engineer, Xome
2. Overview • AI and Mobile – the Convergence • Inference – Today and Tomorrow • TensorFlow Primer • TensorFlow in your Pocket – TensorFlow Mobile – TensorFlow Lite • PokéDemo • Applications and Case Studies • Q & A
3. AI AND MOBILE – THE CONVERGENCE
4. INFERENCE - TODAY AND TOMORROW
5. The “Transfer to Infer” Approach
6. Why On Device Prediction • Data Privacy • Poor Internet Connection • Questionable User Experience
7. Why On Device Prediction Case Study: Portrait Mode
8. TENSORFLOW PRIMER
10. TensorFlow – Deferred Execution Model (Building the Computational Graph) import tensorflow as tf num1 = tf.constant(5) num2 = tf.constant(10) sum = num1 + num2 print(sum) #O/P: Tensor("add:0", shape=(), dtype=int32)
11. TensorFlow – Deferred Execution Model (Running the Computational Graph) import tensorflow as tf num1 = tf.constant(5) num2 = tf.constant(10) sum = num1 + num2 with tf.Session() as sess: print(sess.run(sum)) #O/P: 15
14. TENSORFLOW IN YOUR POCKET
15. Pick Your Weapon • Choose a pre-trained TF Model – Inception V3 Model – MNIST – Smart Reply – Deep Speech • Build a TF Model
16. Sharpen your Sword • Retrain Model as required.
17. Neural Network and Transfer Learning
19. TENSORFLOW MOBILE VS TENSORFLOW LITE
20. TensorFlow Lite • Smaller binary size, better performance. • Ability to leverage hardware acceleration. • Only supports a limited set of operators.
21. TensorFlow Mobile and TensorFlow Lite
22. TensorFlow Mobile and TensorFlow Lite
23. TensorFlow Mobile and TensorFlow Lite
24. Optimization • optimize_for_inference • Quantization
25. Quantization • Round it up • Transform: round_weights • Compression rates: ~8% => ~70% • Shrink down node names • Transform: obfuscate_names • Eight bit calculations
26. Quantization – Eight Bit Calculations
27. Optimization – Before and After
28. TensorFlow Mobile and TensorFlow Lite
29. TensorFlow Mobile and TensorFlow Lite
30. TensorFlow Lite • TOCO – TensorFlow Lite Optimizing Converter – Pruning unused nodes. – Performance Improvements. – Convert to tflite format. (Generate FlatBuffer file.)
31. ü Frozen ü Optimized, Quantized ü .tflite / FlatBuffer
32. How does it work?
33. Packaging App and Model
34. CODE AWAY J
35. Code Away – Gradle Files
36. Code Away :) Tflite = new Interpreter(<loadmodelfile>) tflite.run(giveInput, outputObject) • Create Interpreter • Run model with input, fetch output.
37. POKÉDEMO!
38. PokéDemo
39. APPLICATIONS AND CASE STUDIES
40. Coca Cola
41. Google Assistant
42. Smart Reply
43. Q & A
44. Thank you