Machine Learning for Android Developers with TensorFlow - Droidcon London 2018

During this session, Attila will introduce the TensorFlow framework from an Android developer’s point of view. When it comes to blending AI into mobile apps, there are a couple of tools at developers disposal. You will explore some of these available options, then linger on to the steps and challenges of integrating a TenorFlow model. In the end, you’ll be left with a basic understanding of the process and work with TensorFlow on Android. Hopefully, these new learnings will spark your curiosity and encourage you to experiment and implement your crazy, creative AI ideas.

Transcript

1. Machine Learning for Android Developers With
2. Attila Blénesi Senior Android Engineer @ Halcyon Mobile @ablenessy
3. Is machine learning only for experts ? @ablenessy | @droidconUK | #droidconUK
5. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML @ablenessy | @droidconUK | #droidconUK
6. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML @ablenessy | @droidconUK | #droidconUK
7. Firebase ML Kit @ablenessy | @droidconUK | #droidconUK Source: firebase.google.com/products/ml-kit
8. Text recognition
9. Text recognition
10. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconUK | #droidconUK
11. Text recognition
12. Text recognition
13. val image = FirebaseVisionImage.fromBitmap(selectedImage) val options = FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15) .build() val detector = FirebaseVision.getInstance() .getVisionCloudDocumentTextDetector(options) detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconUK | #droidconUK
14. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML
15. Machine Learning APIs
16. Source: cloud.google.com/vision/
17. google.com/about/stories/machine-learning-qa/
18. google.com/about/stories/machine-learning-qa/
19. Source: cloud.google.com/vision/
20. Source: cloud.google.com/vision/
21. Do I even have to dig deeper ? @ablenessy | @droidconUK | #droidconUK
22. generic model = generic solution Do I even have to dig deeper ? @ablenessy | @droidconUK | #droidconUK
23. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML
24. TensorFlow Cloud ML #edges2cats Source: https://affinelayer.com/pixsrv/
25. TensorFlow Cloud ML #edges2cats
26. Let’s do this! @ablenessy | @droidconUK | #droidconUK
27. @ablenessy | @droidconUK | #droidconUK Let’s do this!
28. workflow Model > Training > Deployment @ablenessy | @droidconUK | #droidconUK
29. How does pix2pix work? Model > Training > Deployment
30. How does pix2pix work? Model > Training > Deployment
31. How does pix2pix work? Source: https://affinelayer.com/pix2pix/
32. How does pix2pix work? Source: https://affinelayer.com/pix2pix/
33. How does pix2pix work? generator INPUT OUTPUT Source: https://affinelayer.com/pix2pix/
34. Source: https://affinelayer.com/pix2pix/
35. How does pix2pix work? Source: https://affinelayer.com/pix2pix/
36. Model > Training > Deployment
37. Training the discriminator Source: https://affinelayer.com/pix2pix/
38. Training the generator Source: https://affinelayer.com/pix2pix/
39. Local ~ 36 hours ~ 12 hours Cloud vs Model > Training > Deployment
40. Model > Training > Deployment Cloud ML
41. Model > Training > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
42. Model > Training > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
43. trainingInput: scaleTier: CUSTOM masterType: standard_gpu # 1 GPU pythonVersion: "3.5" runtimeVersion: "1.8" Model > Training > Deployment Cloud ML
44. trainingInput: scaleTier: CUSTOM masterType: complex_model_m_gpu # 4 GPUs pythonVersion: "3.5" runtimeVersion: "1.8" Model > Training > Deployment Cloud ML
45. Model > Training > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
46. Model > Training > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
47. Model > Training > Deployment
48. Model > Training > Deployment TensorBoard
49. Model > Training > Deployment TensorBoard
50. Model > Training > Deployment TensorBoard tensorboard —logdir /path/to/model http://localhost:6006/
51. Model > Training > Deployment TensorBoard
52. Model > Training > Deployment
53. Model > Training > Deployment ~680 mb ~2600 operations
54. Model > Training > Deployment ~200 mb ~400 operations
55. Model > Training > Deployment checkpoint reduced_model.data-00000-of-00001 reduced_model.meta reduced_model.index frozen_model.pb
56. Model > Training > Deployment # We start a session using a fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the useful nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
57. Model > Training > Deployment # We start a session using a fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the useful nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
58. Model > Training > Deployment # We start a session using a fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the useful nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
59. Model > Training > Deployment # We start a session using a fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the useful nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
60. Model > Training > Deployment checkpoint reduced_model.data-00000-of-00001 reduced_model.meta reduced_model.index frozen_model.pb
61. Model > Training > Deployment // TensorFlow Mobile implementation ‘org.tensorflow:tensorflow-android:1.11.0’
62. Model > Training > Deployment
63. Model > Training > Deployment output_tensor Generated image jalammar.github.io/Supercharging-android-apps-using-tensorflow/ Generator
64. Model > Training > Deployment val inferenceInterface = TensorFlowInferenceInterface( assetManager, "file:///android_asset/frozen_model.pb" ) inputValues = generateInput(inputDrawing) inferenceInterface.feed("input_node", inputValues, inputDimensions) inferenceInterface.run(arrayOf("output_node"), false) inferenceInterface.fetch("output_node", outputValues) val result = generateBitmapFromOutput(outputValues)
65. Model > Training > Deployment
67. Model > Training > Deployment
68. Model > Training > Deployment
69. Demo @ablenessy | @droidconUK | #droidconUK Demo
70. Machine learning is NOT just for experts
71. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML Machine learning is NOT just for experts
72. Questions @ablenessy | @droidconUK | #droidconUK