Attila Blenesi - TensorFlow For AndroidDevs

Transcript

1. For Android devs
2. Attila Blénesi Senior Android Engineer @ Halcyon Mobile @ablenessy
3. Is machine learning only for experts ? @ablenessy | @droidconBerlin | #DCBERLIN18
5. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML @ablenessy | @droidconBerlin | #DCBERLIN18
6. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML @ablenessy | @droidconBerlin | #DCBERLIN18
7. Firebase ML Kit @ablenessy | @droidconBerlin | #DCBERLIN18 Source: firebase.google.com/products/ml-kit
8. Text recognition
9. Text recognition
10. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
11. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
12. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
13. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
14. Text recognition
15. Text recognition
16. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
17. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionTextDetector() detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
18. val image = FirebaseVisionImage.fromBitmap(selectedImage) val detector = FirebaseVision.getInstance() .getVisionCloudDocumentTextDetector(options) detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
19. val image = FirebaseVisionImage.fromBitmap(selectedImage) val options = FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15) .build() val detector = FirebaseVision.getInstance() .getVisionCloudDocumentTextDetector(options) detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
20. val image = FirebaseVisionImage.fromBitmap(selectedImage) val options = FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15) .build() val detector = FirebaseVision.getInstance() .getVisionCloudDocumentTextDetector(options) detector.detectInImage(image) .addOnSuccessListener { texts -> processTextRecognitionResult(texts) }a .addOnFailureListener(...) @ablenessy | @droidconBerlin | #DCBERLIN18
21. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML
22. Machine Learning APIs
23. Source: cloud.google.com/vision/
24. google.com/about/stories/machine-learning-qa/
25. google.com/about/stories/machine-learning-qa/
26. Source: cloud.google.com/vision/
27. Source: cloud.google.com/vision/
28. Do I even have to dig deeper ? @ablenessy | @droidconBerlin | #DCBERLIN18
29. generic model = generic solution Do I even have to dig deeper ? @ablenessy | @droidconBerlin | #DCBERLIN18
30. Application Developer ML Practitioner Data Scientist Firebase ML Kit Machine Learning APIs TensorFlow Cloud ML
31. TensorFlow Cloud ML #edges2cats
32. TensorFlow Cloud ML
33. TensorFlow Cloud ML #edges2cats
34. workflow Model > Training > Optimisation > Deployment @ablenessy | @droidconBerlin | #DCBERLIN18
35. How does pix2pix work? Model > Training > Optimisation > Deployment
36. How does pix2pix work? Model > Training > Optimisation > Deployment
37. How does pix2pix work?
38. How does pix2pix work?
39. How does pix2pix work? INPUT OUTPUT generator
40. How does pix2pix work? INPUT OUTPUT generator
41. How does pix2pix work? INPUT OUTPUT generator
42. How does pix2pix work? generator INPUT OUTPUT
43. How does pix2pix work? generator INPUT OUTPUT That doesn’t look right
44. How does pix2pix work? discriminator in unknown
45. How does pix2pix work? discriminator in unknown
46. How does pix2pix work? discriminator in unknown
47. How does pix2pix work? discriminator in unknown
48. How does pix2pix work? discriminator in unknown
49. How does pix2pix work?
50. Training the discriminator
51. Training the discriminator
52. Training the discriminator
53. Training the generator
54. Training the generator
55. Training the generator
56. Training the generator
57. Training the generator
58. Model > Training > Optimisation > Deployment
59. Local ~ 36 hours ~ 12 hours Cloud vs @ablenessy | @droidconBerlin | #DCBERLIN18
60. Model > Training > Optimisation > Deployment Cloud ML
61. Model > Training > Optimisation > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
62. Model > Training > Optimisation > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
63. trainingInput: scaleTier: CUSTOM masterType: standard_gpu # 1 GPU pythonVersion: "3.5" runtimeVersion: "1.8" Model > Training > Optimisation > Deployment Cloud ML
64. trainingInput: scaleTier: CUSTOM masterType: complex_model_m_gpu # 4 GPUs pythonVersion: "3.5" runtimeVersion: "1.8" Model > Training > Optimisation > Deployment Cloud ML
65. Model > Training > Optimisation > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
66. Model > Training > Optimisation > Deployment gcloud ml-engine jobs submit training $JOB_NAME \ --job-dir $JOB_DIR \ --module-name trainer.pix2pix \ --package-path ./trainer \ --region $REGION \ --config=trainer/cloudml-gpu.yaml \ -- \ --mode train \ --input-dir gs://$BUCKET_NAME/train Cloud ML
67. Model > Training > Optimisation > Deployment
68. Model > Training > Optimisation > Deployment TensorBoard
69. Model > Training > Optimisation > Deployment TensorBoard
70. Model > Training > Optimisation > Deployment TensorBoard tensorboard —logdir /path/to/model http://localhost:6006/
71. Model > Training > Optimisation > Deployment TensorBoard
72. Model > Training > Optimisation > Deployment
73. Model > Training > Optimisation > Deployment ~680 mb ~2600 operations
74. Model > Training > Optimisation > Deployment ~680 mb ~2600 operations
75. Model > Training > Optimisation > Deployment ~680 mb ~2600 operations
76. Model > Training > Optimisation > Deployment ~680 mb ~2600 operations
77. Model > Training > Optimisation > Deployment ~200 mb ~400 operations
78. Model > Training > Optimisation > Deployment checkpoint reduced_model.data-00000-of-00001 reduced_model.meta reduced_model.index frozen_model.pb
79. Model > Training > Optimisation > Deployment # We start a session using a temporary fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph in the current default Graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the usefull nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
80. Model > Training > Optimisation > Deployment # We start a session using a temporary fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph in the current default Graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the usefull nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
81. Model > Training > Optimisation > Deployment # We start a session using a temporary fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph in the current default Graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the usefull nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
82. Model > Training > Optimisation > Deployment # We start a session using a temporary fresh Graph with tf.Session(graph=tf.Graph()) as sess: # We import the meta graph in the current default Graph saver = tf.train.import_meta_graph(input_checkpoint + ‘.meta’,…) saver.restore(sess, input_checkpoint) # We restore the weights output_graph_def = tf.graph_util.convert_variables_to_constants( sess, # The session is used to retrieve the weights tf.get_default_graph().as_graph_def(), # retrieve the nodes output_node_names # select the useful nodes ) with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString())
83. Model > Training > Optimisation > Deployment checkpoint reduced_model.data-00000-of-00001 reduced_model.meta reduced_model.index frozen_model.pb
84. Model > Training > Optimisation > Deployment checkpoint reduced_model.data-00000-of-00001 reduced_model.meta reduced_model.index frozen_model.pb
85. Model > Training > Optimisation > Deployment // TensorFlow Mobile implementation ‘org.tensorflow:tensorflow-android:1.7.0’
86. Model > Training > Optimisation > Deployment
87. Model > Training > Optimisation > Deployment output_tensor Generated image jalammar.github.io/Supercharging-android-apps-using-tensorflow/ Generator
88. Model > Training > Optimisation > Deployment val inferenceInterface = TensorFlowInferenceInterface( assetManager, "file:///android_asset/frozen_model.pb" ) inputValues = generateInput(inputDrawing) inferenceInterface.feed("input_node", inputValues, inputDimensions) inferenceInterface.run(arrayOf("output_node"), false) inferenceInterface.fetch("output_node", outputValues) val result = generateBitmapFromOutput(outputValues)
89. Model > Training > Optimisation > Deployment val inferenceInterface = TensorFlowInferenceInterface( assetManager, "file:///android_asset/frozen_model.pb" ) inputValues = generateInput(inputDrawing) inferenceInterface.feed("input_node", inputValues, inputDimensions) inferenceInterface.run(arrayOf("output_node"), false) inferenceInterface.fetch("output_node", outputValues) val result = generateBitmapFromOutput(outputValues)
90. Model > Training > Optimisation > Deployment val inferenceInterface = TensorFlowInferenceInterface( assetManager, "file:///android_asset/frozen_model.pb" ) inputValues = generateInput(inputDrawing) inferenceInterface.feed("input_node", inputValues, inputDimensions) inferenceInterface.run(arrayOf("output_node"), false) inferenceInterface.fetch("output_node", outputValues) val result = generateBitmapFromOutput(outputValues)
91. Model > Training > Optimisation > Deployment val inferenceInterface = TensorFlowInferenceInterface( assetManager, "file:///android_asset/frozen_model.pb" ) inputValues = generateInput(inputDrawing) inferenceInterface.feed("input_node", inputValues, inputDimensions) inferenceInterface.run(arrayOf("output_node"), false) inferenceInterface.fetch("output_node", outputValues) val result = generateBitmapFromOutput(outputValues)
92. Demo @ablenessy | @droidconBerlin | #DCBERLIN18
93. Demo @ablenessy | @droidconBerlin | #DCBERLIN18
94. @ablenessy | @droidconBerlin | #DCBERLIN18 Machine learning is NOT just for experts
95. @ablenessy | @droidconBerlin | #DCBERLIN18 Machine learning is NOT just for experts
96. @ablenessy | @droidconBerlin | #DCBERLIN18