MLKit DevFest18 ATL

MLKit: AI Commoditized

Incorporating computer vision or artificial intelligence in mobile applications used to involve a number of challenges: specialized expertise, limited computing power, limited budgets. Nowadays, thanks to widespread availability of open source machine learning libraries and models, immense power is within reach of many. MLKit aims to further break down barriers for Android and iOS developers and offers several ready-to-deploy models out of the box.

References
https://www.bignerdranch.com/blog/google-io-2018-ai-commoditized/
https://firebase.google.com/products/ml-kit/
https://medium.com/google-developer-experts/exploring-firebase-mlkit-on-android-introducing-mlkit-part-one-98fcfedbeee0
https://www.youtube.com/watch?v=FwFduRA_L6Q – Convolutional Neural Network Demo 1993
https://en.wikipedia.org/wiki/Artificial_neural_network
https://en.wikipedia.org/wiki/Convolutional_neural_network
https://github.com/zalandoresearch/fashion-mnist
https://code.oursky.com/tensorflow-svm-image-classifications-engine/
http://www.visualcapitalist.com/debunking-8-myths-ai-workplace/
https://research.googleblog.com/2018/03/semantic-image-segmentation-with.html – pixel-level segmentation
https://research.googleblog.com/2017/06/supercharge-your-computer-vision-models.html – bounding boxes
https://www.youtube.com/watch?v=xVJwwWQlQ1o – Neural Style Transfer
https://research.googleblog.com/2016/10/supercharging-style-transfer.html
http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Education
https://www.deeplearning.ai
https://www.deeplearningbook.org
https://ai.google/education
https://developers.google.com/machine-learning/crash-course/
http://cs231n.stanford.edu CNNs for Visual Recognition

Transcript

1. MLKit AI Commoditized Bolot Kerimbaev
3. Supervised Unsupervised Classification Regression Clustering Dimensionality reduction Reinforcement Regularization Bayesian Neural networks Deep learning Support vector machines Decision tree k-Nearest neighbors k-Means Hidden Markov models
4. Machine Learning • Supervised • Classification • Regression • Unsupervised • Clustering • Dimensionality reduction • Reinforcement • Q-Learning
5. Why ML?
6. Why ML? • Task match • State of the art techniques • New products • Automation • Better performance • Move into new business • Competitive advantage • Cost reduction • Consumer demand • Je ne sais quoi
7. Applications • Speech recognition • Face recognition • Machine translation • Image classification • News article generation • Employee access control • Protecting animals • Predicting ER wait times • Identifying heart failure • Predicting strokes and seizures • Predicting hospital readmissions • Recommendation systems
8. Software • Caffe - Berkeley • CNTK - Microsoft • DL4J - Skymind • DSSTNE - Amazon • TensorFlow - Google • Theano - Université de Montréal • Torch - Google DeepMind, Facebook AI, etc. • PyTorch • Keras • Numpy, SciKit • Pandas • Matlab • Octave • R
9. TensorFlow
10. MLKit features • Face detection • Image labeling • Landmark recognition • Barcode scanning • Text recognition
11. Why MLKit? • Simple API • Pre-packaged models • Expandability • Cloud-backed • Model updates
12. MLKit
13. MLKit
14. Firebase
15. Dependencies // ML Kit dependencies implementation 'com.google.firebase:firebase-core:16.0.3' implementation 'com.google.firebase:firebase-ml-common:16.1.2' implementation 'com.google.firebase:firebase-ml-vision:17.0.0' implementation 'com.google.firebase:firebase-ml-vision-image-label-model:15.0.0' implementation 'com.google.firebase:firebase-ml-model-interpreter:16.2.0'
16. MNIST
17. Artificial Neural Networks
18. Fashion MNIST
19. Image labeling https://xkcd.com/1425/
20. Image Labeling FirebaseVision.getInstance().visionLabelDetector .detectInImage(image) .addOnSuccessListener { labels -> val output = labels.map { "$it.label: $it.confidence" } Log.d(TAG, "Found labels:\n$output") }
21. Convolutional Neural Networks
22. Case Study: Puller Detection
24. Inception
25. Inception
26. Classification, Detection, Segmentation
27. Neural Style Transfer
28. Face detection
29. Face detection val options = FirebaseVisionFaceDetectorOptions.Builder() .setClassificationType(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS) .setLandmarkType(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS) .setTrackingEnabled(true) .build() detector = FirebaseVision.getInstance().getVisionFaceDetector(options) detector.detectInImage(image) .addOnSuccessListener { results -> for (i in faces.indices) { val face = faces[i] val faceGraphic = FaceGraphic(graphicOverlay) graphicOverlay.add(faceGraphic) faceGraphic.updateFace(face, metadata.cameraFacing) } } .addOnFailureListener { e -> }
30. Recurrent Neural Networks
31. Case Study: Dropbox OCR
32. Text recognition
33. Text recognition detector = FirebaseVision.getInstance().onDeviceTextRecognizer detector.processImage(image) .addOnSuccessListener { results -> val blocks = results.textBlocks for (i in blocks.indices) { val lines = blocks[i].lines for (j in lines.indices) { val elements = lines[j].elements for (k in elements.indices) { val textGraphic = TextGraphic(graphicOverlay, elements[k]) graphicOverlay.add(textGraphic) } } } .addOnFailureListener { e -> }
34. Text recognition detector = FirebaseVision.getInstance().onDeviceTextRecognizer detector.processImage(image) .addOnSuccessListener { results -> val blocks = results.textBlocks for (i in blocks.indices) { val lines = blocks[i].lines for (j in lines.indices) { val elements = lines[j].elements for (k in elements.indices) { val textGraphic = TextGraphic(graphicOverlay, elements[k]) graphicOverlay.add(textGraphic) } } } .addOnFailureListener { e -> }
35. Text recognition detector = FirebaseVision.getInstance().cloudTextRecognizer detector.processImage(image) .addOnSuccessListener { results -> val blocks = results.textBlocks for (i in blocks.indices) { val lines = blocks[i].lines for (j in lines.indices) { val elements = lines[j].elements for (k in elements.indices) { val textGraphic = TextGraphic(graphicOverlay, elements[k]) graphicOverlay.add(textGraphic) } } } .addOnFailureListener { e -> }
36. Text recognition detector = FirebaseVision.getInstance().cloudTextRecognizer detector.processImage(image) .addOnSuccessListener { results -> val blocks = results.textBlocks for (i in blocks.indices) { val lines = blocks[i].lines for (j in lines.indices) { val elements = lines[j].elements for (k in elements.indices) { val textGraphic = TextGraphic(graphicOverlay, elements[k]) graphicOverlay.add(textGraphic) } } } .addOnFailureListener { e -> }
37. Case study: receipt scanning • Image labeling • Rectangle detection • Text recognition
38. Case Study: Google Street View
39. RNNs
40. RNN: Scrubs Script
41. Image captioning
42. Capsule Networks
43. Generative Adversarial Networks
44. Education • https://www.deeplearning.ai • https://www.deeplearningbook.org • https://ai.google/education • https://developers.google.com/machine-learning/crash-course/ • http://cs231n.stanford.edu CNNs for Visual Recognition
45. Questions? twitter.com/bolot caffeine.tv