[Britt Barak] It’s an ML-full world! MLKit for Android Devs

Presentation from GDG DevFest Ukraine 2018 - the biggest community-driven Google tech conference in the CEE.

Learn more at: https://devfest.gdg.org.ua

__

There was a time when everyone was talking about ML, DL, AI… but as app developers we didn’t really know what all of these mean for our app! During last Google I/O ML Kit was announced, made it a lot easier to use ML capabilities as a part of our product. This session will explore ML Kit features, how and why to use them, and what to do when we have a custom use case, and TensorFlow Lite comes into the picture?

Transcript

1. What an ML-ful world Britt Barak
2. Once upon a time @BrittBarak
3. beta @BrittBarak
4. ML Capability ?! @BrittBarak
5. Who is afraid of Machine Learning? & First Steps With ML-Kit @BrittBarak
6. Britt Barak DevX - Nexmo Google Developer Expert Britt Barak @brittBarak
8. @BrittBarak
9. = @BrittBarak
10. § What’s the difference? @BrittBarak
11. …and classify? @BrittBarak
12. @BrittBarak
13. This is a strawberry @BrittBarak
14. This is a strawberry Red Seeds pattern Narrow top leaves @BrittBarak Pointy at the bottom Round at the top
15. Strawberry Not Not Not Strawberry Strawberry Not Not Not @BrittBarak
16. ~*~ images ~*~ @BrittBarak
17. @BrittBarak Vision library
18. Text Recognition @BrittBarak
19. Face Detection @BrittBarak
20. Barcode Scanning @BrittBarak
21. Landmark Recognition @BrittBarak
22. Image Labelling @BrittBarak
23. Custom Models @BrittBarak
24. Example @BrittBarak
25. @BrittBarak
26. @BrittBarak
27. Detector detector .execute(image) Result: “Ben & Jerry’s pistachio” @BrittBarak
28. 1. Setup Detector @BrittBarak
29. Local or cloud? @BrittBarak
30. @BrittBarak
31. Strawberry @BrittBarak
32. Why a local model? •Security •Privacy •Latency •Bandwith •Performance •Offline support •… @BrittBarak
33. 1. Setup Detector @BrittBarak
34. Text Detector textDetector = FirebaseVision.getInstance() .getOnDeviceTextRecognizer(); @BrittBarak
35. Text Detector textDetector = FirebaseVision.getInstance() .getCloudTextRecognizer(); @BrittBarak
36. 2. Process input @BrittBarak
37. FirebaseVisionImage •Bitmap •byteArray •byteBuffer •image Uri •Media Image @BrittBarak
38. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Text Detector
39. 3. Execute the model @BrittBarak
40. Text Detector detector.processImage(image) .addOnSuccessListener( new OnSuccessListener<List<FirebaseVisionLabel>>(){ void onSuccess(List<FirebaseVisionLabel> labels){ processResult(labels, callback); } }) } @BrittBarak
41. Text Detector detector.processImage(image) .addOnSuccessListener{ } @BrittBarak
42. Text Detector detector.processImage(image) .addOnSuccessListener{ fbVisiontexts -> processOutput(fbVisiontexts) } @BrittBarak
43. 4. Process output @BrittBarak
44. UI textView.text = fbVisionTexts.text @BrittBarak
45. Result @BrittBarak
46. Result @BrittBarak
47. (another) Example : Labelling @BrittBarak
48. Detector detector .execute(image) Result: ice cream pint @BrittBarak
49. 1. Setup Detector @BrittBarak
50. Image Classifier imageDetector = FirebaseVision.getInstance() .getVisionLabelDetector() @BrittBarak
51. Image Classifier imageDetector = FirebaseVision.getInstance() .getVisionCloudLabelDetector() @BrittBarak
52. 2. Process input @BrittBarak
53. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Image Classifier
54. 3. Execute the model @BrittBarak
55. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener( new OnSuccessListener<List<FirebaseVisionLabel>>(){ void onSuccess(List<FirebaseVisionLabel> labels){ processResult(labels, callback); } }) } @BrittBarak
56. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ } @BrittBarak
57. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ fBLabels -> processOutput(fBLabels) } @BrittBarak
58. 4. Process output @BrittBarak
59. UI for (fbLabel in labels) { s = "${fbLabel.label} : ${fbLabel.confidence}" } @BrittBarak
60. Result
61. Result
62. It is an ML-ful world Enjoy!
63. Thank you! Keep in touch!