[Britt Barak] It’s an ML-full world! MLKit for Android Devs

[Britt Barak] It’s an ML-full world! MLKit for Android Devs

Presentation from GDG DevFest Ukraine 2018 - the biggest community-driven Google tech conference in the CEE.

Learn more at: https://devfest.gdg.org.ua

__

There was a time when everyone was talking about ML, DL, AI… but as app developers we didn’t really know what all of these mean for our app! During last Google I/O ML Kit was announced, made it a lot easier to use ML capabilities as a part of our product. This session will explore ML Kit features, how and why to use them, and what to do when we have a custom use case, and TensorFlow Lite comes into the picture?

3a6de6bc902de7f75c0e753b3202ed52?s=128

Google Developers Group Lviv

October 12, 2018
Tweet

Transcript

  1. What an ML-ful world Britt Barak

  2. Once upon a time @BrittBarak

  3. beta @BrittBarak

  4. ML Capability ?! @BrittBarak

  5. Who is afraid of Machine Learning? & First Steps With

    ML-Kit @BrittBarak
  6. Britt Barak DevX - Nexmo Google Developer Expert Britt Barak

    @brittBarak
  7. None
  8. @BrittBarak

  9. = @BrittBarak

  10. § What’s the difference? @BrittBarak

  11. …and classify? @BrittBarak

  12. @BrittBarak

  13. This is a strawberry @BrittBarak

  14. This is a strawberry Red Seeds pattern Narrow top leaves

    @BrittBarak Pointy at the bottom Round at the top
  15. Strawberry Not Not Not Strawberry Strawberry Not Not Not @BrittBarak

  16. ~*~ images ~*~ @BrittBarak

  17. @BrittBarak Vision library

  18. Text Recognition @BrittBarak

  19. Face Detection @BrittBarak

  20. Barcode Scanning @BrittBarak

  21. Landmark Recognition @BrittBarak

  22. Image Labelling @BrittBarak

  23. Custom Models @BrittBarak

  24. Example @BrittBarak

  25. @BrittBarak

  26. @BrittBarak

  27. Detector detector .execute(image) Result: “Ben & Jerry’s pistachio” @BrittBarak

  28. 1. Setup Detector @BrittBarak

  29. Local or cloud? @BrittBarak

  30. @BrittBarak

  31. Strawberry @BrittBarak

  32. Why a local model? •Security •Privacy •Latency •Bandwith •Performance •Offline

    support •… @BrittBarak
  33. 1. Setup Detector @BrittBarak

  34. Text Detector textDetector = FirebaseVision.getInstance() .getOnDeviceTextRecognizer(); @BrittBarak

  35. Text Detector textDetector = FirebaseVision.getInstance() .getCloudTextRecognizer(); @BrittBarak

  36. 2. Process input @BrittBarak

  37. FirebaseVisionImage •Bitmap •byteArray •byteBuffer •image Uri •Media Image @BrittBarak

  38. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Text Detector

  39. 3. Execute the model @BrittBarak

  40. Text Detector detector.processImage(image) .addOnSuccessListener( new OnSuccessListener<List<FirebaseVisionLabel>>(){ void onSuccess(List<FirebaseVisionLabel> labels){ processResult(labels,

    callback); } }) } @BrittBarak
  41. Text Detector detector.processImage(image) .addOnSuccessListener{ } @BrittBarak

  42. Text Detector detector.processImage(image) .addOnSuccessListener{ fbVisiontexts -> processOutput(fbVisiontexts) } @BrittBarak

  43. 4. Process output @BrittBarak

  44. UI textView.text = fbVisionTexts.text @BrittBarak

  45. Result @BrittBarak

  46. Result @BrittBarak

  47. (another) Example : Labelling @BrittBarak

  48. Detector detector .execute(image) Result: ice cream pint @BrittBarak

  49. 1. Setup Detector @BrittBarak

  50. Image Classifier imageDetector = FirebaseVision.getInstance() .getVisionLabelDetector() @BrittBarak

  51. Image Classifier imageDetector = FirebaseVision.getInstance() .getVisionCloudLabelDetector() @BrittBarak

  52. 2. Process input @BrittBarak

  53. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Image Classifier

  54. 3. Execute the model @BrittBarak

  55. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener( new OnSuccessListener<List<FirebaseVisionLabel>>(){ void onSuccess(List<FirebaseVisionLabel> labels){ processResult(labels,

    callback); } }) } @BrittBarak
  56. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ } @BrittBarak

  57. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ fBLabels -> processOutput(fBLabels) } @BrittBarak

  58. 4. Process output @BrittBarak

  59. UI for (fbLabel in labels) { s = "${fbLabel.label} :

    ${fbLabel.confidence}" } @BrittBarak
  60. Result

  61. Result

  62. It is an ML-ful world Enjoy!

  63. Thank you! Keep in touch!