What an ML-ful World! MLKit for Android dev.

What an ML-ful World! MLKit for Android dev.

3142db3adb711e247e371153b5777e04?s=128

Britt Barak

October 12, 2018
Tweet

Transcript

  1. What an ML-ful world Britt Barak

  2. Once upon a time @BrittBarak

  3. beta @BrittBarak

  4. ML Capability ?! @BrittBarak

  5. Who is afraid of Machine Learning? & First Steps With

    ML-Kit @BrittBarak
  6. Britt Barak Developer Experience, Nexmo Google Developer Expert Britt Barak

    @brittBarak
  7. None
  8. @BrittBarak

  9. = @BrittBarak

  10. § What’s the difference? @BrittBarak

  11. …and classify? @BrittBarak

  12. @BrittBarak

  13. This is a strawberry @BrittBarak

  14. This is a strawberry Red Seeds pattern Narrow top leaves

    @BrittBarak Pointy at the bottom Round at the top
  15. Strawberry Not Not Not Strawberry Strawberry Not Not Not @BrittBarak

  16. ~*~ images ~*~ @BrittBarak

  17. @BrittBarak Vision library

  18. Text Recognition @BrittBarak

  19. Face Detection @BrittBarak

  20. Barcode Scanning @BrittBarak

  21. Image Labelling @BrittBarak

  22. Landmark Recognition @BrittBarak

  23. Custom Models @BrittBarak

  24. Example @BrittBarak

  25. @BrittBarak

  26. @BrittBarak

  27. Detector detector .execute(image) Result: @BrittBarak “Ben & Jerry’s pistachio ice

    cream”
  28. 1. Setup Detector @BrittBarak

  29. Local or cloud? @BrittBarak

  30. @BrittBarak

  31. Local •Realtime •Offline support •Security / Privacy •Bandwith •… @BrittBarak

  32. Cloud •More accuracy & features •But more latency •Pricing @BrittBarak

  33. 1. Setup Detector @BrittBarak

  34. Text Detector textDetector = FirebaseVision.getInstance() @BrittBarak

  35. Text Detector textDetector = FirebaseVision.getInstance() .onDeviceTextRecognizer @BrittBarak

  36. Text Detector textDetector = FirebaseVision.getInstance() .cloudTextRecognizer @BrittBarak

  37. 2. Process input @BrittBarak

  38. FirebaseVisionImage •Bitmap •image Uri •Media Image •byteArray •byteBuffer @BrittBarak

  39. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Text Detector

  40. 3. Execute the model @BrittBarak

  41. Text Detector textDetector.processImage(image) @BrittBarak

  42. Text Detector textDetector.processImage(image) .addOnSuccessListener { } @BrittBarak

  43. Text Detector textDetector.processImage(image) .addOnSuccessListener { firebaseVisionTexts -> processOutput(fbVisionTexts) } @BrittBarak

  44. 4. Process output @BrittBarak

  45. firebaseVisionTexts.text @BrittBarak

  46. someTextView.text = firebaseVisionTexts.text @BrittBarak UI

  47. Result @BrittBarak

  48. Result @BrittBarak

  49. (another) Example : Labelling @BrittBarak

  50. Detector detector .execute(image) Result: @BrittBarak ice cream pint

  51. Vegetables Deserts Vegetables

  52. 1. Setup Detector @BrittBarak

  53. Image Classifier imageDetector = FirebaseVision.getInstance() @BrittBarak

  54. Image Classifier imageDetector = FirebaseVision.getInstance() .visionLabelDetector @BrittBarak

  55. Image Classifier imageDetector = FirebaseVision.getInstance .visionCloudLabelDetector @BrittBarak

  56. 2. Process input @BrittBarak

  57. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Image Classifier

  58. 3. Execute the model @BrittBarak

  59. Image Classifier imageDetector.detectInImage(image) @BrittBarak

  60. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ } @BrittBarak

  61. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ fBLabels -> processOutput(fBLabels) } @BrittBarak

  62. 4. Process output @BrittBarak

  63. fbLabel.label fbLabel.confidence fbLabel.entityId @BrittBarak

  64. UI for (fbLabel in labels) { s = "${fbLabel.label} :

    ${fbLabel.confidence}" } @BrittBarak
  65. Result

  66. Result

  67. It is an ML-ful world Enjoy!

  68. Thank you! Keep in touch!