Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What an ML-ful World! MLKit for Android dev.

What an ML-ful World! MLKit for Android dev.

Britt Barak

October 12, 2018
Tweet

More Decks by Britt Barak

Other Decks in Programming

Transcript

  1. What an
    ML-ful world
    Britt Barak

    View Slide

  2. Once upon a time
    @BrittBarak

    View Slide

  3. beta
    @BrittBarak

    View Slide

  4. ML Capability ?!
    @BrittBarak

    View Slide

  5. Who is afraid of
    Machine Learning?
    & First Steps With ML-Kit
    @BrittBarak

    View Slide

  6. Britt Barak
    Developer Experience, Nexmo
    Google Developer Expert
    Britt Barak @brittBarak

    View Slide

  7. View Slide

  8. @BrittBarak

    View Slide

  9. =
    @BrittBarak

    View Slide

  10. §
    What’s the difference?
    @BrittBarak

    View Slide

  11. …and classify?
    @BrittBarak

    View Slide

  12. @BrittBarak

    View Slide

  13. This is a strawberry
    @BrittBarak

    View Slide

  14. This is a strawberry
    Red Seeds
    pattern
    Narrow
    top
    leaves
    @BrittBarak
    Pointy
    at the
    bottom
    Round
    at the top

    View Slide

  15. Strawberry Not
    Not
    Not
    Strawberry
    Strawberry
    Not
    Not
    Not
    @BrittBarak

    View Slide

  16. ~*~ images ~*~
    @BrittBarak

    View Slide

  17. @BrittBarak
    Vision library

    View Slide

  18. Text Recognition
    @BrittBarak

    View Slide

  19. Face Detection
    @BrittBarak

    View Slide

  20. Barcode Scanning
    @BrittBarak

    View Slide

  21. Image Labelling
    @BrittBarak

    View Slide

  22. Landmark Recognition
    @BrittBarak

    View Slide

  23. Custom Models
    @BrittBarak

    View Slide

  24. Example
    @BrittBarak

    View Slide

  25. @BrittBarak

    View Slide

  26. @BrittBarak

    View Slide

  27. Detector
    detector
    .execute(image)
    Result:
    @BrittBarak
    “Ben & Jerry’s
    pistachio
    ice cream”

    View Slide

  28. 1. Setup Detector
    @BrittBarak

    View Slide

  29. Local or cloud?
    @BrittBarak

    View Slide

  30. @BrittBarak

    View Slide

  31. Local
    •Realtime
    •Offline support
    •Security / Privacy
    •Bandwith
    •…
    @BrittBarak

    View Slide

  32. Cloud
    •More accuracy & features
    •But more latency
    •Pricing
    @BrittBarak

    View Slide

  33. 1. Setup Detector
    @BrittBarak

    View Slide

  34. Text Detector
    textDetector =
    FirebaseVision.getInstance()
    @BrittBarak

    View Slide

  35. Text Detector
    textDetector =
    FirebaseVision.getInstance()
    .onDeviceTextRecognizer
    @BrittBarak

    View Slide

  36. Text Detector
    textDetector =
    FirebaseVision.getInstance()
    .cloudTextRecognizer
    @BrittBarak

    View Slide

  37. 2. Process input
    @BrittBarak

    View Slide

  38. FirebaseVisionImage
    •Bitmap
    •image Uri
    •Media Image
    •byteArray
    •byteBuffer
    @BrittBarak

    View Slide

  39. image = FirebaseVisionImage.fromBitmap(bitmap)
    @BrittBarak
    Text Detector

    View Slide

  40. 3. Execute the model
    @BrittBarak

    View Slide

  41. Text Detector
    textDetector.processImage(image)
    @BrittBarak

    View Slide

  42. Text Detector
    textDetector.processImage(image)
    .addOnSuccessListener {
    }
    @BrittBarak

    View Slide

  43. Text Detector
    textDetector.processImage(image)
    .addOnSuccessListener {
    firebaseVisionTexts ->
    processOutput(fbVisionTexts)
    }
    @BrittBarak

    View Slide

  44. 4. Process output
    @BrittBarak

    View Slide

  45. firebaseVisionTexts.text
    @BrittBarak

    View Slide

  46. someTextView.text =
    firebaseVisionTexts.text
    @BrittBarak
    UI

    View Slide

  47. Result
    @BrittBarak

    View Slide

  48. Result
    @BrittBarak

    View Slide

  49. (another) Example : Labelling
    @BrittBarak

    View Slide

  50. Detector
    detector
    .execute(image)
    Result:
    @BrittBarak
    ice cream pint

    View Slide

  51. Vegetables
    Deserts
    Vegetables

    View Slide

  52. 1. Setup Detector
    @BrittBarak

    View Slide

  53. Image Classifier
    imageDetector =
    FirebaseVision.getInstance()
    @BrittBarak

    View Slide

  54. Image Classifier
    imageDetector =
    FirebaseVision.getInstance()
    .visionLabelDetector
    @BrittBarak

    View Slide

  55. Image Classifier
    imageDetector =
    FirebaseVision.getInstance
    .visionCloudLabelDetector
    @BrittBarak

    View Slide

  56. 2. Process input
    @BrittBarak

    View Slide

  57. image = FirebaseVisionImage.fromBitmap(bitmap)
    @BrittBarak
    Image Classifier

    View Slide

  58. 3. Execute the model
    @BrittBarak

    View Slide

  59. Image Classifier
    imageDetector.detectInImage(image)
    @BrittBarak

    View Slide

  60. Image Classifier
    imageDetector.detectInImage(image)
    .addOnSuccessListener{
    }
    @BrittBarak

    View Slide

  61. Image Classifier
    imageDetector.detectInImage(image)
    .addOnSuccessListener{ fBLabels ->
    processOutput(fBLabels)
    }
    @BrittBarak

    View Slide

  62. 4. Process output
    @BrittBarak

    View Slide

  63. fbLabel.label
    fbLabel.confidence
    fbLabel.entityId
    @BrittBarak

    View Slide

  64. UI
    for (fbLabel in labels) {
    s =
    "${fbLabel.label} : ${fbLabel.confidence}"
    }
    @BrittBarak

    View Slide

  65. Result

    View Slide

  66. Result

    View Slide

  67. It is an ML-ful world
    Enjoy!

    View Slide

  68. Thank you!
    Keep in touch!

    View Slide