What an ML-ful World! MLKit for Android dev.

Transcript

1. What an
   ML-ful world
   Britt Barak
   

   View Slide

2. Once upon a time
   @BrittBarak
   

   View Slide

3. beta
   @BrittBarak
   

   View Slide

4. ML Capability ?!
   @BrittBarak
   

   View Slide

5. Who is afraid of
   Machine Learning?
   & First Steps With ML-Kit
   @BrittBarak
   

   View Slide

6. Britt Barak
   Developer Experience, Nexmo
   Google Developer Expert
   Britt Barak @brittBarak
   

   View Slide

7. View Slide

8. @BrittBarak
   

   View Slide

9. =
   @BrittBarak
   

   View Slide

10. §
    What’s the difference?
    @BrittBarak
    

    View Slide

11. …and classify?
    @BrittBarak
    

    View Slide

12. @BrittBarak
    

    View Slide

13. This is a strawberry
    @BrittBarak
    

    View Slide

14. This is a strawberry
    Red Seeds
    pattern
    Narrow
    top
    leaves
    @BrittBarak
    Pointy
    at the
    bottom
    Round
    at the top
    

    View Slide

15. Strawberry Not
    Not
    Not
    Strawberry
    Strawberry
    Not
    Not
    Not
    @BrittBarak
    

    View Slide

16. ~*~ images ~*~
    @BrittBarak
    

    View Slide

17. @BrittBarak
    Vision library
    

    View Slide

18. Text Recognition
    @BrittBarak
    

    View Slide

19. Face Detection
    @BrittBarak
    

    View Slide

20. Barcode Scanning
    @BrittBarak
    

    View Slide

21. Image Labelling
    @BrittBarak
    

    View Slide

22. Landmark Recognition
    @BrittBarak
    

    View Slide

23. Custom Models
    @BrittBarak
    

    View Slide

24. Example
    @BrittBarak
    

    View Slide

25. @BrittBarak
    

    View Slide

26. @BrittBarak
    

    View Slide

27. Detector
    detector
    .execute(image)
    Result:
    @BrittBarak
    “Ben & Jerry’s
    pistachio
    ice cream”
    

    View Slide

28. 1. Setup Detector
    @BrittBarak
    

    View Slide

29. Local or cloud?
    @BrittBarak
    

    View Slide

30. @BrittBarak
    

    View Slide

31. Local
    •Realtime
    •Offline support
    •Security / Privacy
    •Bandwith
    •…
    @BrittBarak
    

    View Slide

32. Cloud
    •More accuracy & features
    •But more latency
    •Pricing
    @BrittBarak
    

    View Slide

33. 1. Setup Detector
    @BrittBarak
    

    View Slide

34. Text Detector
    textDetector =
    FirebaseVision.getInstance()
    @BrittBarak
    

    View Slide

35. Text Detector
    textDetector =
    FirebaseVision.getInstance()
    .onDeviceTextRecognizer
    @BrittBarak
    

    View Slide

36. Text Detector
    textDetector =
    FirebaseVision.getInstance()
    .cloudTextRecognizer
    @BrittBarak
    

    View Slide

37. 2. Process input
    @BrittBarak
    

    View Slide

38. FirebaseVisionImage
    •Bitmap
    •image Uri
    •Media Image
    •byteArray
    •byteBuffer
    @BrittBarak
    

    View Slide

39. image = FirebaseVisionImage.fromBitmap(bitmap)
    @BrittBarak
    Text Detector
    

    View Slide

40. 3. Execute the model
    @BrittBarak
    

    View Slide

41. Text Detector
    textDetector.processImage(image)
    @BrittBarak
    

    View Slide

42. Text Detector
    textDetector.processImage(image)
    .addOnSuccessListener {
    }
    @BrittBarak
    

    View Slide

43. Text Detector
    textDetector.processImage(image)
    .addOnSuccessListener {
    firebaseVisionTexts ->
    processOutput(fbVisionTexts)
    }
    @BrittBarak
    

    View Slide

44. 4. Process output
    @BrittBarak
    

    View Slide

45. firebaseVisionTexts.text
    @BrittBarak
    

    View Slide

46. someTextView.text =
    firebaseVisionTexts.text
    @BrittBarak
    UI
    

    View Slide

47. Result
    @BrittBarak
    

    View Slide

48. Result
    @BrittBarak
    

    View Slide

49. (another) Example : Labelling
    @BrittBarak
    

    View Slide

50. Detector
    detector
    .execute(image)
    Result:
    @BrittBarak
    ice cream pint
    

    View Slide

51. Vegetables
    Deserts
    Vegetables
    

    View Slide

52. 1. Setup Detector
    @BrittBarak
    

    View Slide

53. Image Classifier
    imageDetector =
    FirebaseVision.getInstance()
    @BrittBarak
    

    View Slide

54. Image Classifier
    imageDetector =
    FirebaseVision.getInstance()
    .visionLabelDetector
    @BrittBarak
    

    View Slide

55. Image Classifier
    imageDetector =
    FirebaseVision.getInstance
    .visionCloudLabelDetector
    @BrittBarak
    

    View Slide

56. 2. Process input
    @BrittBarak
    

    View Slide

57. image = FirebaseVisionImage.fromBitmap(bitmap)
    @BrittBarak
    Image Classifier
    

    View Slide

58. 3. Execute the model
    @BrittBarak
    

    View Slide

59. Image Classifier
    imageDetector.detectInImage(image)
    @BrittBarak
    

    View Slide

60. Image Classifier
    imageDetector.detectInImage(image)
    .addOnSuccessListener{
    }
    @BrittBarak
    

    View Slide

61. Image Classifier
    imageDetector.detectInImage(image)
    .addOnSuccessListener{ fBLabels ->
    processOutput(fBLabels)
    }
    @BrittBarak
    

    View Slide

62. 4. Process output
    @BrittBarak
    

    View Slide

63. fbLabel.label
    fbLabel.confidence
    fbLabel.entityId
    @BrittBarak
    

    View Slide

64. UI
    for (fbLabel in labels) {
    s =
    "${fbLabel.label} : ${fbLabel.confidence}"
    }
    @BrittBarak
    

    View Slide

65. Result
    

    View Slide

66. Result
    

    View Slide

67. It is an ML-ful world
    Enjoy!
    

    View Slide

68. Thank you!
    Keep in touch!
    

    View Slide