What an ML-ful World! MLKit for Android dev.

Transcript

1. What an ML-ful world Britt Barak

2. Once upon a time @BrittBarak

3. beta @BrittBarak

4. ML Capability ?! @BrittBarak

5. Who is afraid of Machine Learning? & First Steps With

   ML-Kit @BrittBarak

6. Britt Barak Developer Experience, Nexmo Google Developer Expert Britt Barak

   @brittBarak
7. None

8. @BrittBarak

9. = @BrittBarak

10. § What’s the difference? @BrittBarak

11. …and classify? @BrittBarak

12. @BrittBarak

13. This is a strawberry @BrittBarak

14. This is a strawberry Red Seeds pattern Narrow top leaves

    @BrittBarak Pointy at the bottom Round at the top

15. Strawberry Not Not Not Strawberry Strawberry Not Not Not @BrittBarak

16. ~*~ images ~*~ @BrittBarak

17. @BrittBarak Vision library

18. Text Recognition @BrittBarak

19. Face Detection @BrittBarak

20. Barcode Scanning @BrittBarak

21. Image Labelling @BrittBarak

22. Landmark Recognition @BrittBarak

23. Custom Models @BrittBarak

24. Example @BrittBarak

25. @BrittBarak

26. @BrittBarak

27. Detector detector .execute(image) Result: @BrittBarak “Ben & Jerry’s pistachio ice

    cream”

28. 1. Setup Detector @BrittBarak

29. Local or cloud? @BrittBarak

30. @BrittBarak

31. Local •Realtime •Offline support •Security / Privacy •Bandwith •… @BrittBarak

32. Cloud •More accuracy & features •But more latency •Pricing @BrittBarak

33. 1. Setup Detector @BrittBarak

34. Text Detector textDetector = FirebaseVision.getInstance() @BrittBarak

35. Text Detector textDetector = FirebaseVision.getInstance() .onDeviceTextRecognizer @BrittBarak

36. Text Detector textDetector = FirebaseVision.getInstance() .cloudTextRecognizer @BrittBarak

37. 2. Process input @BrittBarak

38. FirebaseVisionImage •Bitmap •image Uri •Media Image •byteArray •byteBuffer @BrittBarak

39. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Text Detector

40. 3. Execute the model @BrittBarak

41. Text Detector textDetector.processImage(image) @BrittBarak

42. Text Detector textDetector.processImage(image) .addOnSuccessListener { } @BrittBarak

43. Text Detector textDetector.processImage(image) .addOnSuccessListener { firebaseVisionTexts -> processOutput(fbVisionTexts) } @BrittBarak

44. 4. Process output @BrittBarak

45. firebaseVisionTexts.text @BrittBarak

46. someTextView.text = firebaseVisionTexts.text @BrittBarak UI

47. Result @BrittBarak

48. Result @BrittBarak

49. (another) Example : Labelling @BrittBarak

50. Detector detector .execute(image) Result: @BrittBarak ice cream pint

51. Vegetables Deserts Vegetables

52. 1. Setup Detector @BrittBarak

53. Image Classifier imageDetector = FirebaseVision.getInstance() @BrittBarak

54. Image Classifier imageDetector = FirebaseVision.getInstance() .visionLabelDetector @BrittBarak

55. Image Classifier imageDetector = FirebaseVision.getInstance .visionCloudLabelDetector @BrittBarak

56. 2. Process input @BrittBarak

57. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Image Classifier

58. 3. Execute the model @BrittBarak

59. Image Classifier imageDetector.detectInImage(image) @BrittBarak

60. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ } @BrittBarak

61. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ fBLabels -> processOutput(fBLabels) } @BrittBarak

62. 4. Process output @BrittBarak

63. fbLabel.label fbLabel.confidence fbLabel.entityId @BrittBarak

64. UI for (fbLabel in labels) { s = "${fbLabel.label} :

    ${fbLabel.confidence}" } @BrittBarak

65. Result

66. Result

67. It is an ML-ful world Enjoy!

68. Thank you! Keep in touch!