Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What an ML-ful World! MLKit for Android dev.

Britt Barak
October 12, 2018
Programming
0
120

What an ML-ful World! MLKit for Android dev.

Britt Barak

October 12, 2018
Tweet

More Decks by Britt Barak

See All by Britt Barak
[Vonage] Introducing Conversations
brittbarak
1
100
Kids, Play Nice! Kotlin-Java Interop In Mind
brittbarak
2
190
Sharing is Caring- Getting Started with Kotlin Multiplatform
brittbarak
2
1.5k
Between JOMO and FOMO: You are reshaping communication.
brittbarak
2
1.1k
Build Apps For The Ones You Love
brittbarak
1
79
Make your app dance with MotionLayout
brittbarak
8
1.1k
Who's afraid of ML? V2 : First steps with MlKit
brittbarak
1
380
Oh, the places you'll go! Cracking Navigation on Android
brittbarak
0
410
The organic evolution - how and why we created peer mentorship program
brittbarak
0
29

Other Decks in Programming

See All in Programming
20分間で振り返るIn-App Purchaseの歴史
inokinn
3
640
I shall define this only once
freekmurze
0
210
サーバーと同期してリアルタイムに更新する画面を実装する
rizumi
2
180
Swift アクターモデルと Elm Architecture の融合 / iOSDC Japan 2022
inamiy
3
850
Perl でも GraphQL を使いたい
nakaokat
0
1.2k
FluorSwitch / RayTracingCamp8
kugimasa
0
280
Towards Ruby 4 JIT / RubyKaigi 2022
k0kubun
3
2k
出版したら終わり、にしない技術書執筆 / #DevRel #DevReljp
jnchito
0
7.8k
入門Hello world（PHPカンファレンス2022）
yutaron
0
120
YJIT: Dive into Ruby's JIT compiler written in Rust / Rust.Tokyo 2022
k0kubun
1
750
Laravel を低速化する技術 / how to slow laravel
hanhan1978
PRO
0
380
Drupal 10 from the pov of a Symfony 6 core-contributor
nicolasgrekas
1
260

Featured

See All Featured
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
120
28k
The Pragmatic Product Professional
lauravandoore
19
3.2k
Why You Should Never Use an ORM
jnunemaker
PRO
48
7.7k
Building Your Own Lightsaber
phodgson
95
4.7k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
Art, The Web, and Tiny UX
lynnandtonic
281
18k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
237
19k
Adopting Sorbet at Scale
ufuk
65
7.6k
Music & Morning Musume
bryan
35
4.4k
Scaling GitHub
holman
452
140k
Building Adaptive Systems
keathley
25
1.2k
Intergalactic Javascript Robots from Outer Space
tanoku
259
25k

Transcript

  1. What an ML-ful world Britt Barak

  2. Once upon a time @BrittBarak

  3. beta @BrittBarak

  4. ML Capability ?! @BrittBarak

  5. Who is afraid of Machine Learning? & First Steps With

    ML-Kit @BrittBarak

  6. Britt Barak Developer Experience, Nexmo Google Developer Expert Britt Barak

    @brittBarak
  7. None

  8. @BrittBarak

  9. = @BrittBarak

  10. § What’s the difference? @BrittBarak

  11. …and classify? @BrittBarak

  12. @BrittBarak

  13. This is a strawberry @BrittBarak

  14. This is a strawberry Red Seeds pattern Narrow top leaves

    @BrittBarak Pointy at the bottom Round at the top

  15. Strawberry Not Not Not Strawberry Strawberry Not Not Not @BrittBarak

  16. ~*~ images ~*~ @BrittBarak

  17. @BrittBarak Vision library

  18. Text Recognition @BrittBarak

  19. Face Detection @BrittBarak

  20. Barcode Scanning @BrittBarak

  21. Image Labelling @BrittBarak

  22. Landmark Recognition @BrittBarak

  23. Custom Models @BrittBarak

  24. Example @BrittBarak

  25. @BrittBarak

  26. @BrittBarak

  27. Detector detector .execute(image) Result: @BrittBarak “Ben & Jerry’s pistachio ice

    cream”

  28. 1. Setup Detector @BrittBarak

  29. Local or cloud? @BrittBarak

  30. @BrittBarak

  31. Local •Realtime •Offline support •Security / Privacy •Bandwith •… @BrittBarak

  32. Cloud •More accuracy & features •But more latency •Pricing @BrittBarak

  33. 1. Setup Detector @BrittBarak

  34. Text Detector textDetector = FirebaseVision.getInstance() @BrittBarak

  35. Text Detector textDetector = FirebaseVision.getInstance() .onDeviceTextRecognizer @BrittBarak

  36. Text Detector textDetector = FirebaseVision.getInstance() .cloudTextRecognizer @BrittBarak

  37. 2. Process input @BrittBarak

  38. FirebaseVisionImage •Bitmap •image Uri •Media Image •byteArray •byteBuffer @BrittBarak

  39. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Text Detector

  40. 3. Execute the model @BrittBarak

  41. Text Detector textDetector.processImage(image) @BrittBarak

  42. Text Detector textDetector.processImage(image) .addOnSuccessListener { } @BrittBarak

  43. Text Detector textDetector.processImage(image) .addOnSuccessListener { firebaseVisionTexts -> processOutput(fbVisionTexts) } @BrittBarak

  44. 4. Process output @BrittBarak

  45. firebaseVisionTexts.text @BrittBarak

  46. someTextView.text = firebaseVisionTexts.text @BrittBarak UI

  47. Result @BrittBarak

  48. Result @BrittBarak

  49. (another) Example : Labelling @BrittBarak

  50. Detector detector .execute(image) Result: @BrittBarak ice cream pint

  51. Vegetables Deserts Vegetables

  52. 1. Setup Detector @BrittBarak

  53. Image Classifier imageDetector = FirebaseVision.getInstance() @BrittBarak

  54. Image Classifier imageDetector = FirebaseVision.getInstance() .visionLabelDetector @BrittBarak

  55. Image Classifier imageDetector = FirebaseVision.getInstance .visionCloudLabelDetector @BrittBarak

  56. 2. Process input @BrittBarak

  57. image = FirebaseVisionImage.fromBitmap(bitmap) @BrittBarak Image Classifier

  58. 3. Execute the model @BrittBarak

  59. Image Classifier imageDetector.detectInImage(image) @BrittBarak

  60. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ } @BrittBarak

  61. Image Classifier imageDetector.detectInImage(image) .addOnSuccessListener{ fBLabels -> processOutput(fBLabels) } @BrittBarak

  62. 4. Process output @BrittBarak

  63. fbLabel.label fbLabel.confidence fbLabel.entityId @BrittBarak

  64. UI for (fbLabel in labels) { s = "${fbLabel.label} :

    ${fbLabel.confidence}" } @BrittBarak

  65. Result

  66. Result

  67. It is an ML-ful world Enjoy!

  68. Thank you! Keep in touch!