Machine Learning for Android Developers

Machine Learning for Android Developers

Ever wanted to build smart Android apps the easy way? This presentation covers all there is to know about Face Detection, Barcode Detection and Text Recognition with the Mobile Vision API.

E803718649600ddffc1bc625d957e786?s=128

Moyinoluwa Adeyemi

November 25, 2016
Tweet

Transcript

  1. Machine Learning for Android Developers Moyinoluwa Adeyemi November 25, 2016

  2. Android @SwiftaSystems GDG Lagos Co-organizer Android Developer Nanodegree Graduate Half

    Marathoner
  3. How Can You Get Started with Machine Learning? Three ways,

    with varying complexity: (1) Use a Cloud-based or Mobile API (Vision, Natural Language, etc.) (2) Use an existing model architecture, and retrain it or fine tune on your dataset (3) Develop your own machine learning models for new problems More flexible, but more effort required
  4. Mobile Vision API Providing on-device vision for applications

  5. Face API faces, facial landmarks, eyes open, smiling Barcode API

    1D and 2D barcodes Text API Latin-based text / structure Common Mobile Vision API Support for fast image and video on-device detection and tracking. NEW!
  6. Googly Eyes Android App Video credit Google 1. Create a

    face detector for facial landmarks (e.g., eyes) 3. For each face, draw the eyes FaceDetector detector = new FaceDetector.Builder() .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build(); SparseArray<Face> faces = detector.detect(image); for (int i = 0; i < faces.size(); ++i) { Face face = faces.valueAt(i); for (Landmark landmark : face.getLandmarks()) { // Draw eyes 2. Detect faces in the image
  7. Face API Photo credit developers.google.com/vision

  8. Easy to use Java API image detected items Detector 1.

    Create a detector object 2. detectedItems = detector.detect(image) Photo credit developers.google.com/vision
  9. Text Detection Latin based language Understand text structure Photo credit

    Getty Images
  10. Text Structure Blocks Lines Words Lines Words Words Words

  11. Barcode Detection 1D barcodes EAN-13/8 UPC-A/E Code-39/93/128 ITF Codabar 2D

    barcodes QR Code Data Matrix PDF-417 AZTEC UPC DataMatrix QR Code PDF 417 Video and image credit Google
  12. Combined Vision & Translation

  13. Code Walkthrough

  14. Import the Google Play Services SDK for the Mobile Vision

    API compile 'com.google.android.gms:play-services-vision:9.8.0'
  15. Face Detection manifest file meta-data <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="face"/>

  16. Barcode Detection manifest file meta-data <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="barcode"/>

  17. Text Recognition manifest file meta-data <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="ocr"/>

  18. Create simple layout with static image Process

  19. Convert the static image into a Bitmap Bitmap barcodeBitmap =

    BitmapFactory.decodeResource(getResources(), R.drawable.cute_cat_image);
  20. Using the FaceDetector API FaceDetector faceDetector = new FaceDetector.Builder(this) .setTrackingEnabled(false)

    .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build();
  21. Using the BarcodeDetector API BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(this) .setBarcodeFormats(Barcode.ALL_FORMATS)

    .build();
  22. Using the TextRecognizer API TextRecognizer textRecognizer = new TextRecognizer.Builder(this) .build();

  23. Check if the face detector is ready if (!faceDetector.isOperational()) {

    new AlertDialog.Builder(this) .setMessage("Face Detector could not be set up on your device") .show(); return; }
  24. Check if the barcode detector is ready if (!barcodeDetector.isOperational()) {

    new AlertDialog.Builder(this) .setMessage("Barcode Detector could not be set up on your device") .show(); return; }
  25. Check if the text recognizer is ready if (!textRecognizer.isOperational()) {

    new AlertDialog.Builder(this) .setMessage("Text Recognizer could not be set up on your device") .show(); return; }
  26. Create a frame using the face bitmap Frame frame =

    new Frame.Builder().setBitmap(textBitmap).build(); SparseArray<Face> face = faceDetector.detect(frame);
  27. Create a frame using the barcode bitmap Frame frame =

    new Frame.Builder().setBitmap(textBitmap).build(); SparseArray<Barcode> barcode = barcodeDetector.detect(frame);
  28. Create a frame using the text bitmap Frame frame =

    new Frame.Builder().setBitmap(textBitmap).build(); SparseArray<TextBlock> text = textRecognizer.detect(frame);
  29. Use the values from the SparseArray for (int i =

    0; i < sparseArray.size(); i++) { Face face = sparseArray.valueAt(i); float left = face.getPosition().x; float top = face.getPosition().y; float right = left + face.getWidth(); float bottom = right + face.getHeight(); float cornerRadius = 2.0f; RectF rectF = new RectF(left, top, right, bottom); canvas.drawRoundRect(rectF, cornerRadius, cornerRadius, rectPaint); }
  30. Use the values from the SparseArray for (int i =

    0; i < sparseArray.size(); i++) { Face face = sparseArray.valueAt(i); float left = face.getPosition().x; float top = face.getPosition().y; float right = left + face.getWidth(); float bottom = right + face.getHeight(); float cornerRadius = 2.0f; RectF rectF = new RectF(left, top, right, bottom); canvas.drawRoundRect(rectF, cornerRadius, cornerRadius, rectPaint); }
  31. Use the values from the SparseArray if (size == 0)

    { textView.setText("No information available"); } else { for (int i = 0; i < size; i++) { barcodeValue += (barcode.valueAt(i).displayValue + "\n"); } textView.setText(barcodeValue); }
  32. for (int i = 0; i < text.size(); i++) {

    TextBlock textBlock = text.valueAt(i); if (textBlock != null && textBlock.getValue() != null) { detectedText += textBlock.getValue(); } detectedTextView.setText(detectedText); } Use the values from the SparseArray
  33. faceDetector.release(); Remember to release resources

  34. barcodeDetector.release(); Remember to release resources

  35. textRecognizer.release(); Remember to release resources

  36. Results - Face Detection API

  37. Results - Barcode Detection API

  38. Results - Text Recognition API

  39. Mobile Vision: Codelabs and Samples Googly Eyes Code Sample github.com/googlesamples/android-vision/tree/master/visionSamples/googly-eyes

    Codelabs codelabs.developers.google.com/codelabs/face-detection/ codelabs.developers.google.com/codelabs/bar-codes/ codelabs.developers.google.com/codelabs/mobile-vision-ocr/ Mobile Vision Developers developers.google.com/vision/ GitHub Code Samples github.com/googlesamples/android-vision Stack Overflow Find and ask questions under the android-vision tag.
  40. Questions?

  41. @moyheen @moyinoluwa @moyheen Thank you! Moyinoluwa Adeyemi Android @SwiftaSystems GDG

    Lagos co-organizer