Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning for Android Developers

Machine Learning for Android Developers

Ever wanted to build smart Android apps the easy way? This presentation covers all there is to know about Face Detection, Barcode Detection and Text Recognition with the Mobile Vision API.

Moyinoluwa Adeyemi

November 25, 2016
Tweet

More Decks by Moyinoluwa Adeyemi

Other Decks in Technology

Transcript

  1. How Can You Get Started with Machine Learning? Three ways,

    with varying complexity: (1) Use a Cloud-based or Mobile API (Vision, Natural Language, etc.) (2) Use an existing model architecture, and retrain it or fine tune on your dataset (3) Develop your own machine learning models for new problems More flexible, but more effort required
  2. Face API faces, facial landmarks, eyes open, smiling Barcode API

    1D and 2D barcodes Text API Latin-based text / structure Common Mobile Vision API Support for fast image and video on-device detection and tracking. NEW!
  3. Googly Eyes Android App Video credit Google 1. Create a

    face detector for facial landmarks (e.g., eyes) 3. For each face, draw the eyes FaceDetector detector = new FaceDetector.Builder() .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build(); SparseArray<Face> faces = detector.detect(image); for (int i = 0; i < faces.size(); ++i) { Face face = faces.valueAt(i); for (Landmark landmark : face.getLandmarks()) { // Draw eyes 2. Detect faces in the image
  4. Easy to use Java API image detected items Detector 1.

    Create a detector object 2. detectedItems = detector.detect(image) Photo credit developers.google.com/vision
  5. Barcode Detection 1D barcodes EAN-13/8 UPC-A/E Code-39/93/128 ITF Codabar 2D

    barcodes QR Code Data Matrix PDF-417 AZTEC UPC DataMatrix QR Code PDF 417 Video and image credit Google
  6. Import the Google Play Services SDK for the Mobile Vision

    API compile 'com.google.android.gms:play-services-vision:9.8.0'
  7. Convert the static image into a Bitmap Bitmap barcodeBitmap =

    BitmapFactory.decodeResource(getResources(), R.drawable.cute_cat_image);
  8. Check if the face detector is ready if (!faceDetector.isOperational()) {

    new AlertDialog.Builder(this) .setMessage("Face Detector could not be set up on your device") .show(); return; }
  9. Check if the barcode detector is ready if (!barcodeDetector.isOperational()) {

    new AlertDialog.Builder(this) .setMessage("Barcode Detector could not be set up on your device") .show(); return; }
  10. Check if the text recognizer is ready if (!textRecognizer.isOperational()) {

    new AlertDialog.Builder(this) .setMessage("Text Recognizer could not be set up on your device") .show(); return; }
  11. Create a frame using the face bitmap Frame frame =

    new Frame.Builder().setBitmap(textBitmap).build(); SparseArray<Face> face = faceDetector.detect(frame);
  12. Create a frame using the barcode bitmap Frame frame =

    new Frame.Builder().setBitmap(textBitmap).build(); SparseArray<Barcode> barcode = barcodeDetector.detect(frame);
  13. Create a frame using the text bitmap Frame frame =

    new Frame.Builder().setBitmap(textBitmap).build(); SparseArray<TextBlock> text = textRecognizer.detect(frame);
  14. Use the values from the SparseArray for (int i =

    0; i < sparseArray.size(); i++) { Face face = sparseArray.valueAt(i); float left = face.getPosition().x; float top = face.getPosition().y; float right = left + face.getWidth(); float bottom = right + face.getHeight(); float cornerRadius = 2.0f; RectF rectF = new RectF(left, top, right, bottom); canvas.drawRoundRect(rectF, cornerRadius, cornerRadius, rectPaint); }
  15. Use the values from the SparseArray for (int i =

    0; i < sparseArray.size(); i++) { Face face = sparseArray.valueAt(i); float left = face.getPosition().x; float top = face.getPosition().y; float right = left + face.getWidth(); float bottom = right + face.getHeight(); float cornerRadius = 2.0f; RectF rectF = new RectF(left, top, right, bottom); canvas.drawRoundRect(rectF, cornerRadius, cornerRadius, rectPaint); }
  16. Use the values from the SparseArray if (size == 0)

    { textView.setText("No information available"); } else { for (int i = 0; i < size; i++) { barcodeValue += (barcode.valueAt(i).displayValue + "\n"); } textView.setText(barcodeValue); }
  17. for (int i = 0; i < text.size(); i++) {

    TextBlock textBlock = text.valueAt(i); if (textBlock != null && textBlock.getValue() != null) { detectedText += textBlock.getValue(); } detectedTextView.setText(detectedText); } Use the values from the SparseArray
  18. Mobile Vision: Codelabs and Samples Googly Eyes Code Sample github.com/googlesamples/android-vision/tree/master/visionSamples/googly-eyes

    Codelabs codelabs.developers.google.com/codelabs/face-detection/ codelabs.developers.google.com/codelabs/bar-codes/ codelabs.developers.google.com/codelabs/mobile-vision-ocr/ Mobile Vision Developers developers.google.com/vision/ GitHub Code Samples github.com/googlesamples/android-vision Stack Overflow Find and ask questions under the android-vision tag.