Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning for Android Developers

Machine Learning for Android Developers

Ever wanted to build smart Android apps the easy way? This presentation covers all there is to know about Face Detection, Barcode Detection and Text Recognition with the Mobile Vision API.

Moyinoluwa Adeyemi

November 25, 2016
Tweet

More Decks by Moyinoluwa Adeyemi

Other Decks in Technology

Transcript

  1. Machine Learning for Android
    Developers
    Moyinoluwa Adeyemi
    November 25, 2016

    View full-size slide

  2. Android @SwiftaSystems
    GDG Lagos Co-organizer
    Android Developer Nanodegree Graduate
    Half Marathoner

    View full-size slide

  3. How Can You Get Started with Machine Learning?
    Three ways, with varying complexity:
    (1) Use a Cloud-based or Mobile API (Vision, Natural Language,
    etc.)
    (2) Use an existing model architecture, and retrain it or fine tune
    on your dataset
    (3) Develop your own machine learning models for new
    problems
    More
    flexible,
    but more
    effort
    required

    View full-size slide

  4. Mobile Vision API
    Providing on-device vision for applications

    View full-size slide

  5. Face API
    faces, facial landmarks, eyes
    open, smiling
    Barcode API
    1D and 2D barcodes
    Text API
    Latin-based text / structure
    Common Mobile Vision API
    Support for fast image and video on-device detection and tracking.
    NEW!

    View full-size slide

  6. Googly Eyes Android App
    Video credit Google
    1. Create a face detector for facial landmarks (e.g., eyes)
    3. For each face, draw the eyes
    FaceDetector detector = new FaceDetector.Builder()
    .setLandmarkType(FaceDetector.ALL_LANDMARKS)
    .build();
    SparseArray faces = detector.detect(image);
    for (int i = 0; i < faces.size(); ++i) {
    Face face = faces.valueAt(i);
    for (Landmark landmark : face.getLandmarks()) {
    // Draw eyes
    2. Detect faces in the image

    View full-size slide

  7. Face API
    Photo credit developers.google.com/vision

    View full-size slide

  8. Easy to use Java API
    image detected
    items
    Detector
    1. Create a detector object
    2. detectedItems = detector.detect(image)
    Photo credit developers.google.com/vision

    View full-size slide

  9. Text Detection
    Latin based language
    Understand text structure
    Photo credit Getty Images

    View full-size slide

  10. Text Structure
    Blocks
    Lines
    Words
    Lines
    Words Words Words

    View full-size slide

  11. Barcode Detection
    1D barcodes
    EAN-13/8
    UPC-A/E
    Code-39/93/128
    ITF
    Codabar
    2D barcodes
    QR Code
    Data Matrix
    PDF-417
    AZTEC
    UPC
    DataMatrix
    QR Code
    PDF 417
    Video and image credit Google

    View full-size slide

  12. Combined Vision & Translation

    View full-size slide

  13. Code Walkthrough

    View full-size slide

  14. Import the Google Play Services SDK for the
    Mobile Vision API
    compile 'com.google.android.gms:play-services-vision:9.8.0'

    View full-size slide

  15. Face Detection manifest file meta-data
    android:name="com.google.android.gms.vision.DEPENDENCIES"
    android:value="face"/>

    View full-size slide

  16. Barcode Detection manifest file meta-data
    android:name="com.google.android.gms.vision.DEPENDENCIES"
    android:value="barcode"/>

    View full-size slide

  17. Text Recognition manifest file meta-data
    android:name="com.google.android.gms.vision.DEPENDENCIES"
    android:value="ocr"/>

    View full-size slide

  18. Create simple layout with static image
    Process

    View full-size slide

  19. Convert the static image into a Bitmap
    Bitmap barcodeBitmap = BitmapFactory.decodeResource(getResources(),
    R.drawable.cute_cat_image);

    View full-size slide

  20. Using the FaceDetector API
    FaceDetector faceDetector = new FaceDetector.Builder(this)
    .setTrackingEnabled(false)
    .setLandmarkType(FaceDetector.ALL_LANDMARKS)
    .build();

    View full-size slide

  21. Using the BarcodeDetector API
    BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(this)
    .setBarcodeFormats(Barcode.ALL_FORMATS)
    .build();

    View full-size slide

  22. Using the TextRecognizer API
    TextRecognizer textRecognizer = new TextRecognizer.Builder(this)
    .build();

    View full-size slide

  23. Check if the face detector is ready
    if (!faceDetector.isOperational()) {
    new AlertDialog.Builder(this)
    .setMessage("Face Detector could not be set up on your device")
    .show();
    return;
    }

    View full-size slide

  24. Check if the barcode detector is ready
    if (!barcodeDetector.isOperational()) {
    new AlertDialog.Builder(this)
    .setMessage("Barcode Detector could not be set up on your
    device")
    .show();
    return;
    }

    View full-size slide

  25. Check if the text recognizer is ready
    if (!textRecognizer.isOperational()) {
    new AlertDialog.Builder(this)
    .setMessage("Text Recognizer could not be set up on your device")
    .show();
    return;
    }

    View full-size slide

  26. Create a frame using the face bitmap
    Frame frame = new Frame.Builder().setBitmap(textBitmap).build();
    SparseArray face = faceDetector.detect(frame);

    View full-size slide

  27. Create a frame using the barcode bitmap
    Frame frame = new Frame.Builder().setBitmap(textBitmap).build();
    SparseArray barcode = barcodeDetector.detect(frame);

    View full-size slide

  28. Create a frame using the text bitmap
    Frame frame = new Frame.Builder().setBitmap(textBitmap).build();
    SparseArray text = textRecognizer.detect(frame);

    View full-size slide

  29. Use the values from the SparseArray
    for (int i = 0; i < sparseArray.size(); i++) {
    Face face = sparseArray.valueAt(i);
    float left = face.getPosition().x;
    float top = face.getPosition().y;
    float right = left + face.getWidth();
    float bottom = right + face.getHeight();
    float cornerRadius = 2.0f;
    RectF rectF = new RectF(left, top, right, bottom);
    canvas.drawRoundRect(rectF, cornerRadius, cornerRadius, rectPaint);
    }

    View full-size slide

  30. Use the values from the SparseArray
    for (int i = 0; i < sparseArray.size(); i++) {
    Face face = sparseArray.valueAt(i);
    float left = face.getPosition().x;
    float top = face.getPosition().y;
    float right = left + face.getWidth();
    float bottom = right + face.getHeight();
    float cornerRadius = 2.0f;
    RectF rectF = new RectF(left, top, right, bottom);
    canvas.drawRoundRect(rectF, cornerRadius, cornerRadius, rectPaint);
    }

    View full-size slide

  31. Use the values from the SparseArray
    if (size == 0) {
    textView.setText("No information available");
    } else {
    for (int i = 0; i < size; i++) {
    barcodeValue += (barcode.valueAt(i).displayValue + "\n");
    }
    textView.setText(barcodeValue);
    }

    View full-size slide

  32. for (int i = 0; i < text.size(); i++) {
    TextBlock textBlock = text.valueAt(i);
    if (textBlock != null && textBlock.getValue() != null) {
    detectedText += textBlock.getValue();
    }
    detectedTextView.setText(detectedText);
    }
    Use the values from the SparseArray

    View full-size slide

  33. faceDetector.release();
    Remember to release resources

    View full-size slide

  34. barcodeDetector.release();
    Remember to release resources

    View full-size slide

  35. textRecognizer.release();
    Remember to release resources

    View full-size slide

  36. Results - Face Detection API

    View full-size slide

  37. Results - Barcode Detection API

    View full-size slide

  38. Results - Text Recognition API

    View full-size slide

  39. Mobile Vision: Codelabs and Samples
    Googly Eyes Code Sample
    github.com/googlesamples/android-vision/tree/master/visionSamples/googly-eyes
    Codelabs
    codelabs.developers.google.com/codelabs/face-detection/
    codelabs.developers.google.com/codelabs/bar-codes/
    codelabs.developers.google.com/codelabs/mobile-vision-ocr/
    Mobile Vision Developers
    developers.google.com/vision/
    GitHub Code Samples
    github.com/googlesamples/android-vision
    Stack Overflow
    Find and ask questions under the android-vision tag.

    View full-size slide

  40. @moyheen @moyinoluwa
    @moyheen
    Thank you!
    Moyinoluwa Adeyemi
    Android @SwiftaSystems
    GDG Lagos co-organizer

    View full-size slide