Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making Android Apps With Intelligience - GDG Bellevue

Making Android Apps With Intelligience - GDG Bellevue

This talk was presented at the Seattle Eastside Android (GDG Bellevue) meetup on 9/28/2016.

Machine learning is everywhere: from personal assistants to self driving cars to robots. Machine learning is not new, but the exciting news is that the various machine learning services are now available to app developers, some are even open source e.g. TensorFlow. In this talk I will give an overview of how you can incorporate the machine learning services with your Android apps without any deep expertise in ML.

Margaret Maynard-Reid

September 28, 2016
Tweet

More Decks by Margaret Maynard-Reid

Other Decks in Technology

Transcript

  1. Making Android Apps with Intelligence Eastside Android meetup @Offerup, 9/28/2016

    By Margaret Maynard-Reid @margaretmz +MargaretMaynardReid @margaretmz @margaretmz +MargaretMaynardReid @margaretmz
  2. Machine learning everywhere! News stories - • Google TensorFlow makes

    music • IBM Watson creates movie trailer • Uber uses Microsoft Cognitive services to selfie check drivers 3
  3. What is Machine Learning? • A subset of Artificial Intelligence

    • Study of algorithms • Learn from examples and experience (instead of hard-coded programming rules) Source: ML Recipes #1 by Google Developers - https://www.youtube.com/watch?v=cKxRvEZd3Mw Collect training data Train a model Use model to predict 4
  4. Make Apps with Intelligence Complexity ML Services (via REST APIs)

    Vision Speech Language Text… (Google, MS, IBM, HP…) ML Services (build or train your ML models) Google Cloud ML Platform TensorFlow Amazon ML Google Play Services Mobile Vision API 5
  5. Google Mobile Vision API • Part of the Google Play

    services • Very easy to get started • Can detect: ◦ Face (FaceDetector) | codelab ◦ Barcode (BarcodeDetector) | codelab ◦ Text (TextRecognizer) | codelab https://developers.google.com/vision/ 7
  6. Face API • Facial detection not recognition • Detection from

    image or live video • Face can be detected from different angles • Detects activities: ◦ winking, blinking and smiling • Detect landmarks on face 8
  7. Face detector Add this to app build.gradle file Create a

    detector Make sure detector is operational first com.google.android.gms:play-services-vision:9.2.1 FaceDetector faceDetector = new FaceDetector.Builder(getApplicationContext()) .setTrackingEnabled(false) .build(); if(!faceDetector.isOperational()){ // let user know face detector isn’t working // log error etc... } 9
  8. Face detector Detect the face Draw Rectangles on the Faces

    Refer to Face Detection with Vision API code lab for details... Frame frame = new Frame.Builder().setBitmap(myBitmap).build(); SparseArray<Face> faces = faceDetector.detect(frame); 10
  9. Googly Eyes Android App Slide & video credit: Google 1.

    Create a face detector for facial landmarks (e.g., eyes) 3. For each face, draw the eyes FaceDetector detector = new FaceDetector.Builder() .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build(); SparseArray<Face> faces = detector.detect(image); for (int i = 0; i < faces.size(); ++i) { Face face = faces.valueAt(i); for (Landmark landmark : face.getLandmarks()) { // Draw eyes 2. Detect faces in the image
  10. Machine learning services Pre-trained models No need for much ML

    knowledge Just REST API calls “Try the API” directly online from the websites • Google Cloud ML APIs • Microsoft Cognitive Services • IBM Watson • HPE Haven OnDemand 13
  11. Getting started Free trial based on time or # of

    transactions A typical process: • Get API key • Include key in Android app • Download SDK or include dependency in build.gradle (if sdk available) • Make your REST API calls 14
  12. REST API calls ML REST APIs Vision Speech Language Text

    Translate Search…. HTTP response images text HTTP call audio video 15
  13. Microsoft Vision APIs Vision APIs • Computer Vision, Emotion &

    Face Can detect • Gender • Age • Emotions etc. Demo: SampleVision - image caption 16
  14. Demo - Mimicker Alarm Mimicker Alarm uses 3 ML APIs:

    • Computer Vision • Emotion • Speech https://github.com/Microsoft/ProjectOxford-Apps-MimickerAlarm 17
  15. Microsoft LUIS • Language Understanding Intelligence Service https://www.luis.ai/ • Train

    a pre-built language understanding model ◦ Create an Intent - CreateCalendarEvent ◦ Add parameters - when, what, who • Deploy model to an http endpoint - interprets language • Use the model on mobile app - JSON response 18
  16. ML Services Build your own ML models, or train your

    own models. Examples: • Google Cloud ML Platform • Amazon ML 20
  17. Google TensorFlow • Open source machine learning library https://www.tensorflow.org/ •

    A Neural Network Playground - TensorFlow • Code lab: TensorFlow for Poets Demo: Android sample app with TensorFlow (works offline) 21
  18. Amazon Alexa Voice as the user interface! • Train Alexa

    by building new skills • Connect what you build with Android • Create your own echo - DIY echo with Alexa + Raspberry Pi 23
  19. Pepper the Robot + Android 1. Download Android plugin &

    SDK 2. Create an Android project 3. Enable robot project structure Now you are ready to program robots! https://android.aldebaran.com/doc/introduction.html 24
  20. Design considerations • Does your app really need the intelligence?

    • Choose a API/service ◦ ease of use, support, pricing • Get user permissions • Inform user about data privacy • Be mindful with user's data 26
  21. What is next? The future is already here! • Build

    an app with intelligence • Make your own echo • Program a robot • Study machine learning basics 27
  22. Appendix ML Sessions at I/O - • Google’s Vision on

    Machine Learning • Machine Learning for Art • Machine learning is not the future • Breakthroughs in Machine Learning • How to build a smart RasPi Bot with Cloud Vision and Speech API 29
  23. Appendix ML learning resources Udacity • Intro to Machine Learning

    • Deep Learning (TensorFlow) Stanford • Andrew Ng’s ML course on Coursera Google Developers • Machine Learning Recipes Reading • Python Machine Learning by Sabastian Raschka 30