A subset of Artificial Intelligence • Study of algorithms • Learn from examples and experience (instead of hard-coded programming rules) Source: ML Recipes #1 by Google Developers - https://www.youtube.com/watch?v=cKxRvEZd3Mw Use model to predict Train a model Collect training data 6
Get started with making apps with intelligence • Share ML resources • Understand the importance of ML for app developers • Disclaimer: not a crash course on ML 7
ML Services (via REST APIs) Vision Speech Language Text… (Google, Microsoft, IBM, HP…) ML Services ( train or build your ML models) Microsoft LUIS TensorFlow Google Cloud ML Platform Amazon ML Platform Google Play Services Mobile Vision API 8
Part of the Google Play services • Very easy to get started • Can detect: ◦ Face (FaceDetector) | codelab ◦ Barcode (BarcodeDetector) | codelab ◦ Text (TextRecognizer) | codelab • Developer documentation on Mobile Vision API (https://developers.google.com/vision/) • I/O 17 talk - Getting Started with Machine Perception Using the Mobile Vision API (https://youtu.be/KKRbwO06SwQ) 10
Detection • Detect one or multiple faces • Facial detection not recognition • Detection from image or live video • Face can be detected from different angles • Detects activities: ◦ winking, blinking and smiling • Detect landmarks on face 11
app build.gradle file Create a detector Make sure detector is operational first com.google.android.gms:play-services-vision:9.2.1 FaceDetector faceDetector = new FaceDetector.Builder(getApplicationContext()) .setTrackingEnabled(false) .build(); if(!faceDetector.isOperational()){ // let user know face detector isn’t working // log error etc... } 12
Draw Rectangles on the Faces Refer to Face Detection with Vision API code lab for details... Frame frame = new Frame.Builder().setBitmap(myBitmap).build(); SparseArray<Face> faces = faceDetector.detect(frame); 13
1. Create a face detector for facial landmarks (e.g., eyes) 3. For each face, draw the eyes FaceDetector detector = new FaceDetector.Builder() .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build(); SparseArray<Face> faces = detector.detect(image); for (int i = 0; i < faces.size(); ++i) { Face face = faces.valueAt(i); for (Landmark landmark : face.getLandmarks()) { // Draw eyes 2. Detect faces in the image 14 Googly Eyes Android App
Detection • Detects barcodes in real-time, on device, in any orientation. • Detects multiple barcodes at once • Detects and parses barcodes and Qr codes of various formats 15
Recognition • Detects text in real-time, on device. • Detects text in Latin based languages (French, German, English, etc.) • Recognizes text in images and video streams 16
on time or # of transactions A typical process: • Get an API key • Include key in Android app • Download SDK or include dependency in build.gradle (if sdk available) • Make your REST API calls 19
No need for much ML knowledge Just REST API calls “Try out the APIs” directly online from the websites • Google Cloud ML APIs • Microsoft Cognitive Services • IBM Watson • HPE Haven OnDemand 20
Service - • User friendly web interface • Train a pre-built language understanding model • Deploy model to an http endpoint - interprets human language • Use the model on mobile app - JSON response 26
weather app 1. Go to https://www.luis.ai/ 2. Create an Application - “Weather app demo” 3. Add two Pre-built Entities: geography & datetime 4. Add an Intent • Name = “GetWeather” • Example utterance = “What is the weather like in Seattle today?” • Two parameters “where” (geography) & “when” (datetime) 5. Train with a few New Utterances (at least 5) • What is the weather in San Francisco this Sunday? • What is the weather in New York tomorrow? 6. Publish the http endpoint which can then be used in your app 27
Open source Machine Learning library • Especially useful for Deep Learning • For research and production • Apache 2.0 license • 1.0.1 released recently 28
and iOS • Android sample app with TensorFlow ◦ TF Classify (demo) ◦ TF Detect ◦ TF Stylize (demo) • Checkout all TensorFlow Dev Summit videos ◦ Pete Warden’s talk Mobile and Embedded TensorFlow ◦ TensorFlow on Android by Yufeng Guo 29
Android Inference library and Java API to build.gradle dependencies { compile 'org.tensorflow:tensorflow-android:1.2.0-preview' } Link to talk: Android meets TensorFlow 31 +
• Install and run the TensorFlow Docker image • Retrieve the images (flowers: daisy, dandelion, roses, sunflowers, & tulips) • Retrain the final layer of Inception 3 • Use the retrained model to classify an image https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0 32
as the user interface! Train Alexa by building new skills • Create your own echo with Alexa Voice Service on Raspberry Pi, Linux, Mac & Windows. • Access Alexa Voice Service from Android 34
Pepper’s Intelligence is powered by IBM Watson 1. Download Android plugin & SDK 2. Create an Android project 3. Enable robot project structure Now you are ready to program robots! https://android.aldebaran.com/doc/introduction.html 35
app really need the intelligence? • Choose a API/service ◦ ease of use, support, pricing • Get user permissions • Inform user about data privacy • Be mindful with user's data 38
machine learning APIs • Try out TensorFlow tutorials and code labs • Check out the AI experiments • Study machine learning basics • Build an app with intelligence 39
I/O 2016 - • Getting Started with Machine Perception Using the Mobile Vision API • Machine Learning APIs by Example • Effective TensorFlow for Non-Experts • Android Meets TensorFlow: How to Accelerate Your App with AI Google AI Experiments • https://aiexperiments.withgoogle.com/ Udacity blog • The What, How, and Why of Artificial Intelligence, Machine Learning, and Self-Driving Cars 41
learning resources My blog - Get Started with Machine Learning Udacity • Intro to Machine Learning • Deep Learning Nanodegree (TensorFlow, intermediate) • Deep Learning (TensorFlow, advanced) • AI Engineer Nanodegree • ML Engineer Nanodegree Stanford • Andrew Ng’s ML course on Coursera Google Developers • Machine Learning Recipes (short Youtube videos) 42
margaretmz
mokocm
tetsuyaooooo
ebarakazuhiro
yu4u
kenjikondobai
qsona
layerx
oracle4engineer
rinchsan
tsukasa_ishimaru
matsuihidetoshi
wmsbill
chrisshort
thekraken
andyhume
shpigford
productmarketing
destraynor
gr2m
bermonpainter
chriscoyier
keithpitt
morganepeng
phodgson
