Pragmatic Machine Learning

By now, we've all heard that the industry is moving towards an "AI first" vision where machine learning powered features like computer vision and natural language processing, drive innovations like autonomous driving, speech & object recognition, smart assistants and more.

But what does this mean for developers? Do we need PhD-level expertise in data science? What tools & technologies should we master? Where is it useful and why should we care? In this talk, we'll look at machine learning through a pragmatic lens - learning what you need to get the current job done, then exploring paths you can follow to evolve those solutions as your expertise and requirements grow.

Transcript

1. #PragmaticML #DevFestFL @nitya Nov 2017 Pragmatic Machine Learning Nitya Narasimhan | @nitya DevFest Florida | Nov 12, 2017 Image Credit: Unsplash / @liane #PragmaticML #DevFestFL @nitya http://bit.ly/pragmaticML-nov2017
2. #PragmaticML #DevFestFL @nitya Nov 2017 #PragmaticML @nitya #DevFestFL
3. #PragmaticML #DevFestFL @nitya Nov 2017 About Me I am a PhD, an innovator, an engineer with >20+ yrs of software & systems development experience, a community manager & a mom. I’m passionate about inclusive communities and workforce retraining. I believe that the only certainity in tech is change.
4. #PragmaticML #DevFestFL @nitya Nov 2017 1. As a beginner, what is machine learning & why does it matter? 2. As a developer, how do I use it to solve my problems? 3. As a human, what are challenges I will need to address in ML/AI? Goals
5. #PragmaticML #DevFestFL @nitya Nov 2017 AS A BEGINNER WHAT IS MACHINE LEARNING?
6. #PragmaticML #DevFestFL @nitya Nov 2017 turning data into information SCIENTIST // Machine Learning Is Image Credit: Wikimedia Commons It’s a fish!
7. #PragmaticML #DevFestFL @nitya Nov 2017 using data to answer questions Image Credit: Wikimedia Commons What is this? WHAT’S MY NEXT MOVE? It’s a fish! Move A7 to A5 DEVELOPER // Machine Learning Is
8. #PragmaticML #DevFestFL @nitya Nov 2017 Teach concepts using examples Validate learning by testing Assign tasks leveraging knowledge This is an apple What is this? Is it an apple? GET an apple from the bowl for your lunch. PARENT // Machine Learning Is
9. #PragmaticML #DevFestFL @nitya Nov 2017 Learn concepts “Bread” “Jam” “Cookie” Detect patterns Transfer learning “Bread+jam makes a sandwich” “Oh I see! Jam acts as glue” “Mom look - I used jam to make a cookie sandwich” THE MORE CONCEPTS I KNOW, THE MORE CONNECTIONS I MAKE Deep Learning = Finding Complex Patterns in multi-dimensional data
10. #PragmaticML #DevFestFL @nitya Nov 2017 RETAIL (Pinterest Lens) Medium -- 2017 “Lens enables you to go beyond traditional uses of your Phone’s camera–taking selfies or saving a scene– and turns it into a powerful discovery system.” Deep Learning = For Discovery
11. #PragmaticML #DevFestFL @nitya Nov 2017 HEALTHCARE (Detection / Care) Google Research - Mar 2017 | Motley Fool -- Mar 2017 “IBM Watson is now addressing a variety of other medical areas including personalized care, patient engagement, imaging review, and drug discovery” “the localization score (FROC) for the algorithm reached 89%, which significantly exceeded the score of 73% for a pathologist with no time constraint” - Google Research Deep Learning = For Diagnostics
12. #PragmaticML #DevFestFL @nitya Nov 2017 BANKING (Finie “Bot”) The Financial Brand -- January 2017 “32% of financial services executives surveyed confirmed using AI technologies such as predictive analytics, recommendation engines, voice recognition and response. With an origin rooted in risk and fraud detection and cost reduction, AI is increasingly important for financial services firms to be competitive” Deep Learning = For Anomaly Detection
13. #PragmaticML #DevFestFL @nitya Nov 2017 “Snapchat designed geofilters, which are essentially frames users can apply to their pictures, that highlight landmarks on the race course, giving runners a unique way to share their experience.” Prisma Labs is giving its users the ability to make their own neural network-trained filters based on their favorite art or artistic styles. Deep Learning = For Entertainment
14. #PragmaticML #DevFestFL @nitya Nov 2017 AS A BEGINNER WHY DOES IT MATTER?
15. #PragmaticML #DevFestFL @nitya Nov 2017 How to secure your job in The age of automation Inc -- Jan 2017 Automation, bots, and software are getting to the point now where they are becoming more capable of taking over knowledge jobs, whereas before they were only taking over labor intensive jobs such as manufacturing
16. #PragmaticML #DevFestFL @nitya Nov 2017 Google Trends / Captured Nov 2017 Trends reflect this awareness.. What happened in 2015?
17. #PragmaticML #DevFestFL @nitya Nov 2017 The Machine Learning Times of 2015: A Powerful Growth Story (Analytics Vidhya, Nov 2015)
18. #PragmaticML #DevFestFL @nitya Nov 2017 The Machine Learning Times of 2015: A Powerful Growth Story (Analytics Vidhya, Nov 2015)
19. #PragmaticML #DevFestFL @nitya Nov 2017 DATA ubiquity & access DEVELOPER advocacy & training OPEN SOURCE tools & platforms CLOUD scale processing The Perfect Blend Commoditization of ML Democratization of AI
20. #PragmaticML #DevFestFL @nitya Nov 2017 As technology reaches more people, its impact becomes more profound. The next step for AI must be democratization -- lowering barriers (to availability) -Dr. Fei-Fei Li “DEMOCRATIZING AI” Google Cloud Next 17 Keynote -- Mar 2017
21. #PragmaticML #DevFestFL @nitya Nov 2017 AS A DEVELOPER HoW DO I USE IT TO SOLVE PROBLEMS?
22. #PragmaticML #DevFestFL @nitya Nov 2017 prag.mat.ic dealing with things sensibly & realistically in a way based on practical considerations rather than theoretical ones.
23. #PragmaticML #DevFestFL @nitya Nov 2017 sensibly “scope” Is machine learning the right solution for my current problem? What is my expertise level? Can I get initial wins as I gain expertise realistically “leverage - adapt - make”
24. #PragmaticML #DevFestFL @nitya Nov 2017 Think “Evolve” Pragmatic ML S.L.A.M Dunk Scope focus on elimination THINK “EXIT” LEVERAGE focus on straightforward reuse ADAPT focus on customization MAKE build new
25. #PragmaticML #DevFestFL @nitya Nov 2017 Is Machine Learning the right solution for THIS task?
26. #PragmaticML #DevFestFL @nitya Nov 2017 traditional programming machine learning is useful if the data/task is finite / deterministic is useful if the data/task is large / indeterminate ask Can my problem be solved with ML?
27. #PragmaticML #DevFestFL @nitya Nov 2017 learning types ideal starting points to build solutions for those tasks ask Will my problem be solved with existing ML?
28. #PragmaticML #DevFestFL @nitya Nov 2017 Exploit efficiencies by reducing resource use, costs, errors Automation Exploit scale & speed boosts from computer optimizations Performance benefits To applying machine learning Transfer learning to other domains Identify new connections or patterns Creativity ask Should my problem be solved with ML? ELSE “EXIT”
29. #PragmaticML #DevFestFL @nitya Nov 2017 real examples of relevant machine learning problems Anomaly / Unsupervised Fraud Detection Regression / Supervised Weather Forecasting Real-Time / Reinforcement Robot Navigation Gameplay / Reinforcement Strategic Actions Dimensionality Reduction / Unsupervised Big Data Understanding Clustering / Unsupervised Recommender / Targeting Vision / (Un)supervised Image Classification
30. #PragmaticML #DevFestFL @nitya Nov 2017 Has someone solved my problem? Can I quickly “reuse” learning models with just coding skills?
31. #PragmaticML #DevFestFL @nitya Nov 2017 cloud machine learning BUILT / ON BIG DATA / FOR SCALE / INTERACTIVE Google Cloud Microsoft Azure Amazon ML IBM Watson NYC Startup
32. #PragmaticML #DevFestFL @nitya Nov 2017 image analysis text analysis speech recognition language translation video analysis Cloud NLP API Cloud Speech API Cloud Translate API Cloud Vision API Video Intelligence API API Give data Get information Familiar “API” programming constructs. Just bring your data.
33. Faces Faces, facial landmarks, emotions OCR Read and extract text, with support for > 10 languages Label Detect entities from furniture to transportation Logos Identify product logos Landmarks & Image Properties Detect landmarks & dominant color of image Safe Search Detect explicit content - adult, violent, medical and spoof Cloud Vision API DEMO
34. API Usage: Detect Objects in an Image Image Detected Items Vision API Create JSON request with the image or pointer to an image Process the JSON response Call the REST API 1 2 3
35. Confidential & Proprietary Google Cloud Platform 35 Cloud Natural Language API Extract sentence, identify parts of speech and create dependency parse trees for each sentence. Identify entities and label by types such as person, organization, location, events, products and media. Understand the overall sentiment of a block of text. Syntax Analysis Entity Recognition Sentiment Analysis DEMO
36. Confidential & Proprietary Google Cloud Platform 36 Cloud Speech API Automatic Speech Recognition (ASR) powered by deep learning neural networking to power your applications like voice search or speech transcription. Recognizes over 80 languages and variants with an extensive vocabulary. Returns partial recognition results immediately, as they become available. Filter inappropriate content in text results. Audio input can be captured by an application’s microphone or sent from a pre-recorded audio file. Multiple audio file formats are supported, including FLAC, AMR, PCMU and linear-16. Handles noisy audio from many environments without requiring additional noise cancellation. Audio files can be uploaded in the request and, in future releases, integrated with Google Cloud Storage. Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering Streaming Recognition Real-time or Buffered Audio Support Noisy Audio Handling Integrated API DEMO
37. Face API faces, facial landmarks, eyes open, smiling Barcode API 1D and 2D barcodes Text API Latin-based text / structure Common Mobile Vision API Support for fast image and video on-device detection and tracking. REAL-TIME + ON-DEVICE!
38. Googly Eyes Android App Video credit Google 1. Create a face detector for facial landmarks (e.g., eyes) 3. For each face, draw the eyes FaceDetector detector = new FaceDetector.Builder() .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build(); SparseArray<Face> faces = detector.detect(image); for (int i = 0; i < faces.size(); ++i) { Face face = faces.valueAt(i); for (Landmark landmark : face.getLandmarks()) { // Draw eyes 2. Detect faces in the image
39. Easy to use Java API image detected items Detector 1. Create a detector object 2. detectedItems = detector.detect(image) Photo credit developers.google.com/vision
40. Text Detection & Structure Blocks Lines Words Lines Words Words Words Video and image credit Google
41. Barcode Detection 1D barcodes EAN-13/8 UPC-A/E Code-39/93/128 ITF Codabar 2D barcodes QR Code Data Matrix PDF-417 AZTEC UPC DataMatrix QR Code PDF 417 Video and image credit Google
42. Combined Vision & Translation Video and image credit Google
43. #PragmaticML #DevFestFL @nitya Nov 2017 Can I “retrain” their models with my data to get better accuracy or customized information?
44. #PragmaticML #DevFestFL @nitya Nov 2017 #FirebaseCamp NOW YOU NEED TO LOOK UNDER THE HOOD
45. #PragmaticML #DevFestFL @nitya Nov 2017 MACHINE LEARNING 101 TRAINING DATA TEST DATA MODEL Learning Algorithm (function) PREDICTION Parameters Valid inputs & outputs Test Model Accuracy Use with “new” data (problem) Refine if prediction accuracy can be estimated .. Tune parameters for best “fit” to data
46. #PragmaticML #DevFestFL @nitya Nov 2017 THE ML DATA FLOW .. GATHER & PREPARE DATA VISUALIZe & ANALYZE DATA SELECT & INIT MODEL (ALGO) TRAIN MODEL (TUNING) TEST MODEL (ACCURACY) USE MODEL TO MAKE DECISIONS Iterate (tune parameters)
47. #PragmaticML #DevFestFL @nitya Nov 2017 SELECTING MODEL IS KEY Supervised learning Unsupervised learning Reinforcement learning TASK driven DATA driven REWARD driven Labeled Data Classification / discrete Regression / continuous Unlabeled Data Association Analysis / discrete Clustering / continuous Dimensionality Reduction / continuous Real-Time Data Agent Systems
48. #PragmaticML #DevFestFL @nitya Nov 2017 ML DATA FLOW: TOOLS EDITION scipy scikit-learn Pandas, numpy, matplotlib scipy, scikit-learn scipy, scikit-learn scikit-learn matplotlib Data Viz (d3) Data Usage (API) Iterate (tune parameters) Model Selection Data Viz & Analysis Model Training Model Validation
49. #PragmaticML #DevFestFL @nitya Nov 2017 BASIC MACHINE LEARNING QUICK LOOK AT SCIKIT-LEARN
50. #PragmaticML #DevFestFL @nitya Nov 2017 “Iris Data Set” Supervised Learning / Classification setosa virginica versicolor Image Credit: Wikipedia Sepal length Sepal width Petal length Petal width Species 5.0 2.0 3.5 1.0 I. versicolor 6.0 2.2 5.0 1.5 I. virginica 4.5 2.3 1.3 0.3 I. setosa 5.0 2.3 3.3 1.0 I. versicolor Can I predict the “type” of an Iris from examples - given features / labels
51. from sklearn.datasets import load_iris iris_dataset = load_iris() // split into training and test data (randomize, then 80/20 or 70/30) from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split( iris_dataset[‘data’], iris_dataset[‘target], random_state=0) // visualize the data import pandas as pd iris_dataframe = pd.DataFrame(X_train, columns=iris_dataset.feature_names) grr = pd.scatter_matrix(iris_dataframe, c=y_train, figsize=(15,15), marker=’o’, …) Check scikit-learn Site 1-2 Gather & Prep Data Credit: sckit-learn // Andreas Mueller Intro-to-ML-With-Python
52. Check scikit-learn Site 3-4 Visualize Data & Select Model Image Credit: Wikipedia Can the different classes be identified from the data? What’s a good learning funtion that models this relationship?
53. from sklearn.neighbors import KNeighborsClassifier knn = KNeighborsClassifier(n_neighbors=1) // train the classifier // returns the knn object “trained” to fit your data/labels knn.fit(X_train, y_train) // test the classifier // predict label for test data, then evaluate accuracy (e.g., 0.97) prediction = knn.predict(X_test) accuracy = knn.score(X_test, y_test) // use the classifier to predict labels for “new” data prediction = knn.predict(X_new) Check scikit-learn Site 5-6-7 Train, Test & Use Classifier Credit: sckit-learn // Andreas Mueller Intro-to-ML-With-Python
54. #PragmaticML #DevFestFL @nitya Nov 2017 DEEP LEARNING IDEAL FOR FINDING PATTERNS IN HIGH-DIMENSIONAL DATA
56. • Open source Machine Learning library • Especially useful for Deep Learning • For research and production • Apache 2.0 license
57. Google Play Music Recommender Google Photos Smart Search Google Gmail Smart Reply
58. Hello World Image from https://github.com/mnielsen/neural-networks-and-deep-learning ?
59. What we see What the computer “sees”
60. import tensorflow as tf mnist = tf.contrib.learn.datasets.load_dataset('mnist') classifier = tf.learn.LinearClassifier(n_classes=10) classifier.fit(mnist.train.images, mnist.train.labels) score = metrics.accuracy_score(mnist.test.labels, \ classifier.predict(mnist.test.images)) print('Accuracy: {0:f}'.format(score)) How it learns Check TensorFlow Site
61. #PragmaticML #DevFestFL @nitya Nov 2017 Pre-trained or Pre-existing models don’t fit my needs. I need to build one.
62. First, explore ecosystem & build intuition
63. #PragmaticML #DevFestFL @nitya Nov 2017 Intuition is key! Parameter tuning, Feature engineering, Model selection more an art than a science
64. #PragmaticML #DevFestFL @nitya Nov 2017 AS A HUMAN WHAT ARE THE CHALLENGES I FACE OR NEED TO ADDRESS?
65. #PragmaticML #DevFestFL @nitya Nov 2017 Two prominent research-image collections—including one supported by Microsoft and Facebook—display a predictable gender bias in their depiction of activities such as cooking and sports. Images of shopping and washing are linked to women, for example, while coaching and shooting are tied to men. Machine-learning software trained on the datasets didn’t just mirror those biases , it amplified them BIAS
66. #PragmaticML #DevFestFL @nitya Nov 2017 Users of the app have noticed that many of the face-altering filters "whitewash." Upset Snapchatters point to the flower crown filter and the "beautifying" filter, both of which seem to lighten skin and eyes and contour the face to make one's chin and nose appear smaller. BIAS
67. #PragmaticML #DevFestFL @nitya Nov 2017 ETHICS
68. #PragmaticML #DevFestFL @nitya Nov 2017 ETHICS
69. #PragmaticML #DevFestFL @nitya Nov 2017 Recap Does your problem lack a valid “learning” goal? Stop! Do you need better accuracy with existing model? Retrain! Has someone built models for this concept? Reuse! Don’t have a valid model for the concept? Create it! Be Ethical. Bias in ML/AI can cause irreversible damage Be Adversarial. Test your systems for unintended (or learned) malice Be Pragmatic. Know when to exit & when to evolve your learning process..
70. #PragmaticML #DevFestFL @nitya Nov 2017 Shoulders of giants.. Emojis from Emojipedia Art from Unsplash, Nitya Fonts from Google Fonts Icons from NounProject (Pro) Some content from Google Developers
71. Google Cloud ML Resources Codelabs codelabs.developers.google.com/codelabs/cloud-vision-intro/index.html codelabs.developers.google.com/codelabs/cloud-speech-intro/index.html codelabs.developers.google.com/codelabs/cloud-nl-intro/index.html For Developers cloud.google.com/vision/ cloud.google.com/speech/ cloud.google.com/natural-language/ cloud.google.com/translate/ cloud.google.com/video-intelligence Stack Overflow
72. Mobile Vision: Codelabs and Samples Googly Eyes Code Sample github.com/googlesamples/android-vision/tree/master/visionSamples/googly-eyes Codelabs codelabs.developers.google.com/codelabs/face-detection/ codelabs.developers.google.com/codelabs/mobile-vision-ocr/ Mobile Vision Developers developers.google.com/vision/ GitHub Code Samples github.com/googlesamples/android-vision Stack Overflow Find and ask questions under the android-vision tag.
73. #PragmaticML #DevFestFL @nitya Nov 2017 THANK YOU FOR LISTENING I am not a machine learning expert. But I am a curious coder & lifelong learner. I believe ALL our voices are needed in ML/AI to prevent bias, ensure ethical usage & create meaningful value for humans Pragmatic Machine Learning “http://bit.ly/pragmaticML-nov2017”