Pragmatic Machine Learning

5b392dc79d5b6d1dab580bf60e26fb7c?s=47 Nitya Narasimhan, PhD
November 12, 2017

Pragmatic Machine Learning

By now, we've all heard that the industry is moving towards an "AI first" vision where machine learning powered features like computer vision and natural language processing, drive innovations like autonomous driving, speech & object recognition, smart assistants and more.

But what does this mean for developers? Do we need PhD-level expertise in data science? What tools & technologies should we master? Where is it useful and why should we care? In this talk, we'll look at machine learning through a pragmatic lens - learning what you need to get the current job done, then exploring paths you can follow to evolve those solutions as your expertise and requirements grow.


Nitya Narasimhan, PhD

November 12, 2017


  1. #PragmaticML #DevFestFL @nitya Nov 2017 Pragmatic Machine Learning Nitya Narasimhan

    | @nitya DevFest Florida | Nov 12, 2017 Image Credit: Unsplash / @liane #PragmaticML #DevFestFL @nitya
  2. #PragmaticML #DevFestFL @nitya Nov 2017 #PragmaticML @nitya #DevFestFL

  3. #PragmaticML #DevFestFL @nitya Nov 2017 About Me I am a

    PhD, an innovator, an engineer with >20+ yrs of software & systems development experience, a community manager & a mom. I’m passionate about inclusive communities and workforce retraining. I believe that the only certainity in tech is change.
  4. #PragmaticML #DevFestFL @nitya Nov 2017 1. As a beginner, what

    is machine learning & why does it matter? 2. As a developer, how do I use it to solve my problems? 3. As a human, what are challenges I will need to address in ML/AI? Goals
  5. #PragmaticML #DevFestFL @nitya Nov 2017 AS A BEGINNER WHAT IS

  6. #PragmaticML #DevFestFL @nitya Nov 2017 turning data into information SCIENTIST

    // Machine Learning Is Image Credit: Wikimedia Commons It’s a fish!
  7. #PragmaticML #DevFestFL @nitya Nov 2017 using data to answer questions

    Image Credit: Wikimedia Commons What is this? WHAT’S MY NEXT MOVE? It’s a fish! Move A7 to A5 DEVELOPER // Machine Learning Is
  8. #PragmaticML #DevFestFL @nitya Nov 2017 Teach concepts using examples Validate

    learning by testing Assign tasks leveraging knowledge This is an apple What is this? Is it an apple? GET an apple from the bowl for your lunch. PARENT // Machine Learning Is
  9. #PragmaticML #DevFestFL @nitya Nov 2017 Learn concepts “Bread” “Jam” “Cookie”

    Detect patterns Transfer learning “Bread+jam makes a sandwich” “Oh I see! Jam acts as glue” “Mom look - I used jam to make a cookie sandwich” THE MORE CONCEPTS I KNOW, THE MORE CONNECTIONS I MAKE Deep Learning = Finding Complex Patterns in multi-dimensional data
  10. #PragmaticML #DevFestFL @nitya Nov 2017 RETAIL (Pinterest Lens) Medium --

    2017 “Lens enables you to go beyond traditional uses of your Phone’s camera–taking selfies or saving a scene– and turns it into a powerful discovery system.” Deep Learning = For Discovery
  11. #PragmaticML #DevFestFL @nitya Nov 2017 HEALTHCARE (Detection / Care) Google

    Research - Mar 2017 | Motley Fool -- Mar 2017 “IBM Watson is now addressing a variety of other medical areas including personalized care, patient engagement, imaging review, and drug discovery” “the localization score (FROC) for the algorithm reached 89%, which significantly exceeded the score of 73% for a pathologist with no time constraint” - Google Research Deep Learning = For Diagnostics
  12. #PragmaticML #DevFestFL @nitya Nov 2017 BANKING (Finie “Bot”) The Financial

    Brand -- January 2017 “32% of financial services executives surveyed confirmed using AI technologies such as predictive analytics, recommendation engines, voice recognition and response. With an origin rooted in risk and fraud detection and cost reduction, AI is increasingly important for financial services firms to be competitive” Deep Learning = For Anomaly Detection
  13. #PragmaticML #DevFestFL @nitya Nov 2017 “Snapchat designed geofilters, which are

    essentially frames users can apply to their pictures, that highlight landmarks on the race course, giving runners a unique way to share their experience.” Prisma Labs is giving its users the ability to make their own neural network-trained filters based on their favorite art or artistic styles. Deep Learning = For Entertainment
  14. #PragmaticML #DevFestFL @nitya Nov 2017 AS A BEGINNER WHY DOES

  15. #PragmaticML #DevFestFL @nitya Nov 2017 How to secure your job

    in The age of automation Inc -- Jan 2017 Automation, bots, and software are getting to the point now where they are becoming more capable of taking over knowledge jobs, whereas before they were only taking over labor intensive jobs such as manufacturing
  16. #PragmaticML #DevFestFL @nitya Nov 2017 Google Trends / Captured Nov

    2017 Trends reflect this awareness.. What happened in 2015?
  17. #PragmaticML #DevFestFL @nitya Nov 2017 The Machine Learning Times of

    2015: A Powerful Growth Story (Analytics Vidhya, Nov 2015)
  18. #PragmaticML #DevFestFL @nitya Nov 2017 The Machine Learning Times of

    2015: A Powerful Growth Story (Analytics Vidhya, Nov 2015)
  19. #PragmaticML #DevFestFL @nitya Nov 2017 DATA ubiquity & access DEVELOPER

    advocacy & training OPEN SOURCE tools & platforms CLOUD scale processing The Perfect Blend Commoditization of ML Democratization of AI
  20. #PragmaticML #DevFestFL @nitya Nov 2017 As technology reaches more people,

    its impact becomes more profound. The next step for AI must be democratization -- lowering barriers (to availability) -Dr. Fei-Fei Li “DEMOCRATIZING AI” Google Cloud Next 17 Keynote -- Mar 2017
  21. #PragmaticML #DevFestFL @nitya Nov 2017 AS A DEVELOPER HoW DO

  22. #PragmaticML #DevFestFL @nitya Nov 2017 prag.mat.ic dealing with things sensibly

    & realistically in a way based on practical considerations rather than theoretical ones.
  23. #PragmaticML #DevFestFL @nitya Nov 2017 sensibly “scope” Is machine learning

    the right solution for my current problem? What is my expertise level? Can I get initial wins as I gain expertise realistically “leverage - adapt - make”
  24. #PragmaticML #DevFestFL @nitya Nov 2017 Think “Evolve” Pragmatic ML S.L.A.M

    Dunk Scope focus on elimination THINK “EXIT” LEVERAGE focus on straightforward reuse ADAPT focus on customization MAKE build new
  25. #PragmaticML #DevFestFL @nitya Nov 2017 Is Machine Learning the right

    solution for THIS task?
  26. #PragmaticML #DevFestFL @nitya Nov 2017 traditional programming machine learning is

    useful if the data/task is finite / deterministic is useful if the data/task is large / indeterminate ask Can my problem be solved with ML?
  27. #PragmaticML #DevFestFL @nitya Nov 2017 learning types ideal starting points

    to build solutions for those tasks ask Will my problem be solved with existing ML?
  28. #PragmaticML #DevFestFL @nitya Nov 2017 Exploit efficiencies by reducing resource

    use, costs, errors Automation Exploit scale & speed boosts from computer optimizations Performance benefits To applying machine learning Transfer learning to other domains Identify new connections or patterns Creativity ask Should my problem be solved with ML? ELSE “EXIT”
  29. #PragmaticML #DevFestFL @nitya Nov 2017 real examples of relevant machine

    learning problems Anomaly / Unsupervised Fraud Detection Regression / Supervised Weather Forecasting Real-Time / Reinforcement Robot Navigation Gameplay / Reinforcement Strategic Actions Dimensionality Reduction / Unsupervised Big Data Understanding Clustering / Unsupervised Recommender / Targeting Vision / (Un)supervised Image Classification
  30. #PragmaticML #DevFestFL @nitya Nov 2017 Has someone solved my problem?

    Can I quickly “reuse” learning models with just coding skills?
  31. #PragmaticML #DevFestFL @nitya Nov 2017 cloud machine learning BUILT /

    ON BIG DATA / FOR SCALE / INTERACTIVE Google Cloud Microsoft Azure Amazon ML IBM Watson NYC Startup
  32. #PragmaticML #DevFestFL @nitya Nov 2017 image analysis text analysis speech

    recognition language translation video analysis Cloud NLP API Cloud Speech API Cloud Translate API Cloud Vision API Video Intelligence API API Give data Get information Familiar “API” programming constructs. Just bring your data.
  33. Faces Faces, facial landmarks, emotions OCR Read and extract text,

    with support for > 10 languages Label Detect entities from furniture to transportation Logos Identify product logos Landmarks & Image Properties Detect landmarks & dominant color of image Safe Search Detect explicit content - adult, violent, medical and spoof Cloud Vision API DEMO
  34. API Usage: Detect Objects in an Image Image Detected Items

    Vision API Create JSON request with the image or pointer to an image Process the JSON response Call the REST API 1 2 3
  35. Confidential & Proprietary Google Cloud Platform 35 Cloud Natural Language

    API Extract sentence, identify parts of speech and create dependency parse trees for each sentence. Identify entities and label by types such as person, organization, location, events, products and media. Understand the overall sentiment of a block of text. Syntax Analysis Entity Recognition Sentiment Analysis DEMO
  36. Confidential & Proprietary Google Cloud Platform 36 Cloud Speech API

    Automatic Speech Recognition (ASR) powered by deep learning neural networking to power your applications like voice search or speech transcription. Recognizes over 80 languages and variants with an extensive vocabulary. Returns partial recognition results immediately, as they become available. Filter inappropriate content in text results. Audio input can be captured by an application’s microphone or sent from a pre-recorded audio file. Multiple audio file formats are supported, including FLAC, AMR, PCMU and linear-16. Handles noisy audio from many environments without requiring additional noise cancellation. Audio files can be uploaded in the request and, in future releases, integrated with Google Cloud Storage. Automatic Speech Recognition Global Vocabulary Inappropriate Content Filtering Streaming Recognition Real-time or Buffered Audio Support Noisy Audio Handling Integrated API DEMO
  37. Face API faces, facial landmarks, eyes open, smiling Barcode API

    1D and 2D barcodes Text API Latin-based text / structure Common Mobile Vision API Support for fast image and video on-device detection and tracking. REAL-TIME + ON-DEVICE!
  38. Googly Eyes Android App Video credit Google 1. Create a

    face detector for facial landmarks (e.g., eyes) 3. For each face, draw the eyes FaceDetector detector = new FaceDetector.Builder() .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build(); SparseArray<Face> faces = detector.detect(image); for (int i = 0; i < faces.size(); ++i) { Face face = faces.valueAt(i); for (Landmark landmark : face.getLandmarks()) { // Draw eyes 2. Detect faces in the image
  39. Easy to use Java API image detected items Detector 1.

    Create a detector object 2. detectedItems = detector.detect(image) Photo credit
  40. Text Detection & Structure Blocks Lines Words Lines Words Words

    Words Video and image credit Google
  41. Barcode Detection 1D barcodes EAN-13/8 UPC-A/E Code-39/93/128 ITF Codabar 2D

    barcodes QR Code Data Matrix PDF-417 AZTEC UPC DataMatrix QR Code PDF 417 Video and image credit Google
  42. Combined Vision & Translation Video and image credit Google

  43. #PragmaticML #DevFestFL @nitya Nov 2017 Can I “retrain” their models

    with my data to get better accuracy or customized information?
  44. #PragmaticML #DevFestFL @nitya Nov 2017 #FirebaseCamp NOW YOU NEED TO

  45. #PragmaticML #DevFestFL @nitya Nov 2017 MACHINE LEARNING 101 TRAINING DATA

    TEST DATA MODEL Learning Algorithm (function) PREDICTION Parameters Valid inputs & outputs Test Model Accuracy Use with “new” data (problem) Refine if prediction accuracy can be estimated .. Tune parameters for best “fit” to data
  46. #PragmaticML #DevFestFL @nitya Nov 2017 THE ML DATA FLOW ..

  47. #PragmaticML #DevFestFL @nitya Nov 2017 SELECTING MODEL IS KEY Supervised

    learning Unsupervised learning Reinforcement learning TASK driven DATA driven REWARD driven Labeled Data Classification / discrete Regression / continuous Unlabeled Data Association Analysis / discrete Clustering / continuous Dimensionality Reduction / continuous Real-Time Data Agent Systems
  48. #PragmaticML #DevFestFL @nitya Nov 2017 ML DATA FLOW: TOOLS EDITION

    scipy scikit-learn Pandas, numpy, matplotlib scipy, scikit-learn scipy, scikit-learn scikit-learn matplotlib Data Viz (d3) Data Usage (API) Iterate (tune parameters) Model Selection Data Viz & Analysis Model Training Model Validation
  49. #PragmaticML #DevFestFL @nitya Nov 2017 BASIC MACHINE LEARNING QUICK LOOK

  50. #PragmaticML #DevFestFL @nitya Nov 2017 “Iris Data Set” Supervised Learning

    / Classification setosa virginica versicolor Image Credit: Wikipedia Sepal length Sepal width Petal length Petal width Species 5.0 2.0 3.5 1.0 I. versicolor 6.0 2.2 5.0 1.5 I. virginica 4.5 2.3 1.3 0.3 I. setosa 5.0 2.3 3.3 1.0 I. versicolor Can I predict the “type” of an Iris from examples - given features / labels
  51. from sklearn.datasets import load_iris iris_dataset = load_iris() // split into

    training and test data (randomize, then 80/20 or 70/30) from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split( iris_dataset[‘data’], iris_dataset[‘target], random_state=0) // visualize the data import pandas as pd iris_dataframe = pd.DataFrame(X_train, columns=iris_dataset.feature_names) grr = pd.scatter_matrix(iris_dataframe, c=y_train, figsize=(15,15), marker=’o’, …) Check scikit-learn Site 1-2 Gather & Prep Data Credit: sckit-learn // Andreas Mueller Intro-to-ML-With-Python
  52. Check scikit-learn Site 3-4 Visualize Data & Select Model Image

    Credit: Wikipedia Can the different classes be identified from the data? What’s a good learning funtion that models this relationship?
  53. from sklearn.neighbors import KNeighborsClassifier knn = KNeighborsClassifier(n_neighbors=1) // train the

    classifier // returns the knn object “trained” to fit your data/labels, y_train) // test the classifier // predict label for test data, then evaluate accuracy (e.g., 0.97) prediction = knn.predict(X_test) accuracy = knn.score(X_test, y_test) // use the classifier to predict labels for “new” data prediction = knn.predict(X_new) Check scikit-learn Site 5-6-7 Train, Test & Use Classifier Credit: sckit-learn // Andreas Mueller Intro-to-ML-With-Python
  54. #PragmaticML #DevFestFL @nitya Nov 2017 DEEP LEARNING IDEAL FOR FINDING

  55. None
  56. • Open source Machine Learning library • Especially useful for

    Deep Learning • For research and production • Apache 2.0 license
  57. Google Play Music Recommender Google Photos Smart Search Google Gmail

    Smart Reply
  58. Hello World Image from ?

  59. What we see What the computer “sees”

  60. import tensorflow as tf mnist = tf.contrib.learn.datasets.load_dataset('mnist') classifier = tf.learn.LinearClassifier(n_classes=10), mnist.train.labels) score = metrics.accuracy_score(mnist.test.labels, \ classifier.predict(mnist.test.images)) print('Accuracy: {0:f}'.format(score)) How it learns Check TensorFlow Site
  61. #PragmaticML #DevFestFL @nitya Nov 2017 Pre-trained or Pre-existing models don’t

    fit my needs. I need to build one.
  62. First, explore ecosystem & build intuition

  63. #PragmaticML #DevFestFL @nitya Nov 2017 Intuition is key! Parameter tuning,

    Feature engineering, Model selection more an art than a science
  64. #PragmaticML #DevFestFL @nitya Nov 2017 AS A HUMAN WHAT ARE

  65. #PragmaticML #DevFestFL @nitya Nov 2017 Two prominent research-image collections—including one

    supported by Microsoft and Facebook—display a predictable gender bias in their depiction of activities such as cooking and sports. Images of shopping and washing are linked to women, for example, while coaching and shooting are tied to men. Machine-learning software trained on the datasets didn’t just mirror those biases , it amplified them BIAS
  66. #PragmaticML #DevFestFL @nitya Nov 2017 Users of the app have

    noticed that many of the face-altering filters "whitewash." Upset Snapchatters point to the flower crown filter and the "beautifying" filter, both of which seem to lighten skin and eyes and contour the face to make one's chin and nose appear smaller. BIAS
  67. #PragmaticML #DevFestFL @nitya Nov 2017 ETHICS

  68. #PragmaticML #DevFestFL @nitya Nov 2017 ETHICS

  69. #PragmaticML #DevFestFL @nitya Nov 2017 Recap Does your problem lack

    a valid “learning” goal? Stop! Do you need better accuracy with existing model? Retrain! Has someone built models for this concept? Reuse! Don’t have a valid model for the concept? Create it! Be Ethical. Bias in ML/AI can cause irreversible damage Be Adversarial. Test your systems for unintended (or learned) malice Be Pragmatic. Know when to exit & when to evolve your learning process..
  70. #PragmaticML #DevFestFL @nitya Nov 2017 Shoulders of giants.. Emojis from

    Emojipedia Art from Unsplash, Nitya Fonts from Google Fonts Icons from NounProject (Pro) Some content from Google Developers
  71. Google Cloud ML Resources Codelabs For Developers Stack Overflow
  72. Mobile Vision: Codelabs and Samples Googly Eyes Code Sample

    Codelabs Mobile Vision Developers GitHub Code Samples Stack Overflow Find and ask questions under the android-vision tag.
  73. #PragmaticML #DevFestFL @nitya Nov 2017 THANK YOU FOR LISTENING I

    am not a machine learning expert. But I am a curious coder & lifelong learner. I believe ALL our voices are needed in ML/AI to prevent bias, ensure ethical usage & create meaningful value for humans Pragmatic Machine Learning “”