Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Democratize Deep Learning Models

Democratize Deep Learning Models

A perspective on how to make deep learning models more accessible and usable for a wider spectrum of people. Using tensorflow.js

1. Educate: Explorable Explanations
2. Create: Rapid Prototyping
3. Visualise: Model Visualisation
4. Intervene: Decision Making
5. Imagine: Generative Spaces


Amit Kapoor

June 30, 2020


  1. Democratize Deep Learning Models Using TensorFlow.js Perspective & Views Amit

    Kapoor Bargava Subramanian 1
  2. Amit Kapoor Data | Visual | Story — Teaching Analytics

    & Vis — Consulting & Mentoring — Building DataVis Tools amitkaps.com 2
  3. Bargava Subramanian Data | AI & ML | Productivity —

    Startup @Binaize — Data Scientist (ex @Cisco, @RedHat) — Consulting on ML products bargava.com 3
  4. Deep Learning Challenges — Scale Up: Faster data & models

    — Scale Out: Bigger data & models — Scale Wide: Entire data & model pipelines — Scale Across: Beyond data scientists & engineers How do we get more Human in the Loop approach to using Deep Learning? 4
  5. Democratize Deep Learning (DL) 1. Educate: Why will DL work

    for our use case? 2. Create: What can we do to build our DL model? 3. Visualise: So how do we improve our DL models? 4. Intervene: So what actions can we take using DL? 5. Imagine: So what else can we use DL for? 5
  6. Perspective on Democratize DL 1. Educate: Explorable Explanations 2. Create:

    Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 6
  7. Structure for Each Perspective — Conceptual: Meaning & View —

    In Real Life: Context & challenge — Our Approach: Our Solutions & Experiments — Exemplars: Open-source Examples & Libraries — Learnings: Lessons learned 7
  8. Lens – Tensorflow.js — Javascript version of Tensorflow Deep Learning

    Library — Core (Linear Algebra low level API) — Layers (Keras-type high level API) — Runs both on Client-side (JS, WebAssembly, WebGL) and on Server-side (Nodejs with C binary for CPU & GPU) — Can be used for both inference and training — Custom models optimised for Image, Text, Audio etc. — Can Import & use models build in Python version 8
  9. 1. Educate Explorable Explanations “For the things we have to

    learn before we can do them, we learn by doing them.” ― Aristotle 9
  10. Neural Network Playground1 1 Tinker with Neural Network 10

  11. In Real Life: Context Client: Leading US Network Service Provider

    Scope: Threat detection using User & Entity Behaviour Analytics Status: Unsustainable Rule Based Policies Mandate: Evaluate shift to ML/DL Models Challenge: Domain experts skeptical of ML/DL paradigm 11
  12. How to Educate? — Work with sample or open data

    — Show the potential for ML/DL models — Need to showcase visually explorable insights — Need for interactive exploration across stakeholders (CISO, CTO) 12
  13. Explorable Explanations — Showcased multiple models: K-Means, One Class SVM,

    Autoencoders — Created models in Python & exported to Tensorflow.js — Integrated with an interactive dashboard (using AngularJS) — Built-in exploration capability on tweaking (and re- running) models Eventual API & implementation based on K-Means model 13
  14. Exemplar: Blip2 2 Anomaly Detection Dashboard by Fast Forward Labs

  15. Exemplar: Distill.pub3 3 How Momentum Works by Distill.pub 15

  16. Learning on Educate — Discovery by Active Learning is powerful

    — Allows the user to build intuition on DL — Not easy! Requires deep thinking on design and usability 16
  17. 2. Create Rapid Prototyping "The secret of getting started is

    breaking your complex overwhelming tasks into smaller manageable tasks, and then starting on the first one" - Mark Twain 17
  18. Transfer Learning on Images4 4 TensorSpace framework 18

  19. In Real Life: Context Client: Boutique US Software Consulting shop

    Scope: Computer vision use-cases for an Automotive client Status: Full stack web & mobile expertise Mandate: Build & extend capabilities for ML&DL Challenge: No experience in python data stack 19
  20. What to Prototype — Working with sample data — Needed

    to build an image classification model — Show how to integrate that model as API — Create an interactive dashboard to demo to client 20
  21. Create Rapid Prototypes — Built CNN models for tra ic

    sign detection — Built models in tensorflow.js using pre-built models (transfer learning) — Deployed model as API using Node — Built a simple single-page application for user to upload image and get predictions 21
  22. Exemplar: tfjs in Observablehq5 5 Collection: Deep Learning in the

    Browser 22
  23. Learning on Create — Enabling software developers to build ML

    unlocks huge potential — Easy to prototype without adding technical debt and additional technology costs (eg: Python data stack) — Not easy! How to build custom models. 23
  24. 3. Visualise Model Visualisation “The most powerful way to gain

    insight into a system is by moving between levels of abstraction.” — Bret Victor 24
  25. 25

  26. Model Visualisation [0] Visualise the data space [1] Visualise the

    predictions in the data space [2] Visualise the errors in model fitting [3] Visualise with di erent model parameters [4] Visualise with di erent input datasets [5] Visualise the entire model space [6] Visualise the entire feature space [7] Visualise the many models together 26
  27. ModelVis & ML/DL [0] DATA VIS: the data space [1]

    PREDICTION: the predictions in the data space [2] VALIDATION: the errors in model fitting [3] TUNING: with di erent model parameters [4] BOOTSTRAP: with di erent input datasets [5] ENSEMBLE: the entire model space [6] FEATURE ENGG: the entire feature space [7] N-MODELS: the many models together 27
  28. ModelVis Key Concept Use visualisation to aid the transition of

    implicit knowledge about the data to explicit knowledge in the model. — Iterative, not linear — Up and Down, not lateral — Complementary, not exclusive 28
  29. In Real Life: Context Client: Large Consumer Product Company in

    India Scope: Network optimisation of secondary freight Status: Regional & fragmented network Mandate: Evaluate consolidation opportunities Challenge: Limited data on new options, Implicit domain knowledge 29
  30. 30

  31. Applying Model Visualisation — Engage with Supply Chain domain experts

    with limited ML understanding — Need to represent models in the same data visualisation space — Need to showcase visually explorable model options — Need to build interactive exploration for di erent stakeholders (CSO, Managers) 31
  32. 32

  33. Exemplar: Manifold from Uber6 6 Model-Agnostic Visual Debugging tool 33

  34. Exemplar: Feature Visualisation7 7 The Building Blocks of Interpretability 34

  35. Learning on Model Vis — Model improvements require engaging with

    domain through their lens — Visual analytics principles to model visualisation — Decouple model vis from model building compute & pipelines — Multivariate visualisation is more challenging than larger data! 35
  36. 4. Intervene Decision Making "All models are wrong, but some

    are useful" — George Box 36
  37. Using Models for Decisions8 8 Fairness in Machine Learning 37

  38. Context & Use Case Client: Small Regional Movies OTT App

    in India Scope: Map movie ROI & acquisition focus Status: Popularity & heuristic-based decisions Mandate: Build RecSys for engagement & licensing Challenge: Limited user insight, no data team 38
  39. Challenges — User and movie personas were hard-coded — No

    real-time user insight to drive recommendations — Hard to link user behaviour to movie acquisition — Tool should work for business (content acquisition team), not only technology team 39
  40. Decision Making Tool — Built multiple RecSys models to drive

    app recommendations — Interactive dashboard of user & movie embeddings — Created "What-If" scenario explorer - to link changing user behaviour with movie acquisition Shifted from Heuristic-based to Data-driven decision 40
  41. Exemplar: RecSys & Business9 9 Understanding Latent Factors by StitchFix

  42. Exemplar: CounterFactuals10 10 What-If tool by Google Pair 42

  43. 5. Imagine Generative Spaces "I don't know, what I don't

    know" — The second level of ignorance 43
  44. Font Design Space11 11 Using Artificial Intelligence to Augment Human

    Intelligence 44
  45. Generative Spaces — Moving beyond Supervised learning paradigms — Using

    generative models to invent meaningful creative operations — Designing and exploring latent spaces (with constraints) — Applications across design, product development, service design 45
  46. Examples & Experiments using tfjs — ML for the web:

    ml5.js — Music & Art: magenta.js — Machine Learning for Artists: ml4a — AI Google Experiments: AI Collections 46
  47. Democratize Deep Learning (DL) 1. Educate: Explorable Explanations 2. Create:

    Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 47
  48. Access this presentation at: https://speakerdeck.com/amitkaps Amit Kapoor amitkaps.com Bargava Subramanian

    bargava.com 48