Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Democratize Deep Learning Models

Democratize Deep Learning Models

A perspective on how to make deep learning models more accessible and usable for a wider spectrum of people. Using tensorflow.js

1. Educate: Explorable Explanations
2. Create: Rapid Prototyping
3. Visualise: Model Visualisation
4. Intervene: Decision Making
5. Imagine: Generative Spaces

Amit Kapoor

June 30, 2020

More Decks by Amit Kapoor

Other Decks in Technology


  1. Amit Kapoor Data | Visual | Story — Teaching Analytics

    & Vis — Consulting & Mentoring — Building DataVis Tools amitkaps.com 2
  2. Bargava Subramanian Data | AI & ML | Productivity —

    Startup @Binaize — Data Scientist (ex @Cisco, @RedHat) — Consulting on ML products bargava.com 3
  3. Deep Learning Challenges — Scale Up: Faster data & models

    — Scale Out: Bigger data & models — Scale Wide: Entire data & model pipelines — Scale Across: Beyond data scientists & engineers How do we get more Human in the Loop approach to using Deep Learning? 4
  4. Democratize Deep Learning (DL) 1. Educate: Why will DL work

    for our use case? 2. Create: What can we do to build our DL model? 3. Visualise: So how do we improve our DL models? 4. Intervene: So what actions can we take using DL? 5. Imagine: So what else can we use DL for? 5
  5. Perspective on Democratize DL 1. Educate: Explorable Explanations 2. Create:

    Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 6
  6. Structure for Each Perspective — Conceptual: Meaning & View —

    In Real Life: Context & challenge — Our Approach: Our Solutions & Experiments — Exemplars: Open-source Examples & Libraries — Learnings: Lessons learned 7
  7. Lens – Tensorflow.js — Javascript version of Tensorflow Deep Learning

    Library — Core (Linear Algebra low level API) — Layers (Keras-type high level API) — Runs both on Client-side (JS, WebAssembly, WebGL) and on Server-side (Nodejs with C binary for CPU & GPU) — Can be used for both inference and training — Custom models optimised for Image, Text, Audio etc. — Can Import & use models build in Python version 8
  8. 1. Educate Explorable Explanations “For the things we have to

    learn before we can do them, we learn by doing them.” ― Aristotle 9
  9. In Real Life: Context Client: Leading US Network Service Provider

    Scope: Threat detection using User & Entity Behaviour Analytics Status: Unsustainable Rule Based Policies Mandate: Evaluate shift to ML/DL Models Challenge: Domain experts skeptical of ML/DL paradigm 11
  10. How to Educate? — Work with sample or open data

    — Show the potential for ML/DL models — Need to showcase visually explorable insights — Need for interactive exploration across stakeholders (CISO, CTO) 12
  11. Explorable Explanations — Showcased multiple models: K-Means, One Class SVM,

    Autoencoders — Created models in Python & exported to Tensorflow.js — Integrated with an interactive dashboard (using AngularJS) — Built-in exploration capability on tweaking (and re- running) models Eventual API & implementation based on K-Means model 13
  12. Learning on Educate — Discovery by Active Learning is powerful

    — Allows the user to build intuition on DL — Not easy! Requires deep thinking on design and usability 16
  13. 2. Create Rapid Prototyping "The secret of getting started is

    breaking your complex overwhelming tasks into smaller manageable tasks, and then starting on the first one" - Mark Twain 17
  14. In Real Life: Context Client: Boutique US Software Consulting shop

    Scope: Computer vision use-cases for an Automotive client Status: Full stack web & mobile expertise Mandate: Build & extend capabilities for ML&DL Challenge: No experience in python data stack 19
  15. What to Prototype — Working with sample data — Needed

    to build an image classification model — Show how to integrate that model as API — Create an interactive dashboard to demo to client 20
  16. Create Rapid Prototypes — Built CNN models for tra ic

    sign detection — Built models in tensorflow.js using pre-built models (transfer learning) — Deployed model as API using Node — Built a simple single-page application for user to upload image and get predictions 21
  17. Learning on Create — Enabling software developers to build ML

    unlocks huge potential — Easy to prototype without adding technical debt and additional technology costs (eg: Python data stack) — Not easy! How to build custom models. 23
  18. 3. Visualise Model Visualisation “The most powerful way to gain

    insight into a system is by moving between levels of abstraction.” — Bret Victor 24
  19. 25

  20. Model Visualisation [0] Visualise the data space [1] Visualise the

    predictions in the data space [2] Visualise the errors in model fitting [3] Visualise with di erent model parameters [4] Visualise with di erent input datasets [5] Visualise the entire model space [6] Visualise the entire feature space [7] Visualise the many models together 26
  21. ModelVis & ML/DL [0] DATA VIS: the data space [1]

    PREDICTION: the predictions in the data space [2] VALIDATION: the errors in model fitting [3] TUNING: with di erent model parameters [4] BOOTSTRAP: with di erent input datasets [5] ENSEMBLE: the entire model space [6] FEATURE ENGG: the entire feature space [7] N-MODELS: the many models together 27
  22. ModelVis Key Concept Use visualisation to aid the transition of

    implicit knowledge about the data to explicit knowledge in the model. — Iterative, not linear — Up and Down, not lateral — Complementary, not exclusive 28
  23. In Real Life: Context Client: Large Consumer Product Company in

    India Scope: Network optimisation of secondary freight Status: Regional & fragmented network Mandate: Evaluate consolidation opportunities Challenge: Limited data on new options, Implicit domain knowledge 29
  24. 30

  25. Applying Model Visualisation — Engage with Supply Chain domain experts

    with limited ML understanding — Need to represent models in the same data visualisation space — Need to showcase visually explorable model options — Need to build interactive exploration for di erent stakeholders (CSO, Managers) 31
  26. 32

  27. Learning on Model Vis — Model improvements require engaging with

    domain through their lens — Visual analytics principles to model visualisation — Decouple model vis from model building compute & pipelines — Multivariate visualisation is more challenging than larger data! 35
  28. Context & Use Case Client: Small Regional Movies OTT App

    in India Scope: Map movie ROI & acquisition focus Status: Popularity & heuristic-based decisions Mandate: Build RecSys for engagement & licensing Challenge: Limited user insight, no data team 38
  29. Challenges — User and movie personas were hard-coded — No

    real-time user insight to drive recommendations — Hard to link user behaviour to movie acquisition — Tool should work for business (content acquisition team), not only technology team 39
  30. Decision Making Tool — Built multiple RecSys models to drive

    app recommendations — Interactive dashboard of user & movie embeddings — Created "What-If" scenario explorer - to link changing user behaviour with movie acquisition Shifted from Heuristic-based to Data-driven decision 40
  31. 5. Imagine Generative Spaces "I don't know, what I don't

    know" — The second level of ignorance 43
  32. Generative Spaces — Moving beyond Supervised learning paradigms — Using

    generative models to invent meaningful creative operations — Designing and exploring latent spaces (with constraints) — Applications across design, product development, service design 45
  33. Examples & Experiments using tfjs — ML for the web:

    ml5.js — Music & Art: magenta.js — Machine Learning for Artists: ml4a — AI Google Experiments: AI Collections 46
  34. Democratize Deep Learning (DL) 1. Educate: Explorable Explanations 2. Create:

    Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 47