Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Democratize Deep Learning Models

Democratize Deep Learning Models

A perspective on how to make deep learning models more accessible and usable for a wider spectrum of people. Using tensorflow.js

1. Educate: Explorable Explanations
2. Create: Rapid Prototyping
3. Visualise: Model Visualisation
4. Intervene: Decision Making
5. Imagine: Generative Spaces

Amit Kapoor

June 30, 2020
Tweet

More Decks by Amit Kapoor

Other Decks in Technology

Transcript

  1. Democratize Deep Learning Models
    Using TensorFlow.js
    Perspective & Views
    Amit Kapoor
    Bargava Subramanian
    1

    View Slide

  2. Amit Kapoor
    Data | Visual | Story
    — Teaching Analytics & Vis
    — Consulting & Mentoring
    — Building DataVis Tools
    amitkaps.com
    2

    View Slide

  3. Bargava Subramanian
    Data | AI & ML | Productivity
    — Startup @Binaize
    — Data Scientist (ex @Cisco,
    @RedHat)
    — Consulting on ML products
    bargava.com
    3

    View Slide

  4. Deep Learning Challenges
    — Scale Up: Faster data & models
    — Scale Out: Bigger data & models
    — Scale Wide: Entire data & model pipelines
    — Scale Across: Beyond data scientists & engineers
    How do we get more Human in the Loop approach to
    using Deep Learning?
    4

    View Slide

  5. Democratize Deep Learning (DL)
    1. Educate: Why will DL work for our use case?
    2. Create: What can we do to build our DL model?
    3. Visualise: So how do we improve our DL models?
    4. Intervene: So what actions can we take using DL?
    5. Imagine: So what else can we use DL for?
    5

    View Slide

  6. Perspective on Democratize DL
    1. Educate: Explorable Explanations
    2. Create: Rapid Prototyping
    3. Visualise: Model Visualisation
    4. Intervene: Decision Making
    5. Imagine: Generative Spaces
    6

    View Slide

  7. Structure for Each Perspective
    — Conceptual: Meaning & View
    — In Real Life: Context & challenge
    — Our Approach: Our Solutions & Experiments
    — Exemplars: Open-source Examples & Libraries
    — Learnings: Lessons learned
    7

    View Slide

  8. Lens – Tensorflow.js
    — Javascript version of Tensorflow Deep Learning Library
    — Core (Linear Algebra low level API)
    — Layers (Keras-type high level API)
    — Runs both on Client-side (JS, WebAssembly, WebGL) and on
    Server-side (Nodejs with C binary for CPU & GPU)
    — Can be used for both inference and training
    — Custom models optimised for Image, Text, Audio etc.
    — Can Import & use models build in Python version
    8

    View Slide

  9. 1. Educate
    Explorable Explanations
    “For the things we have to learn before we can do
    them, we learn by doing them.”
    ― Aristotle
    9

    View Slide

  10. Neural Network Playground1
    1 Tinker with Neural Network
    10

    View Slide

  11. In Real Life: Context
    Client: Leading US Network Service Provider
    Scope: Threat detection using User & Entity
    Behaviour Analytics
    Status: Unsustainable Rule Based Policies
    Mandate: Evaluate shift to ML/DL Models
    Challenge: Domain experts skeptical of ML/DL
    paradigm
    11

    View Slide

  12. How to Educate?
    — Work with sample or open data
    — Show the potential for ML/DL models
    — Need to showcase visually explorable insights
    — Need for interactive exploration across
    stakeholders (CISO, CTO)
    12

    View Slide

  13. Explorable Explanations
    — Showcased multiple models: K-Means, One Class SVM,
    Autoencoders
    — Created models in Python & exported to Tensorflow.js
    — Integrated with an interactive dashboard (using AngularJS)
    — Built-in exploration capability on tweaking (and re-
    running) models
    Eventual API & implementation based on K-Means model
    13

    View Slide

  14. Exemplar: Blip2
    2 Anomaly Detection Dashboard by Fast Forward Labs
    14

    View Slide

  15. Exemplar: Distill.pub3
    3 How Momentum Works by Distill.pub
    15

    View Slide

  16. Learning on Educate
    — Discovery by Active Learning is powerful
    — Allows the user to build intuition on DL
    — Not easy! Requires deep thinking on design and
    usability
    16

    View Slide

  17. 2. Create
    Rapid Prototyping
    "The secret of getting started is breaking your
    complex overwhelming tasks into smaller manageable
    tasks, and then starting on the first one"
    - Mark Twain
    17

    View Slide

  18. Transfer Learning on Images4
    4 TensorSpace framework
    18

    View Slide

  19. In Real Life: Context
    Client: Boutique US Software Consulting shop
    Scope: Computer vision use-cases for an Automotive
    client
    Status: Full stack web & mobile expertise
    Mandate: Build & extend capabilities for ML&DL
    Challenge: No experience in python data stack
    19

    View Slide

  20. What to Prototype
    — Working with sample data
    — Needed to build an image classification model
    — Show how to integrate that model as API
    — Create an interactive dashboard to demo to client
    20

    View Slide

  21. Create Rapid Prototypes
    — Built CNN models for tra ic sign detection
    — Built models in tensorflow.js using pre-built
    models (transfer learning)
    — Deployed model as API using Node
    — Built a simple single-page application for user
    to upload image and get predictions
    21

    View Slide

  22. Exemplar: tfjs in Observablehq5
    5 Collection: Deep Learning in the Browser
    22

    View Slide

  23. Learning on Create
    — Enabling software developers to build ML unlocks
    huge potential
    — Easy to prototype without adding technical debt
    and additional technology costs (eg: Python data
    stack)
    — Not easy! How to build custom models.
    23

    View Slide

  24. 3. Visualise
    Model Visualisation
    “The most powerful way to gain insight into a
    system is by moving between levels of abstraction.”
    — Bret Victor
    24

    View Slide

  25. 25

    View Slide

  26. Model Visualisation
    [0] Visualise the data space
    [1] Visualise the predictions in the data space
    [2] Visualise the errors in model fitting
    [3] Visualise with di erent model parameters
    [4] Visualise with di erent input datasets
    [5] Visualise the entire model space
    [6] Visualise the entire feature space
    [7] Visualise the many models together
    26

    View Slide

  27. ModelVis & ML/DL
    [0] DATA VIS: the data space
    [1] PREDICTION: the predictions in the data space
    [2] VALIDATION: the errors in model fitting
    [3] TUNING: with di erent model parameters
    [4] BOOTSTRAP: with di erent input datasets
    [5] ENSEMBLE: the entire model space
    [6] FEATURE ENGG: the entire feature space
    [7] N-MODELS: the many models together
    27

    View Slide

  28. ModelVis Key Concept
    Use visualisation to aid the transition of implicit
    knowledge about the data to explicit knowledge in
    the model.
    — Iterative, not linear
    — Up and Down, not lateral
    — Complementary, not exclusive
    28

    View Slide

  29. In Real Life: Context
    Client: Large Consumer Product Company in India
    Scope: Network optimisation of secondary freight
    Status: Regional & fragmented network
    Mandate: Evaluate consolidation opportunities
    Challenge: Limited data on new options, Implicit
    domain knowledge
    29

    View Slide

  30. 30

    View Slide

  31. Applying Model Visualisation
    — Engage with Supply Chain domain experts with
    limited ML understanding
    — Need to represent models in the same data
    visualisation space
    — Need to showcase visually explorable model options
    — Need to build interactive exploration for di erent
    stakeholders (CSO, Managers)
    31

    View Slide

  32. 32

    View Slide

  33. Exemplar: Manifold from Uber6
    6 Model-Agnostic Visual Debugging tool
    33

    View Slide

  34. Exemplar: Feature Visualisation7
    7 The Building Blocks of Interpretability
    34

    View Slide

  35. Learning on Model Vis
    — Model improvements require engaging with domain
    through their lens
    — Visual analytics principles to model visualisation
    — Decouple model vis from model building compute &
    pipelines
    — Multivariate visualisation is more challenging
    than larger data!
    35

    View Slide

  36. 4. Intervene
    Decision Making
    "All models are wrong, but some are useful"
    — George Box
    36

    View Slide

  37. Using Models for Decisions8
    8 Fairness in Machine Learning
    37

    View Slide

  38. Context & Use Case
    Client: Small Regional Movies OTT App in India
    Scope: Map movie ROI & acquisition focus
    Status: Popularity & heuristic-based decisions
    Mandate: Build RecSys for engagement & licensing
    Challenge: Limited user insight, no data team
    38

    View Slide

  39. Challenges
    — User and movie personas were hard-coded
    — No real-time user insight to drive
    recommendations
    — Hard to link user behaviour to movie acquisition
    — Tool should work for business (content
    acquisition team), not only technology team
    39

    View Slide

  40. Decision Making Tool
    — Built multiple RecSys models to drive app
    recommendations
    — Interactive dashboard of user & movie embeddings
    — Created "What-If" scenario explorer - to link
    changing user behaviour with movie acquisition
    Shifted from Heuristic-based to Data-driven decision
    40

    View Slide

  41. Exemplar: RecSys & Business9
    9 Understanding Latent Factors by StitchFix
    41

    View Slide

  42. Exemplar: CounterFactuals10
    10 What-If tool by Google Pair
    42

    View Slide

  43. 5. Imagine
    Generative Spaces
    "I don't know, what I don't know"
    — The second level of ignorance
    43

    View Slide

  44. Font Design Space11
    11 Using Artificial Intelligence to Augment Human Intelligence
    44

    View Slide

  45. Generative Spaces
    — Moving beyond Supervised learning paradigms
    — Using generative models to invent meaningful
    creative operations
    — Designing and exploring latent spaces (with
    constraints)
    — Applications across design, product development,
    service design
    45

    View Slide

  46. Examples & Experiments using tfjs
    — ML for the web: ml5.js
    — Music & Art: magenta.js
    — Machine Learning for Artists: ml4a
    — AI Google Experiments: AI Collections
    46

    View Slide

  47. Democratize Deep Learning (DL)
    1. Educate: Explorable Explanations
    2. Create: Rapid Prototyping
    3. Visualise: Model Visualisation
    4. Intervene: Decision Making
    5. Imagine: Generative Spaces
    47

    View Slide

  48. Access this presentation at:
    https://speakerdeck.com/amitkaps
    Amit Kapoor
    amitkaps.com
    Bargava Subramanian
    bargava.com
    48

    View Slide