Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Democratize Deep Learning Models

Democratize Deep Learning Models

A perspective on how to make deep learning models more accessible and usable for a wider spectrum of people. Using tensorflow.js

1. Educate: Explorable Explanations
2. Create: Rapid Prototyping
3. Visualise: Model Visualisation
4. Intervene: Decision Making
5. Imagine: Generative Spaces

Amit Kapoor

June 30, 2020
Tweet

More Decks by Amit Kapoor

Other Decks in Technology

Transcript

  1. Democratize Deep Learning Models
    Using TensorFlow.js
    Perspective & Views
    Amit Kapoor
    Bargava Subramanian
    1

    View full-size slide

  2. Amit Kapoor
    Data | Visual | Story
    — Teaching Analytics & Vis
    — Consulting & Mentoring
    — Building DataVis Tools
    amitkaps.com
    2

    View full-size slide

  3. Bargava Subramanian
    Data | AI & ML | Productivity
    — Startup @Binaize
    — Data Scientist (ex @Cisco,
    @RedHat)
    — Consulting on ML products
    bargava.com
    3

    View full-size slide

  4. Deep Learning Challenges
    — Scale Up: Faster data & models
    — Scale Out: Bigger data & models
    — Scale Wide: Entire data & model pipelines
    — Scale Across: Beyond data scientists & engineers
    How do we get more Human in the Loop approach to
    using Deep Learning?
    4

    View full-size slide

  5. Democratize Deep Learning (DL)
    1. Educate: Why will DL work for our use case?
    2. Create: What can we do to build our DL model?
    3. Visualise: So how do we improve our DL models?
    4. Intervene: So what actions can we take using DL?
    5. Imagine: So what else can we use DL for?
    5

    View full-size slide

  6. Perspective on Democratize DL
    1. Educate: Explorable Explanations
    2. Create: Rapid Prototyping
    3. Visualise: Model Visualisation
    4. Intervene: Decision Making
    5. Imagine: Generative Spaces
    6

    View full-size slide

  7. Structure for Each Perspective
    — Conceptual: Meaning & View
    — In Real Life: Context & challenge
    — Our Approach: Our Solutions & Experiments
    — Exemplars: Open-source Examples & Libraries
    — Learnings: Lessons learned
    7

    View full-size slide

  8. Lens – Tensorflow.js
    — Javascript version of Tensorflow Deep Learning Library
    — Core (Linear Algebra low level API)
    — Layers (Keras-type high level API)
    — Runs both on Client-side (JS, WebAssembly, WebGL) and on
    Server-side (Nodejs with C binary for CPU & GPU)
    — Can be used for both inference and training
    — Custom models optimised for Image, Text, Audio etc.
    — Can Import & use models build in Python version
    8

    View full-size slide

  9. 1. Educate
    Explorable Explanations
    “For the things we have to learn before we can do
    them, we learn by doing them.”
    ― Aristotle
    9

    View full-size slide

  10. Neural Network Playground1
    1 Tinker with Neural Network
    10

    View full-size slide

  11. In Real Life: Context
    Client: Leading US Network Service Provider
    Scope: Threat detection using User & Entity
    Behaviour Analytics
    Status: Unsustainable Rule Based Policies
    Mandate: Evaluate shift to ML/DL Models
    Challenge: Domain experts skeptical of ML/DL
    paradigm
    11

    View full-size slide

  12. How to Educate?
    — Work with sample or open data
    — Show the potential for ML/DL models
    — Need to showcase visually explorable insights
    — Need for interactive exploration across
    stakeholders (CISO, CTO)
    12

    View full-size slide

  13. Explorable Explanations
    — Showcased multiple models: K-Means, One Class SVM,
    Autoencoders
    — Created models in Python & exported to Tensorflow.js
    — Integrated with an interactive dashboard (using AngularJS)
    — Built-in exploration capability on tweaking (and re-
    running) models
    Eventual API & implementation based on K-Means model
    13

    View full-size slide

  14. Exemplar: Blip2
    2 Anomaly Detection Dashboard by Fast Forward Labs
    14

    View full-size slide

  15. Exemplar: Distill.pub3
    3 How Momentum Works by Distill.pub
    15

    View full-size slide

  16. Learning on Educate
    — Discovery by Active Learning is powerful
    — Allows the user to build intuition on DL
    — Not easy! Requires deep thinking on design and
    usability
    16

    View full-size slide

  17. 2. Create
    Rapid Prototyping
    "The secret of getting started is breaking your
    complex overwhelming tasks into smaller manageable
    tasks, and then starting on the first one"
    - Mark Twain
    17

    View full-size slide

  18. Transfer Learning on Images4
    4 TensorSpace framework
    18

    View full-size slide

  19. In Real Life: Context
    Client: Boutique US Software Consulting shop
    Scope: Computer vision use-cases for an Automotive
    client
    Status: Full stack web & mobile expertise
    Mandate: Build & extend capabilities for ML&DL
    Challenge: No experience in python data stack
    19

    View full-size slide

  20. What to Prototype
    — Working with sample data
    — Needed to build an image classification model
    — Show how to integrate that model as API
    — Create an interactive dashboard to demo to client
    20

    View full-size slide

  21. Create Rapid Prototypes
    — Built CNN models for tra ic sign detection
    — Built models in tensorflow.js using pre-built
    models (transfer learning)
    — Deployed model as API using Node
    — Built a simple single-page application for user
    to upload image and get predictions
    21

    View full-size slide

  22. Exemplar: tfjs in Observablehq5
    5 Collection: Deep Learning in the Browser
    22

    View full-size slide

  23. Learning on Create
    — Enabling software developers to build ML unlocks
    huge potential
    — Easy to prototype without adding technical debt
    and additional technology costs (eg: Python data
    stack)
    — Not easy! How to build custom models.
    23

    View full-size slide

  24. 3. Visualise
    Model Visualisation
    “The most powerful way to gain insight into a
    system is by moving between levels of abstraction.”
    — Bret Victor
    24

    View full-size slide

  25. Model Visualisation
    [0] Visualise the data space
    [1] Visualise the predictions in the data space
    [2] Visualise the errors in model fitting
    [3] Visualise with di erent model parameters
    [4] Visualise with di erent input datasets
    [5] Visualise the entire model space
    [6] Visualise the entire feature space
    [7] Visualise the many models together
    26

    View full-size slide

  26. ModelVis & ML/DL
    [0] DATA VIS: the data space
    [1] PREDICTION: the predictions in the data space
    [2] VALIDATION: the errors in model fitting
    [3] TUNING: with di erent model parameters
    [4] BOOTSTRAP: with di erent input datasets
    [5] ENSEMBLE: the entire model space
    [6] FEATURE ENGG: the entire feature space
    [7] N-MODELS: the many models together
    27

    View full-size slide

  27. ModelVis Key Concept
    Use visualisation to aid the transition of implicit
    knowledge about the data to explicit knowledge in
    the model.
    — Iterative, not linear
    — Up and Down, not lateral
    — Complementary, not exclusive
    28

    View full-size slide

  28. In Real Life: Context
    Client: Large Consumer Product Company in India
    Scope: Network optimisation of secondary freight
    Status: Regional & fragmented network
    Mandate: Evaluate consolidation opportunities
    Challenge: Limited data on new options, Implicit
    domain knowledge
    29

    View full-size slide

  29. Applying Model Visualisation
    — Engage with Supply Chain domain experts with
    limited ML understanding
    — Need to represent models in the same data
    visualisation space
    — Need to showcase visually explorable model options
    — Need to build interactive exploration for di erent
    stakeholders (CSO, Managers)
    31

    View full-size slide

  30. Exemplar: Manifold from Uber6
    6 Model-Agnostic Visual Debugging tool
    33

    View full-size slide

  31. Exemplar: Feature Visualisation7
    7 The Building Blocks of Interpretability
    34

    View full-size slide

  32. Learning on Model Vis
    — Model improvements require engaging with domain
    through their lens
    — Visual analytics principles to model visualisation
    — Decouple model vis from model building compute &
    pipelines
    — Multivariate visualisation is more challenging
    than larger data!
    35

    View full-size slide

  33. 4. Intervene
    Decision Making
    "All models are wrong, but some are useful"
    — George Box
    36

    View full-size slide

  34. Using Models for Decisions8
    8 Fairness in Machine Learning
    37

    View full-size slide

  35. Context & Use Case
    Client: Small Regional Movies OTT App in India
    Scope: Map movie ROI & acquisition focus
    Status: Popularity & heuristic-based decisions
    Mandate: Build RecSys for engagement & licensing
    Challenge: Limited user insight, no data team
    38

    View full-size slide

  36. Challenges
    — User and movie personas were hard-coded
    — No real-time user insight to drive
    recommendations
    — Hard to link user behaviour to movie acquisition
    — Tool should work for business (content
    acquisition team), not only technology team
    39

    View full-size slide

  37. Decision Making Tool
    — Built multiple RecSys models to drive app
    recommendations
    — Interactive dashboard of user & movie embeddings
    — Created "What-If" scenario explorer - to link
    changing user behaviour with movie acquisition
    Shifted from Heuristic-based to Data-driven decision
    40

    View full-size slide

  38. Exemplar: RecSys & Business9
    9 Understanding Latent Factors by StitchFix
    41

    View full-size slide

  39. Exemplar: CounterFactuals10
    10 What-If tool by Google Pair
    42

    View full-size slide

  40. 5. Imagine
    Generative Spaces
    "I don't know, what I don't know"
    — The second level of ignorance
    43

    View full-size slide

  41. Font Design Space11
    11 Using Artificial Intelligence to Augment Human Intelligence
    44

    View full-size slide

  42. Generative Spaces
    — Moving beyond Supervised learning paradigms
    — Using generative models to invent meaningful
    creative operations
    — Designing and exploring latent spaces (with
    constraints)
    — Applications across design, product development,
    service design
    45

    View full-size slide

  43. Examples & Experiments using tfjs
    — ML for the web: ml5.js
    — Music & Art: magenta.js
    — Machine Learning for Artists: ml4a
    — AI Google Experiments: AI Collections
    46

    View full-size slide

  44. Democratize Deep Learning (DL)
    1. Educate: Explorable Explanations
    2. Create: Rapid Prototyping
    3. Visualise: Model Visualisation
    4. Intervene: Decision Making
    5. Imagine: Generative Spaces
    47

    View full-size slide

  45. Access this presentation at:
    https://speakerdeck.com/amitkaps
    Amit Kapoor
    amitkaps.com
    Bargava Subramanian
    bargava.com
    48

    View full-size slide