Democratize Deep Learning Models

Slide 1

Slide 1 text

Democratize Deep Learning Models Using TensorFlow.js Perspective & Views Amit Kapoor Bargava Subramanian 1

Slide 2

Slide 2 text

Amit Kapoor Data | Visual | Story — Teaching Analytics & Vis — Consulting & Mentoring — Building DataVis Tools amitkaps.com 2

Slide 3

Slide 3 text

Bargava Subramanian Data | AI & ML | Productivity — Startup @Binaize — Data Scientist (ex @Cisco, @RedHat) — Consulting on ML products bargava.com 3

Slide 4

Slide 4 text

Deep Learning Challenges — Scale Up: Faster data & models — Scale Out: Bigger data & models — Scale Wide: Entire data & model pipelines — Scale Across: Beyond data scientists & engineers How do we get more Human in the Loop approach to using Deep Learning? 4

Slide 5

Slide 5 text

Democratize Deep Learning (DL) 1. Educate: Why will DL work for our use case? 2. Create: What can we do to build our DL model? 3. Visualise: So how do we improve our DL models? 4. Intervene: So what actions can we take using DL? 5. Imagine: So what else can we use DL for? 5

Slide 6

Slide 6 text

Perspective on Democratize DL 1. Educate: Explorable Explanations 2. Create: Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 6

Slide 7

Slide 7 text

Structure for Each Perspective — Conceptual: Meaning & View — In Real Life: Context & challenge — Our Approach: Our Solutions & Experiments — Exemplars: Open-source Examples & Libraries — Learnings: Lessons learned 7

Slide 8

Slide 8 text

Lens – Tensorﬂow.js — Javascript version of Tensorﬂow Deep Learning Library — Core (Linear Algebra low level API) — Layers (Keras-type high level API) — Runs both on Client-side (JS, WebAssembly, WebGL) and on Server-side (Nodejs with C binary for CPU & GPU) — Can be used for both inference and training — Custom models optimised for Image, Text, Audio etc. — Can Import & use models build in Python version 8

Slide 9

Slide 9 text

1. Educate Explorable Explanations “For the things we have to learn before we can do them, we learn by doing them.” ― Aristotle 9

Slide 10

Slide 10 text

Neural Network Playground1 1 Tinker with Neural Network 10

Slide 11

Slide 11 text

In Real Life: Context Client: Leading US Network Service Provider Scope: Threat detection using User & Entity Behaviour Analytics Status: Unsustainable Rule Based Policies Mandate: Evaluate shift to ML/DL Models Challenge: Domain experts skeptical of ML/DL paradigm 11

Slide 12

Slide 12 text

How to Educate? — Work with sample or open data — Show the potential for ML/DL models — Need to showcase visually explorable insights — Need for interactive exploration across stakeholders (CISO, CTO) 12

Slide 13

Slide 13 text

Explorable Explanations — Showcased multiple models: K-Means, One Class SVM, Autoencoders — Created models in Python & exported to Tensorﬂow.js — Integrated with an interactive dashboard (using AngularJS) — Built-in exploration capability on tweaking (and re- running) models Eventual API & implementation based on K-Means model 13

Slide 14

Slide 14 text

Exemplar: Blip2 2 Anomaly Detection Dashboard by Fast Forward Labs 14

Slide 15

Slide 15 text

Exemplar: Distill.pub3 3 How Momentum Works by Distill.pub 15

Slide 16

Slide 16 text

Learning on Educate — Discovery by Active Learning is powerful — Allows the user to build intuition on DL — Not easy! Requires deep thinking on design and usability 16

Slide 17

Slide 17 text

2. Create Rapid Prototyping "The secret of getting started is breaking your complex overwhelming tasks into smaller manageable tasks, and then starting on the ﬁrst one" - Mark Twain 17

Slide 18

Slide 18 text

Transfer Learning on Images4 4 TensorSpace framework 18

Slide 19

Slide 19 text

In Real Life: Context Client: Boutique US Software Consulting shop Scope: Computer vision use-cases for an Automotive client Status: Full stack web & mobile expertise Mandate: Build & extend capabilities for ML&DL Challenge: No experience in python data stack 19

Slide 20

Slide 20 text

What to Prototype — Working with sample data — Needed to build an image classiﬁcation model — Show how to integrate that model as API — Create an interactive dashboard to demo to client 20

Slide 21

Slide 21 text

Create Rapid Prototypes — Built CNN models for tra ic sign detection — Built models in tensorﬂow.js using pre-built models (transfer learning) — Deployed model as API using Node — Built a simple single-page application for user to upload image and get predictions 21

Slide 22

Slide 22 text

Exemplar: tfjs in Observablehq5 5 Collection: Deep Learning in the Browser 22

Slide 23

Slide 23 text

Learning on Create — Enabling software developers to build ML unlocks huge potential — Easy to prototype without adding technical debt and additional technology costs (eg: Python data stack) — Not easy! How to build custom models. 23

Slide 24

Slide 24 text

3. Visualise Model Visualisation “The most powerful way to gain insight into a system is by moving between levels of abstraction.” — Bret Victor 24

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Model Visualisation [0] Visualise the data space [1] Visualise the predictions in the data space [2] Visualise the errors in model ﬁtting [3] Visualise with di erent model parameters [4] Visualise with di erent input datasets [5] Visualise the entire model space [6] Visualise the entire feature space [7] Visualise the many models together 26

Slide 27

Slide 27 text

ModelVis & ML/DL [0] DATA VIS: the data space [1] PREDICTION: the predictions in the data space [2] VALIDATION: the errors in model ﬁtting [3] TUNING: with di erent model parameters [4] BOOTSTRAP: with di erent input datasets [5] ENSEMBLE: the entire model space [6] FEATURE ENGG: the entire feature space [7] N-MODELS: the many models together 27

Slide 28

Slide 28 text

ModelVis Key Concept Use visualisation to aid the transition of implicit knowledge about the data to explicit knowledge in the model. — Iterative, not linear — Up and Down, not lateral — Complementary, not exclusive 28

Slide 29

Slide 29 text

In Real Life: Context Client: Large Consumer Product Company in India Scope: Network optimisation of secondary freight Status: Regional & fragmented network Mandate: Evaluate consolidation opportunities Challenge: Limited data on new options, Implicit domain knowledge 29

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Applying Model Visualisation — Engage with Supply Chain domain experts with limited ML understanding — Need to represent models in the same data visualisation space — Need to showcase visually explorable model options — Need to build interactive exploration for di erent stakeholders (CSO, Managers) 31

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Exemplar: Manifold from Uber6 6 Model-Agnostic Visual Debugging tool 33

Slide 34

Slide 34 text

Exemplar: Feature Visualisation7 7 The Building Blocks of Interpretability 34

Slide 35

Slide 35 text

Learning on Model Vis — Model improvements require engaging with domain through their lens — Visual analytics principles to model visualisation — Decouple model vis from model building compute & pipelines — Multivariate visualisation is more challenging than larger data! 35

Slide 36

Slide 36 text

4. Intervene Decision Making "All models are wrong, but some are useful" — George Box 36

Slide 37

Slide 37 text

Using Models for Decisions8 8 Fairness in Machine Learning 37

Slide 38

Slide 38 text

Context & Use Case Client: Small Regional Movies OTT App in India Scope: Map movie ROI & acquisition focus Status: Popularity & heuristic-based decisions Mandate: Build RecSys for engagement & licensing Challenge: Limited user insight, no data team 38

Slide 39

Slide 39 text

Challenges — User and movie personas were hard-coded — No real-time user insight to drive recommendations — Hard to link user behaviour to movie acquisition — Tool should work for business (content acquisition team), not only technology team 39

Slide 40

Slide 40 text

Decision Making Tool — Built multiple RecSys models to drive app recommendations — Interactive dashboard of user & movie embeddings — Created "What-If" scenario explorer - to link changing user behaviour with movie acquisition Shifted from Heuristic-based to Data-driven decision 40

Slide 41

Slide 41 text

Exemplar: RecSys & Business9 9 Understanding Latent Factors by StitchFix 41

Slide 42

Slide 42 text

Exemplar: CounterFactuals10 10 What-If tool by Google Pair 42

Slide 43

Slide 43 text

5. Imagine Generative Spaces "I don't know, what I don't know" — The second level of ignorance 43

Slide 44

Slide 44 text

Font Design Space11 11 Using Artiﬁcial Intelligence to Augment Human Intelligence 44

Slide 45

Slide 45 text

Generative Spaces — Moving beyond Supervised learning paradigms — Using generative models to invent meaningful creative operations — Designing and exploring latent spaces (with constraints) — Applications across design, product development, service design 45

Slide 46

Slide 46 text

Examples & Experiments using tfjs — ML for the web: ml5.js — Music & Art: magenta.js — Machine Learning for Artists: ml4a — AI Google Experiments: AI Collections 46

Slide 47

Slide 47 text

Democratize Deep Learning (DL) 1. Educate: Explorable Explanations 2. Create: Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 47

Slide 48

Slide 48 text

Access this presentation at: https://speakerdeck.com/amitkaps Amit Kapoor amitkaps.com Bargava Subramanian bargava.com 48