Democratize Deep Learning Models

A perspective on how to make deep learning models more accessible and usable for a wider spectrum of people. Using tensorflow.js

1. Educate: Explorable Explanations
2. Create: Rapid Prototyping
3. Visualise: Model Visualisation
4. Intervene: Decision Making
5. Imagine: Generative Spaces

Transcript

1. Democratize Deep Learning Models Using TensorFlow.js Perspective & Views Amit Kapoor Bargava Subramanian 1
2. Amit Kapoor Data | Visual | Story — Teaching Analytics & Vis — Consulting & Mentoring — Building DataVis Tools amitkaps.com 2
3. Bargava Subramanian Data | AI & ML | Productivity — Startup @Binaize — Data Scientist (ex @Cisco, @RedHat) — Consulting on ML products bargava.com 3
4. Deep Learning Challenges — Scale Up: Faster data & models — Scale Out: Bigger data & models — Scale Wide: Entire data & model pipelines — Scale Across: Beyond data scientists & engineers How do we get more Human in the Loop approach to using Deep Learning? 4
5. Democratize Deep Learning (DL) 1. Educate: Why will DL work for our use case? 2. Create: What can we do to build our DL model? 3. Visualise: So how do we improve our DL models? 4. Intervene: So what actions can we take using DL? 5. Imagine: So what else can we use DL for? 5
6. Perspective on Democratize DL 1. Educate: Explorable Explanations 2. Create: Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 6
7. Structure for Each Perspective — Conceptual: Meaning & View — In Real Life: Context & challenge — Our Approach: Our Solutions & Experiments — Exemplars: Open-source Examples & Libraries — Learnings: Lessons learned 7
8. Lens – Tensorflow.js — Javascript version of Tensorflow Deep Learning Library — Core (Linear Algebra low level API) — Layers (Keras-type high level API) — Runs both on Client-side (JS, WebAssembly, WebGL) and on Server-side (Nodejs with C binary for CPU & GPU) — Can be used for both inference and training — Custom models optimised for Image, Text, Audio etc. — Can Import & use models build in Python version 8
9. 1. Educate Explorable Explanations “For the things we have to learn before we can do them, we learn by doing them.” ― Aristotle 9
10. Neural Network Playground1 1 Tinker with Neural Network 10
11. In Real Life: Context Client: Leading US Network Service Provider Scope: Threat detection using User & Entity Behaviour Analytics Status: Unsustainable Rule Based Policies Mandate: Evaluate shift to ML/DL Models Challenge: Domain experts skeptical of ML/DL paradigm 11
12. How to Educate? — Work with sample or open data — Show the potential for ML/DL models — Need to showcase visually explorable insights — Need for interactive exploration across stakeholders (CISO, CTO) 12
13. Explorable Explanations — Showcased multiple models: K-Means, One Class SVM, Autoencoders — Created models in Python & exported to Tensorflow.js — Integrated with an interactive dashboard (using AngularJS) — Built-in exploration capability on tweaking (and re- running) models Eventual API & implementation based on K-Means model 13
14. Exemplar: Blip2 2 Anomaly Detection Dashboard by Fast Forward Labs 14
15. Exemplar: Distill.pub3 3 How Momentum Works by Distill.pub 15
16. Learning on Educate — Discovery by Active Learning is powerful — Allows the user to build intuition on DL — Not easy! Requires deep thinking on design and usability 16
17. 2. Create Rapid Prototyping "The secret of getting started is breaking your complex overwhelming tasks into smaller manageable tasks, and then starting on the first one" - Mark Twain 17
18. Transfer Learning on Images4 4 TensorSpace framework 18
19. In Real Life: Context Client: Boutique US Software Consulting shop Scope: Computer vision use-cases for an Automotive client Status: Full stack web & mobile expertise Mandate: Build & extend capabilities for ML&DL Challenge: No experience in python data stack 19
20. What to Prototype — Working with sample data — Needed to build an image classification model — Show how to integrate that model as API — Create an interactive dashboard to demo to client 20
21. Create Rapid Prototypes — Built CNN models for tra ic sign detection — Built models in tensorflow.js using pre-built models (transfer learning) — Deployed model as API using Node — Built a simple single-page application for user to upload image and get predictions 21
22. Exemplar: tfjs in Observablehq5 5 Collection: Deep Learning in the Browser 22
23. Learning on Create — Enabling software developers to build ML unlocks huge potential — Easy to prototype without adding technical debt and additional technology costs (eg: Python data stack) — Not easy! How to build custom models. 23
24. 3. Visualise Model Visualisation “The most powerful way to gain insight into a system is by moving between levels of abstraction.” — Bret Victor 24
26. Model Visualisation [0] Visualise the data space [1] Visualise the predictions in the data space [2] Visualise the errors in model fitting [3] Visualise with di erent model parameters [4] Visualise with di erent input datasets [5] Visualise the entire model space [6] Visualise the entire feature space [7] Visualise the many models together 26
27. ModelVis & ML/DL [0] DATA VIS: the data space [1] PREDICTION: the predictions in the data space [2] VALIDATION: the errors in model fitting [3] TUNING: with di erent model parameters [4] BOOTSTRAP: with di erent input datasets [5] ENSEMBLE: the entire model space [6] FEATURE ENGG: the entire feature space [7] N-MODELS: the many models together 27
28. ModelVis Key Concept Use visualisation to aid the transition of implicit knowledge about the data to explicit knowledge in the model. — Iterative, not linear — Up and Down, not lateral — Complementary, not exclusive 28
29. In Real Life: Context Client: Large Consumer Product Company in India Scope: Network optimisation of secondary freight Status: Regional & fragmented network Mandate: Evaluate consolidation opportunities Challenge: Limited data on new options, Implicit domain knowledge 29
31. Applying Model Visualisation — Engage with Supply Chain domain experts with limited ML understanding — Need to represent models in the same data visualisation space — Need to showcase visually explorable model options — Need to build interactive exploration for di erent stakeholders (CSO, Managers) 31
33. Exemplar: Manifold from Uber6 6 Model-Agnostic Visual Debugging tool 33
34. Exemplar: Feature Visualisation7 7 The Building Blocks of Interpretability 34
35. Learning on Model Vis — Model improvements require engaging with domain through their lens — Visual analytics principles to model visualisation — Decouple model vis from model building compute & pipelines — Multivariate visualisation is more challenging than larger data! 35
36. 4. Intervene Decision Making "All models are wrong, but some are useful" — George Box 36
37. Using Models for Decisions8 8 Fairness in Machine Learning 37
38. Context & Use Case Client: Small Regional Movies OTT App in India Scope: Map movie ROI & acquisition focus Status: Popularity & heuristic-based decisions Mandate: Build RecSys for engagement & licensing Challenge: Limited user insight, no data team 38
39. Challenges — User and movie personas were hard-coded — No real-time user insight to drive recommendations — Hard to link user behaviour to movie acquisition — Tool should work for business (content acquisition team), not only technology team 39
40. Decision Making Tool — Built multiple RecSys models to drive app recommendations — Interactive dashboard of user & movie embeddings — Created "What-If" scenario explorer - to link changing user behaviour with movie acquisition Shifted from Heuristic-based to Data-driven decision 40
41. Exemplar: RecSys & Business9 9 Understanding Latent Factors by StitchFix 41
42. Exemplar: CounterFactuals10 10 What-If tool by Google Pair 42
43. 5. Imagine Generative Spaces "I don't know, what I don't know" — The second level of ignorance 43
44. Font Design Space11 11 Using Artificial Intelligence to Augment Human Intelligence 44
45. Generative Spaces — Moving beyond Supervised learning paradigms — Using generative models to invent meaningful creative operations — Designing and exploring latent spaces (with constraints) — Applications across design, product development, service design 45
46. Examples & Experiments using tfjs — ML for the web: ml5.js — Music & Art: magenta.js — Machine Learning for Artists: ml4a — AI Google Experiments: AI Collections 46
47. Democratize Deep Learning (DL) 1. Educate: Explorable Explanations 2. Create: Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 47
48. Access this presentation at: https://speakerdeck.com/amitkaps Amit Kapoor amitkaps.com Bargava Subramanian bargava.com 48