Deep Learning in the Browser

We showcase examples of doing deep learning (DL) in the browser - for building explorable explanations to aid insight, for building model inference applications and even, for rapid prototyping and training ML model - using the emerging client-side Javascript libraries for DL.

Transcript

1. Deep Learning in the Browser Explorables | Inference | Prototyping Amit Kapoor amitkaps.com
2. Data | Visuals | Story
3. Learning Paradigm
4. Augmenting Intelligence
5. Human in the Loop
6. Users | Creators | Builders
7. Learn | Play | Create
8. In the Browser, Really?
9. Immediate Accees Reduce Friction Wider Audience
10. Historical Challenges ▸ Early JS libraries were too slow - CPU based ▸ Poor support for numerical operations ▸ Underdeveloped ecosystem for handling data and preprocessing, declarative visualisation, and reactive runtime
11. Webgl accelerated Learning Frameworks
12. Webgl accelerated Learning Frameworks ▸ tensorflow.js ▸ tensorfire ▸ webdnn.js [Inference only] ▸ keras.js [Inference only]
13. How do we Learn?
14. Explorable Explanations
15. Discovery by Active Learning
16. Allowing the users to build intuition on Deep Learning
17. Algorithms Data Data-Models
18. Algorithm: Maze Generation1 1 Visualising Algorithms - Mike Bostock
19. Algorithm: Simple NN2 (2014) 2 Demo of Convnet.js by Andrej Karpathy
20. Algorithm: Neural Network (2016)3 3 Tinker with Neural Network
21. Algorithm: Universal Apprx. Concept 4 4 Universal Approximation using Tensorflow.js
22. Data: Multi-Dimensional Visualisation5 5 Facet Dive
23. Data: T-SNE Projector6 6 Embedding Projector
24. Data-Models: Feature Vis7 7 The Building Blocks of Interpretability
25. Data-Model: Fairness8 8 Fairness in Machine Learning
26. Visualisation Reactivity Immediacy
27. Multi-Disciplinary Skills Needed
28. How do we Create?
29. Model Inference
30. Bring your Deep Learning Model to the Data.
31. Abstract | Perceptual
32. Model Inference: Text9 9 IMDB Sentiment with Keras.js
33. Model Inference: Images10 10 Mnist with Keras.js
34. Model Inference: Style Transfer11 11 Fast Neural Transfer with deeplearn.js
35. Model Inference: Image Augmentation
36. Semi-Supervised Learning12 12 Quick Draw Crowd Sourced Data
37. Creative magenta.js | ml5js
38. Data vs. Model privacy
39. MODEL SIZE: quantization EMBEDDING SIZE: word2bit
40. Easier build for Web & Mobile
41. How do we Build?
42. Rapid Prototyping
43. Experiment and try new things
44. UI Based: Object Detection13 13 Model Builder in Deeplearn.js
45. Immediate environment to write code
46. Reactive Notebooks: ObservableHQ14 14 Reactive Notebook in the Browser
47. Challenges with Rapid Prototyping ▸ Training on GPUs (integration with Node.js) ▸ Custom layers not supported ▸ Lack of feature parity with Python / C++ APIs
48. Improved JS Ecosystem ▸ Declarative visualisation: vega-lite.js ▸ Reactive Runtime: observablehq.com ▸ Columnar In-Memory: arrow.js
49. Way Forward
50. Stronger Ecosystem Stable Libraries Deeper Use Cases
51. Deep Learning in the Browser Explorables | Inference | Prototyping Amit Kapoor amitkaps.com