Deep Learning in the Browser
Explorables | Inference | Prototyping
Amit Kapoor
amitkaps.com
Slide 2
Slide 2 text
Data | Visuals | Story
Slide 3
Slide 3 text
Learning
Paradigm
Slide 4
Slide 4 text
Augmenting
Intelligence
Slide 5
Slide 5 text
Human in the Loop
Slide 6
Slide 6 text
Users | Creators | Builders
Slide 7
Slide 7 text
Learn | Play | Create
Slide 8
Slide 8 text
In the Browser,
Really?
Slide 9
Slide 9 text
Immediate Accees
Reduce Friction
Wider Audience
Slide 10
Slide 10 text
Historical Challenges
▸ Early JS libraries were too slow -
CPU based
▸ Poor support for
numerical operations
▸ Underdeveloped ecosystem for
handling data
and preprocessing,
declarative visualisation, and
reactive runtime
Data-Models: Feature Vis7
7 The Building Blocks of Interpretability
Slide 25
Slide 25 text
Data-Model: Fairness8
8 Fairness in Machine Learning
Slide 26
Slide 26 text
Visualisation
Reactivity
Immediacy
Slide 27
Slide 27 text
Multi-Disciplinary Skills Needed
Slide 28
Slide 28 text
How do we Create?
Slide 29
Slide 29 text
Model Inference
Slide 30
Slide 30 text
Bring your Deep Learning
Model to the Data.
Slide 31
Slide 31 text
Abstract | Perceptual
Slide 32
Slide 32 text
Model Inference: Text9
9 IMDB Sentiment with Keras.js
Slide 33
Slide 33 text
Model Inference: Images10
10 Mnist with Keras.js
Slide 34
Slide 34 text
Model Inference: Style Transfer11
11 Fast Neural Transfer with deeplearn.js
Slide 35
Slide 35 text
Model Inference: Image Augmentation
Slide 36
Slide 36 text
Semi-Supervised Learning12
12 Quick Draw Crowd Sourced Data
Slide 37
Slide 37 text
Creative
magenta.js | ml5js
Slide 38
Slide 38 text
Data vs. Model privacy
Slide 39
Slide 39 text
MODEL SIZE: quantization
EMBEDDING SIZE: word2bit
Slide 40
Slide 40 text
Easier build for Web &
Mobile
Slide 41
Slide 41 text
How do we
Build?
Slide 42
Slide 42 text
Rapid
Prototyping
Slide 43
Slide 43 text
Experiment and try new
things
Slide 44
Slide 44 text
UI Based: Object Detection13
13 Model Builder in Deeplearn.js
Slide 45
Slide 45 text
Immediate environment to
write code
Slide 46
Slide 46 text
Reactive Notebooks: ObservableHQ14
14 Reactive Notebook in the Browser
Slide 47
Slide 47 text
Challenges with Rapid Prototyping
▸ Training on GPUs (integration with Node.js)
▸ Custom layers not supported
▸ Lack of feature parity with Python / C++
APIs