Deep Learning in the Browser

Deep Learning in the Browser

We showcase examples of doing deep learning (DL) in the browser - for building explorable explanations to aid insight, for building model inference applications and even, for rapid prototyping and training ML model - using the emerging client-side Javascript libraries for DL.

A4325416b0ed5218af22cf0473fe3d2b?s=128

Amit Kapoor

July 25, 2018
Tweet

Transcript

  1. Deep Learning in the Browser Explorables | Inference | Prototyping

    Amit Kapoor amitkaps.com
  2. Data | Visuals | Story

  3. Learning Paradigm

  4. Augmenting Intelligence

  5. Human in the Loop

  6. Users | Creators | Builders

  7. Learn | Play | Create

  8. In the Browser, Really?

  9. Immediate Accees Reduce Friction Wider Audience

  10. Historical Challenges ▸ Early JS libraries were too slow -

    CPU based ▸ Poor support for numerical operations ▸ Underdeveloped ecosystem for handling data and preprocessing, declarative visualisation, and reactive runtime
  11. Webgl accelerated Learning Frameworks

  12. Webgl accelerated Learning Frameworks ▸ tensorflow.js ▸ tensorfire ▸ webdnn.js

    [Inference only] ▸ keras.js [Inference only]
  13. How do we Learn?

  14. Explorable Explanations

  15. Discovery by Active Learning

  16. Allowing the users to build intuition on Deep Learning

  17. Algorithms Data Data-Models

  18. Algorithm: Maze Generation1 1 Visualising Algorithms - Mike Bostock

  19. Algorithm: Simple NN2 (2014) 2 Demo of Convnet.js by Andrej

    Karpathy
  20. Algorithm: Neural Network (2016)3 3 Tinker with Neural Network

  21. Algorithm: Universal Apprx. Concept 4 4 Universal Approximation using Tensorflow.js

  22. Data: Multi-Dimensional Visualisation5 5 Facet Dive

  23. Data: T-SNE Projector6 6 Embedding Projector

  24. Data-Models: Feature Vis7 7 The Building Blocks of Interpretability

  25. Data-Model: Fairness8 8 Fairness in Machine Learning

  26. Visualisation Reactivity Immediacy

  27. Multi-Disciplinary Skills Needed

  28. How do we Create?

  29. Model Inference

  30. Bring your Deep Learning Model to the Data.

  31. Abstract | Perceptual

  32. Model Inference: Text9 9 IMDB Sentiment with Keras.js

  33. Model Inference: Images10 10 Mnist with Keras.js

  34. Model Inference: Style Transfer11 11 Fast Neural Transfer with deeplearn.js

  35. Model Inference: Image Augmentation

  36. Semi-Supervised Learning12 12 Quick Draw Crowd Sourced Data

  37. Creative magenta.js | ml5js

  38. Data vs. Model privacy

  39. MODEL SIZE: quantization EMBEDDING SIZE: word2bit

  40. Easier build for Web & Mobile

  41. How do we Build?

  42. Rapid Prototyping

  43. Experiment and try new things

  44. UI Based: Object Detection13 13 Model Builder in Deeplearn.js

  45. Immediate environment to write code

  46. Reactive Notebooks: ObservableHQ14 14 Reactive Notebook in the Browser

  47. Challenges with Rapid Prototyping ▸ Training on GPUs (integration with

    Node.js) ▸ Custom layers not supported ▸ Lack of feature parity with Python / C++ APIs
  48. Improved JS Ecosystem ▸ Declarative visualisation: vega-lite.js ▸ Reactive Runtime:

    observablehq.com ▸ Columnar In-Memory: arrow.js
  49. Way Forward

  50. Stronger Ecosystem Stable Libraries Deeper Use Cases

  51. Deep Learning in the Browser Explorables | Inference | Prototyping

    Amit Kapoor amitkaps.com