Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning in the Browser

Deep Learning in the Browser

We showcase examples of doing deep learning (DL) in the browser - for building explorable explanations to aid insight, for building model inference applications and even, for rapid prototyping and training ML model - using the emerging client-side Javascript libraries for DL.

Amit Kapoor

July 25, 2018
Tweet

More Decks by Amit Kapoor

Other Decks in Technology

Transcript

  1. Deep Learning in the Browser
    Explorables | Inference | Prototyping
    Amit Kapoor
    amitkaps.com

    View full-size slide

  2. Data | Visuals | Story

    View full-size slide

  3. Learning
    Paradigm

    View full-size slide

  4. Augmenting
    Intelligence

    View full-size slide

  5. Human in the Loop

    View full-size slide

  6. Users | Creators | Builders

    View full-size slide

  7. Learn | Play | Create

    View full-size slide

  8. In the Browser,
    Really?

    View full-size slide

  9. Immediate Accees
    Reduce Friction
    Wider Audience

    View full-size slide

  10. Historical Challenges
    ▸ Early JS libraries were too slow -
    CPU based
    ▸ Poor support for
    numerical operations
    ▸ Underdeveloped ecosystem for
    handling data
    and preprocessing,
    declarative visualisation, and
    reactive runtime

    View full-size slide

  11. Webgl accelerated Learning
    Frameworks

    View full-size slide

  12. Webgl accelerated Learning Frameworks
    ▸ tensorflow.js
    ▸ tensorfire
    ▸ webdnn.js [Inference only]
    ▸ keras.js [Inference only]

    View full-size slide

  13. How do we Learn?

    View full-size slide

  14. Explorable
    Explanations

    View full-size slide

  15. Discovery by Active Learning

    View full-size slide

  16. Allowing the users to build
    intuition on Deep Learning

    View full-size slide

  17. Algorithms
    Data
    Data-Models

    View full-size slide

  18. Algorithm: Maze Generation1
    1 Visualising Algorithms - Mike Bostock

    View full-size slide

  19. Algorithm: Simple NN2 (2014)
    2 Demo of Convnet.js by Andrej Karpathy

    View full-size slide

  20. Algorithm: Neural Network (2016)3
    3 Tinker with Neural Network

    View full-size slide

  21. Algorithm: Universal Apprx. Concept 4
    4 Universal Approximation using Tensorflow.js

    View full-size slide

  22. Data: Multi-Dimensional Visualisation5
    5 Facet Dive

    View full-size slide

  23. Data: T-SNE Projector6
    6 Embedding Projector

    View full-size slide

  24. Data-Models: Feature Vis7
    7 The Building Blocks of Interpretability

    View full-size slide

  25. Data-Model: Fairness8
    8 Fairness in Machine Learning

    View full-size slide

  26. Visualisation
    Reactivity
    Immediacy

    View full-size slide

  27. Multi-Disciplinary Skills Needed

    View full-size slide

  28. How do we Create?

    View full-size slide

  29. Model Inference

    View full-size slide

  30. Bring your Deep Learning
    Model to the Data.

    View full-size slide

  31. Abstract | Perceptual

    View full-size slide

  32. Model Inference: Text9
    9 IMDB Sentiment with Keras.js

    View full-size slide

  33. Model Inference: Images10
    10 Mnist with Keras.js

    View full-size slide

  34. Model Inference: Style Transfer11
    11 Fast Neural Transfer with deeplearn.js

    View full-size slide

  35. Model Inference: Image Augmentation

    View full-size slide

  36. Semi-Supervised Learning12
    12 Quick Draw Crowd Sourced Data

    View full-size slide

  37. Creative
    magenta.js | ml5js

    View full-size slide

  38. Data vs. Model privacy

    View full-size slide

  39. MODEL SIZE: quantization
    EMBEDDING SIZE: word2bit

    View full-size slide

  40. Easier build for Web &
    Mobile

    View full-size slide

  41. How do we
    Build?

    View full-size slide

  42. Rapid
    Prototyping

    View full-size slide

  43. Experiment and try new
    things

    View full-size slide

  44. UI Based: Object Detection13
    13 Model Builder in Deeplearn.js

    View full-size slide

  45. Immediate environment to
    write code

    View full-size slide

  46. Reactive Notebooks: ObservableHQ14
    14 Reactive Notebook in the Browser

    View full-size slide

  47. Challenges with Rapid Prototyping
    ▸ Training on GPUs (integration with Node.js)
    ▸ Custom layers not supported
    ▸ Lack of feature parity with Python / C++
    APIs

    View full-size slide

  48. Improved JS Ecosystem
    ▸ Declarative visualisation: vega-lite.js
    ▸ Reactive Runtime: observablehq.com
    ▸ Columnar In-Memory: arrow.js

    View full-size slide

  49. Stronger Ecosystem
    Stable Libraries
    Deeper Use Cases

    View full-size slide

  50. Deep Learning in the Browser
    Explorables | Inference | Prototyping
    Amit Kapoor
    amitkaps.com

    View full-size slide