Deep Learning in the Browser

We showcase examples of doing deep learning (DL) in the browser - for building explorable explanations to aid insight, for building model inference applications and even, for rapid prototyping and training ML model - using the emerging client-side Javascript libraries for DL.

Transcript

1. Deep Learning in the Browser
   Explorables | Inference | Prototyping
   Amit Kapoor
   amitkaps.com
   

   View Slide

2. Data | Visuals | Story
   

   View Slide

3. Learning
   Paradigm
   

   View Slide

4. Augmenting
   Intelligence
   

   View Slide

5. Human in the Loop
   

   View Slide

6. Users | Creators | Builders
   

   View Slide

7. Learn | Play | Create
   

   View Slide

8. In the Browser,
   Really?
   

   View Slide

9. Immediate Accees
   Reduce Friction
   Wider Audience
   

   View Slide

10. Historical Challenges
    ▸ Early JS libraries were too slow -
    CPU based
    ▸ Poor support for
    numerical operations
    ▸ Underdeveloped ecosystem for
    handling data
    and preprocessing,
    declarative visualisation, and
    reactive runtime
    

    View Slide

11. Webgl accelerated Learning
    Frameworks
    

    View Slide

12. Webgl accelerated Learning Frameworks
    ▸ tensorflow.js
    ▸ tensorfire
    ▸ webdnn.js [Inference only]
    ▸ keras.js [Inference only]
    

    View Slide

13. How do we Learn?
    

    View Slide

14. Explorable
    Explanations
    

    View Slide

15. Discovery by Active Learning
    

    View Slide

16. Allowing the users to build
    intuition on Deep Learning
    

    View Slide

17. Algorithms
    Data
    Data-Models
    

    View Slide

18. Algorithm: Maze Generation1
    1 Visualising Algorithms - Mike Bostock
    

    View Slide

19. Algorithm: Simple NN2 (2014)
    2 Demo of Convnet.js by Andrej Karpathy
    

    View Slide

20. Algorithm: Neural Network (2016)3
    3 Tinker with Neural Network
    

    View Slide

21. Algorithm: Universal Apprx. Concept 4
    4 Universal Approximation using Tensorflow.js
    

    View Slide

22. Data: Multi-Dimensional Visualisation5
    5 Facet Dive
    

    View Slide

23. Data: T-SNE Projector6
    6 Embedding Projector
    

    View Slide

24. Data-Models: Feature Vis7
    7 The Building Blocks of Interpretability
    

    View Slide

25. Data-Model: Fairness8
    8 Fairness in Machine Learning
    

    View Slide

26. Visualisation
    Reactivity
    Immediacy
    

    View Slide

27. Multi-Disciplinary Skills Needed
    

    View Slide

28. How do we Create?
    

    View Slide

29. Model Inference
    

    View Slide

30. Bring your Deep Learning
    Model to the Data.
    

    View Slide

31. Abstract | Perceptual
    

    View Slide

32. Model Inference: Text9
    9 IMDB Sentiment with Keras.js
    

    View Slide

33. Model Inference: Images10
    10 Mnist with Keras.js
    

    View Slide

34. Model Inference: Style Transfer11
    11 Fast Neural Transfer with deeplearn.js
    

    View Slide

35. Model Inference: Image Augmentation
    

    View Slide

36. Semi-Supervised Learning12
    12 Quick Draw Crowd Sourced Data
    

    View Slide

37. Creative
    magenta.js | ml5js
    

    View Slide

38. Data vs. Model privacy
    

    View Slide

39. MODEL SIZE: quantization
    EMBEDDING SIZE: word2bit
    

    View Slide

40. Easier build for Web &
    Mobile
    

    View Slide

41. How do we
    Build?
    

    View Slide

42. Rapid
    Prototyping
    

    View Slide

43. Experiment and try new
    things
    

    View Slide

44. UI Based: Object Detection13
    13 Model Builder in Deeplearn.js
    

    View Slide

45. Immediate environment to
    write code
    

    View Slide

46. Reactive Notebooks: ObservableHQ14
    14 Reactive Notebook in the Browser
    

    View Slide

47. Challenges with Rapid Prototyping
    ▸ Training on GPUs (integration with Node.js)
    ▸ Custom layers not supported
    ▸ Lack of feature parity with Python / C++
    APIs
    

    View Slide

48. Improved JS Ecosystem
    ▸ Declarative visualisation: vega-lite.js
    ▸ Reactive Runtime: observablehq.com
    ▸ Columnar In-Memory: arrow.js
    

    View Slide

49. Way Forward
    

    View Slide

50. Stronger Ecosystem
    Stable Libraries
    Deeper Use Cases
    

    View Slide

51. Deep Learning in the Browser
    Explorables | Inference | Prototyping
    Amit Kapoor
    amitkaps.com
    

    View Slide