Deep Learning - an Introduction for Ruby Developers

7a6310a990212e2b392d95a67855afc2?s=47 Geoffrey Litt
September 13, 2016

Deep Learning - an Introduction for Ruby Developers

Deep learning is exploding in popularity these days, powering everything from photo tagging to self driving cars – and you don’t need to hire a data science team or learn C++ to use machine learning models in your application. We'll learn some modern deep learning techniques, targeted at Ruby developers with no prior experience in the field. We’ll also cover the various tools available in the Ruby machine learning ecosystem.

7a6310a990212e2b392d95a67855afc2?s=128

Geoffrey Litt

September 13, 2016
Tweet

Transcript

  1. 3.
  2. 4.
  3. 5.
  4. 7.
  5. 8.
  6. 9.
  7. 10.
  8. 11.

    What we’ll cover today • Some fundamental intuition for how

    machine learning works • Recent developments in deep neural networks • Ruby tools for machine learning, including a new emerging project
  9. 12.
  10. 13.

    Apartment  size  (sq ft) Monthly rent  ($) 500 1000 650

    1200 800 1650 1000 2200 1050 2250 1200 2350 1300 2600
  11. 14.

    0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size
  12. 15.
  13. 16.
  14. 17.

    0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size m    =    -­‐1 b      =    2500 Error = High
  15. 18.

    0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size m    =    0 b      =    1750 Error = Lower
  16. 19.

    0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size m    =      2.2 b =      -­‐200 Error = Low
  17. 21.
  18. 22.
  19. 23.
  20. 24.
  21. 26.
  22. 33.

    Image from http://cs231n.github.io/ 0.1    0.8    0.9    0.1

    0.0    0.2    0.8    0.1 0.0    0.4    0.3    0.0 0.1    0.5    0.1    0.0 P(cat) P(dog) Learning Weights Weight updates flow backwards through the network Error = High
  23. 34.

    Image from http://cs231n.github.io/ 0.1    0.5    0.1    0.0

    0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8 P(cat) P(dog) Learning Weights
  24. 35.

    Image from http://cs231n.github.io/ 0.1    0.5    0.1    0.0

    0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8 P(cat) P(dog) Learning Weights Error = Lower
  25. 36.

    Image from http://cs231n.github.io/ 0.0    0.3    0.5    0.1

    0.1    0.2    0.1    0.0 0.1    0.9    0.5    0.6 0.5    0.9    0.7    0.8 P(cat) P(dog) Learning Weights Error = Lower
  26. 37.

    Image from http://cs231n.github.io/ P(cat) P(dog) Learning Weights Error = Low

    0.1    0.5    0.1    0.0 0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8
  27. 38.

    “Deep Learning” = more layers = more data and compute

    needed to train = more parameters to train = things we have a ton of in 2016 neural nets with…
  28. 40.

    Convolutional neural networks 0% 5% 10% 15% 20% 25% 30%

    2011 2012 2013 2014 2015 ImageNet Error  rate Human  benchmark AlexNet,  2012 GoogLeNet,  2014
  29. 41.

    Recurrent neural networks Vinyals et  al  (2014).  Show  and  Tell:

     A  Neural   Image  Caption  Generator. CoRR,  abs/1411.4555,  .
  30. 46.

    Deep Learning = Less Feature Engineering Early part of network

    learns features better than humans can create
  31. 47.
  32. 50.

    - + • Avoid new dependencies • Flexibility • Understand

    your solution • Lots of work • Testing/debugging • Performance
  33. 53.
  34. 54.

    - + • Quick results • No knowledge needed •

    Often basic algorithms • Only set use cases
  35. 56.

    0 20 40 60 80 100 120 140 160 180

    200 Number of machine learning libraries • scipy • numpy • NLTK • scikit-learn • keras • theano • caffe • tensorflow Source: https://github.com/josephmisiti/awesome-machine-learning
  36. 57.
  37. 59.
  38. 60.

    0 5000 10000 15000 20000 25000 30000 35000 fann theano

    caffe tensorflow Github Stars (9/5/16) ?
  39. 61.
  40. 62.
  41. 63.
  42. 64.
  43. 65.

    C++ Core graph execution (C++) Python Ruby “The system includes

    front-ends for specifying TensorFlow computations in Python and C++, and we expect other front- ends to be added over time in response to the desires of both internal Google users and the broader open-source community.” SWIG
  44. 66.
  45. 68.
  46. 69.

    Next steps… • Complete enough functionality to train basic neural

    networks • Gradient descent optimizer • Create easy install flow for dependencies • Add more test coverage • (Still far from production ready)
  47. 70.

    To learn more… • Andrew Ng’s Machine Learning Coursera class

    • Google’s Udacity Tensorflow class • Contributors to tensorflow.rb welcome!: • github.com/somaticio/tensorflow.rb • I’ll tweet slides: @geoffreylitt
  48. 71.