Deep Learning - an Introduction for Ruby Developers

Deep learning is exploding in popularity these days, powering everything from photo tagging to self driving cars – and you don’t need to hire a data science team or learn C++ to use machine learning models in your application. We'll learn some modern deep learning techniques, targeted at Ruby developers with no prior experience in the field. We’ll also cover the various tools available in the Ruby machine learning ecosystem.

Transcript

1. Deep Learning: An Introduction for Ruby Developers
2. @geoffreylitt
6. Machine learning = learning functions from data Machine learning
11. What we’ll cover today • Some fundamental intuition for how machine learning works • Recent developments in deep neural networks • Ruby tools for machine learning, including a new emerging project
13. Apartment  size  (sq ft) Monthly rent  ($) 500 1000 650 1200 800 1650 1000 2200 1050 2250 1200 2350 1300 2600
14. 0 500 1000 1500 2000 2500 3000 400 600 800 1000 1200 1400 Monthly  rent Apartment  size
17. 0 500 1000 1500 2000 2500 3000 400 600 800 1000 1200 1400 Monthly  rent Apartment  size m    =    -­‐1 b      =    2500 Error = High
18. 0 500 1000 1500 2000 2500 3000 400 600 800 1000 1200 1400 Monthly  rent Apartment  size m    =    0 b      =    1750 Error = Lower
19. 0 500 1000 1500 2000 2500 3000 400 600 800 1000 1200 1400 Monthly  rent Apartment  size m    =      2.2 b =      -­‐200 Error = Low
20. m b Cost (aka how far off are we?)
25. Image from http://neuralnetworksanddeeplearning.com/
27. inputs[0] inputs[1] inputs[2] output output
28. inputs[0] inputs[1] inputs[2] output output weights[0] weights[1] weights[2]
29. inputs[0] inputs[1] inputs[2] output output weights[0] weights[1] weights[2]
30. inputs[0] inputs[1] inputs[2] output output weights[0] weights[1] weights[2]
31. inputs[0] inputs[1] inputs[2] output output weights[0] weights[1] weights[2]
32. inputs[0] inputs[1] inputs[2] output output weights[0] weights[1] weights[2]
33. Image from http://cs231n.github.io/ 0.1    0.8    0.9    0.1 0.0    0.2    0.8    0.1 0.0    0.4    0.3    0.0 0.1    0.5    0.1    0.0 P(cat) P(dog) Learning Weights Weight updates flow backwards through the network Error = High
34. Image from http://cs231n.github.io/ 0.1    0.5    0.1    0.0 0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8 P(cat) P(dog) Learning Weights
35. Image from http://cs231n.github.io/ 0.1    0.5    0.1    0.0 0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8 P(cat) P(dog) Learning Weights Error = Lower
36. Image from http://cs231n.github.io/ 0.0    0.3    0.5    0.1 0.1    0.2    0.1    0.0 0.1    0.9    0.5    0.6 0.5    0.9    0.7    0.8 P(cat) P(dog) Learning Weights Error = Lower
37. Image from http://cs231n.github.io/ P(cat) P(dog) Learning Weights Error = Low 0.1    0.5    0.1    0.0 0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8
38. “Deep Learning” = more layers = more data and compute needed to train = more parameters to train = things we have a ton of in 2016 neural nets with…
39. Convolutional neural networks
40. Convolutional neural networks 0% 5% 10% 15% 20% 25% 30% 2011 2012 2013 2014 2015 ImageNet Error  rate Human  benchmark AlexNet,  2012 GoogLeNet,  2014
41. Recurrent neural networks Vinyals et  al  (2014).  Show  and  Tell:  A  Neural   Image  Caption  Generator. CoRR,  abs/1411.4555,  .
42. Recurrent neural networks Karpathy (2015).  Deep  Visual-­‐Semantic  Alignments   for  Generating  Image  Descriptions. CVPR
43. Recurrent neural networks http://karpathy.github.io/2015/05/21/rnn-­‐effectiveness/
44. Recurrent neural networks http://karpathy.github.io/2015/05/21/rnn-­‐effectiveness/
45. Deep Learning = Less Feature Engineering Fancy audio feature engineering Fancy text feature engineering
46. Deep Learning = Less Feature Engineering Early part of network learns features better than humans can create
48. Option 1: Roll your own
49. Source: https://blog.intercom.io/machine-learning-way-easier-than-it-looks/
50. - + • Avoid new dependencies • Flexibility • Understand your solution • Lots of work • Testing/debugging • Performance
51. Option 2: Use a high-level library
52. https://davidcel.is/recommendable/
54. - + • Quick results • No knowledge needed • Often basic algorithms • Only set use cases
55. Option 3: Use a low-level library
56. 0 20 40 60 80 100 120 140 160 180 200 Number of machine learning libraries • scipy • numpy • NLTK • scikit-learn • keras • theano • caffe • tensorflow Source: https://github.com/josephmisiti/awesome-machine-learning
58. ruby-fann (thin Ruby wrapper) libfann (written in C)
59. 0 2000 4000 6000 8000 10000 12000 14000 fann theano caffe Github Stars (9/5/16)
60. 0 5000 10000 15000 20000 25000 30000 35000 fann theano caffe tensorflow Github Stars (9/5/16) ?
65. C++ Core graph execution (C++) Python Ruby “The system includes front-ends for specifying TensorFlow computations in Python and C++, and we expect other front- ends to be added over time in response to the desires of both internal Google users and the broader open-source community.” SWIG
67. tensorflow.rb
69. Next steps… • Complete enough functionality to train basic neural networks • Gradient descent optimizer • Create easy install flow for dependencies • Add more test coverage • (Still far from production ready)
70. To learn more… • Andrew Ng’s Machine Learning Coursera class • Google’s Udacity Tensorflow class • Contributors to tensorflow.rb welcome!: • github.com/somaticio/tensorflow.rb • I’ll tweet slides: @geoffreylitt
71. Thanks!