Deep Learning

Introductory talk on deep learning

Transcript

1. D E E P L E A R N I N G
2. models AE / SAE RBM / DBN CNN RNN / LSTM Memnet / NTM agenda questions What ? Why ? How ? Next ?
3. what why how next What ? AI technique for learning multiple levels of abstractions directly from raw information
4. what why how next Primitive rule based AI Tailored systems Hand Crafted Program Output Input
5. what why how next Classical machine learning Learning from custom features Hand Crafted Features Learning System Output Input
6. what why how next Deep Learning based AI Learn everything Learned Features (Lower Level) Learned Features (Higher Level) Learning System Output Input
8. https://www.youtube.com/watch?v=Q70ulPJW3Gk PPTX  PDF (link to video below)
9. With the capacity to represent the world in signs and symbols, comes the capacity to change it Elizabeth Kolbert (The Sixth Extinction) “
10. Why The buzz ?
11. what why how next Google Trends Deep Learning
12. what why how next
13. Crude timeline of Neural Networks 1950 1980 1990 2000 Perceptron Backprop & Application NN Winter
14. 2010 Stacking RBMs Deep Learning fuss
15. HUGE DATA Large Synoptic Survey Telescope (2022) 30 TB/night
16. HUGE CAPABILITIES GPGPU  ~20x speedup Powerful Clusters
17. HUGE SUCCESS Speech, text understanding Robotics / Computer Vision Business / Big Data Artificial General Intelligence (AGI)
18. How its done ?
19. what why how next Shallow Network ℎ ℎ = (, 0) = ′(ℎ, 1) = (, ) minimize
20. what why how next Deep Network
21. what why how next Deep Network More abstract features Stellar performance Vanishing Gradient Overfitting
22. what why how next Autoencoder ℎ Unsupervised Feature Learning
23. what why how next Stacked Autoencoder Y. Bengio et. all; Greedy Layer-Wise Training of Deep Networks
24. what why how next Stacked Autoencoder 1. Unsupervised, layer by layer pretraining 2. Supervised fine tuning
25. what why how next Deep Belief Network 2006 breakthrough Stacking Restricted Boltzmann Machines (RBMs) Hinton, G. E., Osindero, S. and Teh, Y.; A fast learning algorithm for deep belief nets
26. Rethinking Computer Vision
27. what why how next Traditional Image Classification pipeline Feature Extraction (SIFT, SURF etc.) Classifier (SVM, NN etc.)
28. what why how next Convolutional Neural Network Images taken from deeplearning.net
29. what why how next Convolutional Neural Network
30. what why how next Convolutional Neural Network Images taken from deeplearning.net
31. what why how next Convolutional Neural Network
32. what why how next The Starry Night Vincent van Gogh Leon A. Gatys, Alexander S. Ecker and Matthias Bethge; A Neural Algorithm of Artistic Style
33. what why how next
34. what why how next Scene Description CNN + RNN Oriol Vinyals et. all; Show and Tell: A Neural Image Caption Generator
35. Learning Sequences
36. what why how next Recurrent Neural Network Simple Elman Version ℎ ℎ = ( , ℎ−1 , 0, 1) = ′(ℎ , 2)
37. what why how next Long Short Term Memory (LSTM) add memory cells learn access mechanism Sepp Hochreiter and Jürgen Schmidhuber; Long short-term memory
39. what why how next
40. what why how next Fooling Deep Networks Anh Nguyen, Jason Yosinski, Jeff Clune; Deep Neural Networks are Easily Fooled
41. Next Cool things to try
42. what why how next Hyperparameter optimization bayesian Optimization methods adadelta, rmsprop . . . Regularization dropout, dither . . .
43. what why how next Attention & Memory NTMs, Memory Networks, Stack RNNs . . . NLP Translation, description
44. what why how next Cognitive Hardware FPGA, GPU, Neuromorphic Chips Scalable DL map-reduce, compute clusters
45. what why how next Deep Reinforcement Learning deepmindish things, deep Q learning Energy models RBMs, DBNs . . .
46. https://www.reddit.com/r/MachineLearning/wiki
47. Theano (Python) | Torch (lua) | Caffe (C++) Github is a friend
48. @AbhinavTushar ?