Deep Learning

Transcript

1. D E E P L E A R N I

   N G

2. models AE / SAE RBM / DBN CNN RNN /

   LSTM Memnet / NTM agenda questions What ? Why ? How ? Next ?

3. what why how next What ? AI technique for learning

   multiple levels of abstractions directly from raw information

4. what why how next Primitive rule based AI Tailored systems

   Hand Crafted Program Output Input

5. what why how next Classical machine learning Learning from custom

   features Hand Crafted Features Learning System Output Input

6. what why how next Deep Learning based AI Learn everything

   Learned Features (Lower Level) Learned Features (Higher Level) Learning System Output Input
7. None

8. https://www.youtube.com/watch?v=Q70ulPJW3Gk PPTX  PDF (link to video below)

9. With the capacity to represent the world in signs and

   symbols, comes the capacity to change it Elizabeth Kolbert (The Sixth Extinction) “

10. Why The buzz ?

11. what why how next Google Trends Deep Learning

12. what why how next

13. Crude timeline of Neural Networks 1950 1980 1990 2000 Perceptron

    Backprop & Application NN Winter

14. 2010 Stacking RBMs Deep Learning fuss

15. HUGE DATA Large Synoptic Survey Telescope (2022) 30 TB/night

16. HUGE CAPABILITIES GPGPU  ~20x speedup Powerful Clusters

17. HUGE SUCCESS Speech, text understanding Robotics / Computer Vision Business

    / Big Data Artificial General Intelligence (AGI)

18. How its done ?

19. what why how next Shallow Network ℎ ℎ = (,

    0) = ′(ℎ, 1) = (, ) minimize

20. what why how next Deep Network

21. what why how next Deep Network More abstract features Stellar

    performance Vanishing Gradient Overfitting

22. what why how next Autoencoder ℎ Unsupervised Feature Learning

23. what why how next Stacked Autoencoder Y. Bengio et. all;

    Greedy Layer-Wise Training of Deep Networks

24. what why how next Stacked Autoencoder 1. Unsupervised, layer by

    layer pretraining 2. Supervised fine tuning

25. what why how next Deep Belief Network 2006 breakthrough Stacking

    Restricted Boltzmann Machines (RBMs) Hinton, G. E., Osindero, S. and Teh, Y.; A fast learning algorithm for deep belief nets

26. Rethinking Computer Vision

27. what why how next Traditional Image Classification pipeline Feature Extraction

    (SIFT, SURF etc.) Classifier (SVM, NN etc.)

28. what why how next Convolutional Neural Network Images taken from

    deeplearning.net

29. what why how next Convolutional Neural Network

30. what why how next Convolutional Neural Network Images taken from

    deeplearning.net

31. what why how next Convolutional Neural Network

32. what why how next The Starry Night Vincent van Gogh

    Leon A. Gatys, Alexander S. Ecker and Matthias Bethge; A Neural Algorithm of Artistic Style

33. what why how next

34. what why how next Scene Description CNN + RNN Oriol

    Vinyals et. all; Show and Tell: A Neural Image Caption Generator

35. Learning Sequences

36. what why how next Recurrent Neural Network Simple Elman Version

    ℎ ℎ = ( , ℎ−1 , 0, 1) = ′(ℎ , 2)

37. what why how next Long Short Term Memory (LSTM) add

    memory cells learn access mechanism Sepp Hochreiter and Jürgen Schmidhuber; Long short-term memory
38. None

39. what why how next

40. what why how next Fooling Deep Networks Anh Nguyen, Jason

    Yosinski, Jeff Clune; Deep Neural Networks are Easily Fooled

41. Next Cool things to try

42. what why how next Hyperparameter optimization bayesian Optimization methods adadelta,

    rmsprop . . . Regularization dropout, dither . . .

43. what why how next Attention & Memory NTMs, Memory Networks,

    Stack RNNs . . . NLP Translation, description

44. what why how next Cognitive Hardware FPGA, GPU, Neuromorphic Chips

    Scalable DL map-reduce, compute clusters

45. what why how next Deep Reinforcement Learning deepmindish things, deep

    Q learning Energy models RBMs, DBNs . . .

46. https://www.reddit.com/r/MachineLearning/wiki

47. Theano (Python) | Torch (lua) | Caffe (C++) Github is

    a friend

48. @AbhinavTushar ?