Deep Learning

A64e058cacd2dbf5522c131f698a8dc6?s=47 Abhinav Tushar
September 10, 2015

Deep Learning

Introductory talk on deep learning


Abhinav Tushar

September 10, 2015


  1. D E E P L E A R N I

    N G
  2. models AE / SAE RBM / DBN CNN RNN /

    LSTM Memnet / NTM agenda questions What ? Why ? How ? Next ?
  3. what why how next What ? AI technique for learning

    multiple levels of abstractions directly from raw information
  4. what why how next Primitive rule based AI Tailored systems

    Hand Crafted Program Output Input
  5. what why how next Classical machine learning Learning from custom

    features Hand Crafted Features Learning System Output Input
  6. what why how next Deep Learning based AI Learn everything

    Learned Features (Lower Level) Learned Features (Higher Level) Learning System Output Input
  7. None
  8. PPTX  PDF (link to video below)

  9. With the capacity to represent the world in signs and

    symbols, comes the capacity to change it Elizabeth Kolbert (The Sixth Extinction) “
  10. Why The buzz ?

  11. what why how next Google Trends Deep Learning

  12. what why how next

  13. Crude timeline of Neural Networks 1950 1980 1990 2000 Perceptron

    Backprop & Application NN Winter
  14. 2010 Stacking RBMs Deep Learning fuss

  15. HUGE DATA Large Synoptic Survey Telescope (2022) 30 TB/night

  16. HUGE CAPABILITIES GPGPU  ~20x speedup Powerful Clusters

  17. HUGE SUCCESS Speech, text understanding Robotics / Computer Vision Business

    / Big Data Artificial General Intelligence (AGI)
  18. How its done ?

  19. what why how next Shallow Network ℎ ℎ = (,

    0) = ′(ℎ, 1) = (, ) minimize
  20. what why how next Deep Network

  21. what why how next Deep Network More abstract features Stellar

    performance Vanishing Gradient Overfitting
  22. what why how next Autoencoder ℎ Unsupervised Feature Learning

  23. what why how next Stacked Autoencoder Y. Bengio et. all;

    Greedy Layer-Wise Training of Deep Networks
  24. what why how next Stacked Autoencoder 1. Unsupervised, layer by

    layer pretraining 2. Supervised fine tuning
  25. what why how next Deep Belief Network 2006 breakthrough Stacking

    Restricted Boltzmann Machines (RBMs) Hinton, G. E., Osindero, S. and Teh, Y.; A fast learning algorithm for deep belief nets
  26. Rethinking Computer Vision

  27. what why how next Traditional Image Classification pipeline Feature Extraction

    (SIFT, SURF etc.) Classifier (SVM, NN etc.)
  28. what why how next Convolutional Neural Network Images taken from
  29. what why how next Convolutional Neural Network

  30. what why how next Convolutional Neural Network Images taken from
  31. what why how next Convolutional Neural Network

  32. what why how next The Starry Night Vincent van Gogh

    Leon A. Gatys, Alexander S. Ecker and Matthias Bethge; A Neural Algorithm of Artistic Style
  33. what why how next

  34. what why how next Scene Description CNN + RNN Oriol

    Vinyals et. all; Show and Tell: A Neural Image Caption Generator
  35. Learning Sequences

  36. what why how next Recurrent Neural Network Simple Elman Version

    ℎ ℎ = ( , ℎ−1 , 0, 1) = ′(ℎ , 2)
  37. what why how next Long Short Term Memory (LSTM) add

    memory cells learn access mechanism Sepp Hochreiter and Jürgen Schmidhuber; Long short-term memory
  38. None
  39. what why how next

  40. what why how next Fooling Deep Networks Anh Nguyen, Jason

    Yosinski, Jeff Clune; Deep Neural Networks are Easily Fooled
  41. Next Cool things to try

  42. what why how next Hyperparameter optimization bayesian Optimization methods adadelta,

    rmsprop . . . Regularization dropout, dither . . .
  43. what why how next Attention & Memory NTMs, Memory Networks,

    Stack RNNs . . . NLP Translation, description
  44. what why how next Cognitive Hardware FPGA, GPU, Neuromorphic Chips

    Scalable DL map-reduce, compute clusters
  45. what why how next Deep Reinforcement Learning deepmindish things, deep

    Q learning Energy models RBMs, DBNs . . .

  47. Theano (Python) | Torch (lua) | Caffe (C++) Github is

    a friend
  48. @AbhinavTushar ?