Irene Chen - A Beginner's Guide to Deep Learning

Irene Chen - A Beginner's Guide to Deep Learning

What is deep learning? It has recently exploded in popularity as a complex and incredibly powerful tool. This talk will present the basic concepts underlying deep learning in understandable pieces for complete beginners to machine learning. We will review the math, code up a simple neural network, and provide contextual background on how deep learning is used in production now.

https://us.pycon.org/2016/schedule/presentation/2112/

Eec9d25835717f1f1f12a354faf68d87?s=128

PyCon 2016

May 29, 2016
Tweet

Transcript

  1. A BEGINNER’S GUIDE TO DEEP LEARNING Irene Chen @irenetrampoline PyCon

    2016
  2. “A beginner’s guide to deep learning”

  3. “A beginner’s guide to deep learning”

  4. Convolutional nets Backpropagation Image recognition Restricted BolNmann machines

  5. DeepMind’s AlphaGo beating professional Go player Lee Sedol Nvidia and

    its latest GPU architecture Toyota’s $1 billion AI investment Facebook is building AI that builds AI
  6. Geoff Hinton Yann LeCun Andrew Ng Yoshua Bengio

  7. None
  8. Too much math Too much code

  9. Today • Why now? • Neural Networks in 7 minutes • Deep nets

    in Caffe
  10. WHY NOW?

  11. None
  12. None
  13. Engine (neural network)

  14. Engine (neural network) Fuel (data)

  15. Classifier

  16. Classifier Input Output

  17. Classifier Ripe?

  18. Classifier Ripe?

  19. Trained Classifier Ripe?

  20. Logistic regression Naïve Bayes Support vector machine K-nearest neighbors Random

    forests
  21. Trained Classifier Ripe?

  22. None
  23. Lesson 1: Why now? Big data, big processing power, robust

    neural networks
  24. NEURAL NETWORKS IN 7

  25. Photo: Rebecca-Lee (Flickr)

  26. None
  27. None
  28. None
  29. Input Nodes Output Nodes

  30. Input Nodes Output Nodes

  31. Input Nodes Output Nodes

  32. Input Nodes Output Nodes A B

  33. Input Nodes Output Nodes A B C

  34. Wikipedia

  35. Input Nodes Output Nodes A B C

  36. Input Nodes Output Nodes A B C D

  37. Input Nodes Output Nodes

  38. Input Nodes Output Nodes Hidden layers

  39. Input Nodes Output Nodes

  40. Input Nodes Output Nodes

  41. Input Nodes Output Nodes 10 0.5 200 4.1

  42. Input Nodes Output Nodes 10 0.5 200 4.1

  43. Input Nodes Output Nodes 10 0.5 200 4.1

  44. Input Nodes Output Nodes 10 0.5 200 4.1

  45. Input Nodes Output Nodes 10 0.5 200 4.1 95 17

  46. Forward propagation

  47. Input Nodes Output Nodes

  48. Input Nodes Output Nodes

  49. Input Nodes Output Nodes 10 0.5 200 4.1

  50. Input Nodes Output Nodes 10 0.5 200 4.1 95 17

  51. No randomness!

  52. Input Nodes ? ? ? ? ? ? ? ?

    ? ? ? ? ? ? ? Output Nodes
  53. None
  54. Backpropagation

  55. Input Nodes 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 Output Nodes
  56. Input Nodes Output Nodes 10 0.5 200 4.1 1 1

    1 1 1 1 1 1 1 1 1 1 1 1 1
  57. Input Nodes Output Nodes 10 0.5 200 4.1 1 1

    1 1 1 1 1 1 1 1 1 1 1 1 1
  58. Input Nodes Output Nodes 10 0.5 200 4.1 1 1

    1 1 1 1 1 1 1 1 1 1 1 1 1
  59. Input Nodes Output Nodes 10 0.5 200 4.1 1 1

    1 1 1 1 1 1 1 1 1 1 1 1 1
  60. Input Nodes Output Nodes 10 0.5 200 4.1 4 20

  61. Input Nodes Output Nodes 10 0.5 200 4.1 4 20

  62. Input Nodes Output Nodes 10 0.5 200 4.1 4 20

  63. Input Nodes Output Nodes 10 0.5 200 4.1 5 19

  64. None
  65. Values of the nodes Amount of error Weights of edges

    Learning rate
  66. Input Nodes Output Nodes 10 0.5 200 4.1 5 19

  67. Input Nodes Output Nodes 10 0.5 200 4.1 5 19

  68. Input Nodes Output Nodes 10 0.5 200 4.1 5 19

  69. Input Nodes Output Nodes 10 0.5 200 4.1 5 19

  70. None
  71. None
  72. Tuning parameters

  73. Input Nodes Carlos Xavier Soto Output Nodes

  74. Lesson 2: Neural networks can be trained on labeled data

    to classify avocados
  75. DEEP NETS ON CAFFE

  76. Scikit-learn Caffe Theano iPython Notebook

  77. None
  78. None
  79. None
  80. None
  81. Scikit-learn Caffe Theano iPython Notebook

  82. Loading a pre-trained network into Caffe

  83. Large Scale Visual Recognition Challenge 2010 (ILSVRC 2010)

  84. 10 million images 10,000 object classes 310,000 iterations

  85. None
  86. None
  87. None
  88. Tabby cat Tabby cat Tiger cat Egyptian cat Red fox

    Lynx
  89. Lesson 3: Caffe provides pre-trained networks to jumpstart learning

  90. Today • Lesson 1: Why now? Big data, big processing power,

    robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
  91. What do you go from here?

  92. Today • Lesson 1: Why now? Big data, big processing power,

    robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
  93. Cuda implementations Theano, Tensorflow, etc

  94. Today • Lesson 1: Why now? Big data, big processing power,

    robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
  95. Restricted BolNmann Machines Recurrent network Convolutional network

  96. Today • Lesson 1: Why now? Big data, big processing power,

    robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
  97. Caffe iPython notebooks Kaggle competitions

  98. None
  99. Thank you! irenetrampoline@gmail.com