Irene Chen - A Beginner's Guide to Deep Learning

What is deep learning? It has recently exploded in popularity as a complex and incredibly powerful tool. This talk will present the basic concepts underlying deep learning in understandable pieces for complete beginners to machine learning. We will review the math, code up a simple neural network, and provide contextual background on how deep learning is used in production now.

https://us.pycon.org/2016/schedule/presentation/2112/

Transcript

1. A BEGINNER’S GUIDE TO DEEP LEARNING Irene Chen @irenetrampoline PyCon 2016
2. “A beginner’s guide to deep learning”
3. “A beginner’s guide to deep learning”
4. Convolutional nets Backpropagation Image recognition Restricted BolNmann machines
5. DeepMind’s AlphaGo beating professional Go player Lee Sedol Nvidia and its latest GPU architecture Toyota’s $1 billion AI investment Facebook is building AI that builds AI
6. Geoff Hinton Yann LeCun Andrew Ng Yoshua Bengio
8. Too much math Too much code
9. Today • Why now? • Neural Networks in 7 minutes • Deep nets in Caffe
10. WHY NOW?
13. Engine (neural network)
14. Engine (neural network) Fuel (data)
15. Classifier
16. Classifier Input Output
17. Classifier Ripe?
18. Classifier Ripe?
19. Trained Classifier Ripe?
20. Logistic regression Naïve Bayes Support vector machine K-nearest neighbors Random forests
21. Trained Classifier Ripe?
23. Lesson 1: Why now? Big data, big processing power, robust neural networks
24. NEURAL NETWORKS IN 7
25. Photo: Rebecca-Lee (Flickr)
29. Input Nodes Output Nodes
30. Input Nodes Output Nodes
31. Input Nodes Output Nodes
32. Input Nodes Output Nodes A B
33. Input Nodes Output Nodes A B C
34. Wikipedia
35. Input Nodes Output Nodes A B C
36. Input Nodes Output Nodes A B C D
37. Input Nodes Output Nodes
38. Input Nodes Output Nodes Hidden layers
39. Input Nodes Output Nodes
40. Input Nodes Output Nodes
41. Input Nodes Output Nodes 10 0.5 200 4.1
42. Input Nodes Output Nodes 10 0.5 200 4.1
43. Input Nodes Output Nodes 10 0.5 200 4.1
44. Input Nodes Output Nodes 10 0.5 200 4.1
45. Input Nodes Output Nodes 10 0.5 200 4.1 95 17
46. Forward propagation
47. Input Nodes Output Nodes
48. Input Nodes Output Nodes
49. Input Nodes Output Nodes 10 0.5 200 4.1
50. Input Nodes Output Nodes 10 0.5 200 4.1 95 17
51. No randomness!
52. Input Nodes ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Output Nodes
54. Backpropagation
55. Input Nodes 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Output Nodes
56. Input Nodes Output Nodes 10 0.5 200 4.1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
57. Input Nodes Output Nodes 10 0.5 200 4.1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
58. Input Nodes Output Nodes 10 0.5 200 4.1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
59. Input Nodes Output Nodes 10 0.5 200 4.1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
60. Input Nodes Output Nodes 10 0.5 200 4.1 4 20
61. Input Nodes Output Nodes 10 0.5 200 4.1 4 20
62. Input Nodes Output Nodes 10 0.5 200 4.1 4 20
63. Input Nodes Output Nodes 10 0.5 200 4.1 5 19
65. Values of the nodes Amount of error Weights of edges Learning rate
66. Input Nodes Output Nodes 10 0.5 200 4.1 5 19
67. Input Nodes Output Nodes 10 0.5 200 4.1 5 19
68. Input Nodes Output Nodes 10 0.5 200 4.1 5 19
69. Input Nodes Output Nodes 10 0.5 200 4.1 5 19
72. Tuning parameters
73. Input Nodes Carlos Xavier Soto Output Nodes
74. Lesson 2: Neural networks can be trained on labeled data to classify avocados
75. DEEP NETS ON CAFFE
76. Scikit-learn Caffe Theano iPython Notebook
81. Scikit-learn Caffe Theano iPython Notebook
82. Loading a pre-trained network into Caffe
83. Large Scale Visual Recognition Challenge 2010 (ILSVRC 2010)
84. 10 million images 10,000 object classes 310,000 iterations
88. Tabby cat Tabby cat Tiger cat Egyptian cat Red fox Lynx
89. Lesson 3: Caffe provides pre-trained networks to jumpstart learning
90. Today • Lesson 1: Why now? Big data, big processing power, robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
91. What do you go from here?
92. Today • Lesson 1: Why now? Big data, big processing power, robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
93. Cuda implementations Theano, Tensorflow, etc
94. Today • Lesson 1: Why now? Big data, big processing power, robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
95. Restricted BolNmann Machines Recurrent network Convolutional network
96. Today • Lesson 1: Why now? Big data, big processing power, robust neural networks • Lesson 2: Neural networks can be trained on labeled data to classify avocados • Lesson 3: Caffe provides pre-trained networks to jumpstart learning
97. Caffe iPython notebooks Kaggle competitions
99. Thank you! irenetrampoline@gmail.com