$30 off During Our Annual Pro Sale. View Details »

Irene Chen - A Beginner's Guide to Deep Learning

PyCon 2016
May 29, 2016
Programming
10
2.5k

Irene Chen - A Beginner's Guide to Deep Learning

What is deep learning? It has recently exploded in popularity as a complex and incredibly powerful tool. This talk will present the basic concepts underlying deep learning in understandable pieces for complete beginners to machine learning. We will review the math, code up a simple neural network, and provide contextual background on how deep learning is used in production now.

https://us.pycon.org/2016/schedule/presentation/2112/

PyCon 2016

May 29, 2016
Tweet

More Decks by PyCon 2016

See All by PyCon 2016
Kelsey Gilmore-Innis - Seriously Strong Security on a Shoestring
pycon2016
6
840
Manuel Ebert - Putting 1 million new words into the dictionary
pycon2016
6
870
Brett Slatkin - Refactoring Python: Why and how to restructure your code
pycon2016
17
5.2k
Mike Graham - The Life Cycle of a Python Class
pycon2016
7
8.5k
Nathaniel Manista, Augie Fackler - Code Unto Others
pycon2016
0
470
Alex Gaynor - The cobbler's children have no shoes, or building better tools for ourselves
pycon2016
0
610
Adrienne Lowe - Bake the Cookies, Wear the Dress: Connecting with Confident Authenticity
pycon2016
0
310
Jake Vanderplas - Statistics for Hackers
pycon2016
17
4.5k
Daniele Procida - Documentation-driven development - lessons from the Django Project
pycon2016
3
790

Other Decks in Programming

See All in Programming
エンジニアもUIを作るならユーザーについて知ろう！
ohakutsu
1
120
Kaggle-Vesuvius-Challenge-Ink-Detection-2nd-Solution
mipypf
0
1k
DIGGLEの分析基盤アーキテクチャ
zakky21
0
150
GoにおけるCall Graphを用いた 大規模コードベースの影響調査
ffjlabo
0
110
BigQuery のデータ品質やデータ活用を高める Dataplex 等の活用
mot_techtalk
4
940
「価値」あるものを作るためにエンジニアができること - NIFTY Tech Day 2023
niftycorp
0
120
The Secret Ingredient: How to Understand and Resolve Just about Any Flaky Test
aridlehoover
PRO
1
230
Developing a Vim plugin with Ruby
okuramasafumi
0
220
Using AI in Software Design: How ChatGPT Can Help With Creating a Solution Architecture
rdmueller
0
170
Intro to Stateful Services or How to get 1 million RPS from a single node
antyadev
0
140
Cloudflare Workers は楽しい！
codehex
8
2.2k
動くコードを書こう / let's code a process
kishida
21
6.1k

Featured

See All Featured
Building a Scalable Design System with Sketch
lauravandoore
453
32k
Web development in the modern age
philhawksworth
201
9.9k
Infographics Made Easy
chrislema
236
17k
Side Projects
sachag
450
39k
We Have a Design System, Now What?
morganepeng
41
6.5k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
113
17k
Build your cross-platform service in a week with App Engine
jlugia
223
17k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
271
12k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
123
31k
Put a Button on it: Removing Barriers to Going Fast.
kastner
56
2.8k
GraphQLの誤解/rethinking-graphql
sonatard
45
8.7k
Learning to Love Humans: Emotional Interface Design
aarron
265
39k

Transcript

  1. A BEGINNER’S GUIDE
    TO DEEP LEARNING
    Irene Chen
    @irenetrampoline
    PyCon 2016

    View Slide

  2. “A beginner’s guide to
    deep learning”

    View Slide

  3. “A beginner’s guide to
    deep learning”

    View Slide

  4. Convolutional nets
    Backpropagation
    Image recognition
    Restricted BolNmann machines

    View Slide

  5. DeepMind’s AlphaGo beating
    professional Go player Lee Sedol
    Nvidia and its latest GPU architecture
    Toyota’s $1 billion AI investment
    Facebook is building AI that builds AI

    View Slide

  6. Geoff Hinton
    Yann LeCun
    Andrew Ng
    Yoshua Bengio

    View Slide

  7. View Slide

  8. Too much math Too much code

    View Slide

  9. Today
    • Why now?
    • Neural Networks in 7 minutes
    • Deep nets in Caffe

    View Slide

  10. WHY NOW?

    View Slide

  11. View Slide

  12. View Slide

  13. Engine (neural
    network)

    View Slide

  14. Engine (neural
    network)
    Fuel
    (data)

    View Slide

  15. Classifier

    View Slide

  16. Classifier
    Input Output

    View Slide

  17. Classifier Ripe?

    View Slide

  18. Classifier Ripe?

    View Slide

  19. Trained Classifier
    Ripe?

    View Slide

  20. Logistic regression
    Naïve Bayes
    Support vector machine
    K-nearest neighbors
    Random forests

    View Slide

  21. Trained Classifier
    Ripe?

    View Slide

  22. View Slide

  23. Lesson 1: Why now? Big
    data, big processing power,
    robust neural networks

    View Slide

  24. NEURAL NETWORKS IN 7

    View Slide

  25. Photo: Rebecca-Lee (Flickr)

    View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. Input Nodes
    Output Nodes

    View Slide

  30. Input Nodes
    Output Nodes

    View Slide

  31. Input Nodes
    Output Nodes

    View Slide

  32. Input Nodes
    Output Nodes
    A
    B

    View Slide

  33. Input Nodes
    Output Nodes
    A
    B
    C

    View Slide

  34. Wikipedia

    View Slide

  35. Input Nodes
    Output Nodes
    A
    B
    C

    View Slide

  36. Input Nodes
    Output Nodes
    A
    B
    C D

    View Slide

  37. Input Nodes
    Output Nodes

    View Slide

  38. Input Nodes
    Output
    Nodes
    Hidden
    layers

    View Slide

  39. Input Nodes
    Output Nodes

    View Slide

  40. Input Nodes
    Output Nodes

    View Slide

  41. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1

    View Slide

  42. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1

    View Slide

  43. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1

    View Slide

  44. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1

    View Slide

  45. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    95
    17

    View Slide

  46. Forward propagation

    View Slide

  47. Input Nodes
    Output Nodes

    View Slide

  48. Input Nodes
    Output Nodes

    View Slide

  49. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1

    View Slide

  50. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    95
    17

    View Slide

  51. No randomness!

    View Slide

  52. Input Nodes
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    ?
    Output Nodes

    View Slide

  53. View Slide

  54. Backpropagation

    View Slide

  55. Input Nodes
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    Output Nodes

    View Slide

  56. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1

    View Slide

  57. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1

    View Slide

  58. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1

    View Slide

  59. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1
    1

    View Slide

  60. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    4
    20

    View Slide

  61. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    4
    20

    View Slide

  62. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    4
    20

    View Slide

  63. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    5
    19

    View Slide

  64. View Slide

  65. Values of the nodes
    Amount of error
    Weights of edges
    Learning rate

    View Slide

  66. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    5
    19

    View Slide

  67. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    5
    19

    View Slide

  68. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    5
    19

    View Slide

  69. Input Nodes
    Output Nodes
    10
    0.5
    200
    4.1
    5
    19

    View Slide

  70. View Slide

  71. View Slide

  72. Tuning parameters

    View Slide

  73. Input Nodes
    Carlos Xavier Soto
    Output Nodes

    View Slide

  74. Lesson 2: Neural networks
    can be trained on labeled
    data to classify avocados

    View Slide

  75. DEEP NETS ON CAFFE

    View Slide

  76. Scikit-learn
    Caffe
    Theano
    iPython Notebook

    View Slide

  77. View Slide

  78. View Slide

  79. View Slide

  80. View Slide

  81. Scikit-learn
    Caffe
    Theano
    iPython Notebook

    View Slide

  82. Loading a pre-trained
    network into Caffe

    View Slide

  83. Large Scale Visual
    Recognition Challenge 2010
    (ILSVRC 2010)

    View Slide

  84. 10 million images
    10,000 object classes
    310,000 iterations

    View Slide

  85. View Slide

  86. View Slide

  87. View Slide

  88. Tabby cat
    Tabby cat
    Tiger cat
    Egyptian cat
    Red fox
    Lynx

    View Slide

  89. Lesson 3: Caffe provides
    pre-trained networks to
    jumpstart learning

    View Slide

  90. Today
    • Lesson 1: Why now? Big data, big
    processing power, robust neural
    networks
    • Lesson 2: Neural networks can be
    trained on labeled data to classify
    avocados
    • Lesson 3: Caffe provides pre-trained
    networks to jumpstart learning

    View Slide

  91. What do you go from here?

    View Slide

  92. Today
    • Lesson 1: Why now? Big data, big
    processing power, robust neural
    networks
    • Lesson 2: Neural networks can be
    trained on labeled data to classify
    avocados
    • Lesson 3: Caffe provides pre-trained
    networks to jumpstart learning

    View Slide

  93. Cuda implementations
    Theano, Tensorflow, etc

    View Slide

  94. Today
    • Lesson 1: Why now? Big data, big
    processing power, robust neural
    networks
    • Lesson 2: Neural networks can be
    trained on labeled data to classify
    avocados
    • Lesson 3: Caffe provides pre-trained
    networks to jumpstart learning

    View Slide

  95. Restricted BolNmann Machines
    Recurrent network
    Convolutional network

    View Slide

  96. Today
    • Lesson 1: Why now? Big data, big
    processing power, robust neural
    networks
    • Lesson 2: Neural networks can be
    trained on labeled data to classify
    avocados
    • Lesson 3: Caffe provides pre-trained
    networks to jumpstart learning

    View Slide

  97. Caffe iPython notebooks
    Kaggle competitions

    View Slide

  98. View Slide

  99. Thank you!
    [email protected]

    View Slide