Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Gradient Descent Easy
Search
soulchild
July 23, 2014
Science
0
80
Gradient Descent Easy
Easy/brief version of Gradient Descent from Artificial Intelligence Lecture
soulchild
July 23, 2014
Tweet
Share
More Decks by soulchild
See All by soulchild
Similarities between macOS and iOS development
soulchild
0
120
N Tier Architecture for MMORPG
soulchild
0
91
Other Decks in Science
See All in Science
機械学習 - 決定木からはじめる機械学習
trycycle
PRO
0
920
WCS-LA-2024
lcolladotor
0
230
機械学習 - K-means & 階層的クラスタリング
trycycle
PRO
0
810
アナログ計算機『計算尺』を愛でる Midosuji Tech #4/Analog Computing Device Slide Rule now and then
quiver
1
160
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
0
120
統計的因果探索: 背景知識とデータにより因果仮説を探索する
sshimizu2006
3
880
Agent開発フレームワークのOverviewとW&B Weaveとのインテグレーション
siyoo
0
210
Transport information Geometry: Current and Future II
lwc2017
0
140
MoveItを使った産業用ロボット向け動作作成方法の紹介 / Introduction to creating motion for industrial robots using MoveIt
ry0_ka
0
470
02_西村訓弘_プログラムディレクター_人口減少を機にひらく未来社会.pdf
sip3ristex
0
450
システム数理と応用分野の未来を切り拓くロードマップ・エンターテインメント(スポーツ)への応用 / Applied mathematics for sports entertainment
konakalab
1
330
証明支援系LEANに入門しよう
unaoya
0
1k
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
Thoughts on Productivity
jonyablonski
69
4.7k
The Cult of Friendly URLs
andyhume
78
6.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.7k
The Power of CSS Pseudo Elements
geoffreycrofte
76
5.8k
Scaling GitHub
holman
459
140k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.6k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Transcript
Artificial Intelligence Gradient Descent soulchild
Gradient Descent Let computer find the minimum point in a
given graph or equation
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 Minimum point
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 Minimum point How does a computer know that this is a minimum point?
Gradient Descent Ans : By brute-forcing the derivative until a
value equal or near to 0 is found y = x2 dy dx = 2x Then guess x by starting from, eg: -6 to 6
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4 dy dx = 0
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4 dy dx = 0 Minimum point found, stop
Gradient Descent A better way to brute force
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent How to set a good learning rate (α)
?
Gradient Descent When to stop searching ?
Gradient Descent When to stop searching ? Set a maximum
number of iteration
Gradient Descent When to stop searching ? Set a maximum
number of iteration dy dx < n When n can be 0.1, 0.01,.. etc
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0 then computer stop finding
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0 then computer stop finding What about this?! smaller than previous point wor
Gradient Descent Weakness of Gradient Descent Gradient Descent may stuck
in a local minima thus can’t find the global minima
Gradient Descent Q&A