Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Gradient Descent Easy
Search
soulchild
July 23, 2014
Science
0
80
Gradient Descent Easy
Easy/brief version of Gradient Descent from Artificial Intelligence Lecture
soulchild
July 23, 2014
Tweet
Share
More Decks by soulchild
See All by soulchild
Similarities between macOS and iOS development
soulchild
0
120
N Tier Architecture for MMORPG
soulchild
0
91
Other Decks in Science
See All in Science
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
konakalab
0
210
生成検索エンジン最適化に関する研究の紹介
ynakano
2
1k
Collective Predictive Coding Hypothesis and Beyond (@Japanese Association for Philosophy of Science, 26th October 2024)
tanichu
0
130
機械学習 - DBSCAN
trycycle
PRO
0
890
機械学習 - 決定木からはじめる機械学習
trycycle
PRO
0
950
MCMCのR-hatは分散分析である
moricup
0
350
ほたるのひかり/RayTracingCamp10
kugimasa
1
710
実力評価性能を考慮した弓道高校生全国大会の大会制度設計の提案 / (konakalab presentation at MSS 2025.03)
konakalab
2
170
点群ライブラリPDALをGoogleColabにて実行する方法の紹介
kentaitakura
1
290
機械学習 - SVM
trycycle
PRO
1
820
統計的因果探索: 背景知識とデータにより因果仮説を探索する
sshimizu2006
4
910
06_浅井雄一郎_株式会社浅井農園代表取締役社長_紹介資料.pdf
sip3ristex
0
490
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.8k
GitHub's CSS Performance
jonrohan
1031
460k
Why You Should Never Use an ORM
jnunemaker
PRO
57
9.4k
The Cult of Friendly URLs
andyhume
79
6.5k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
490
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
The Invisible Side of Design
smashingmag
299
51k
The Language of Interfaces
destraynor
158
25k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
4 Signs Your Business is Dying
shpigford
184
22k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.5k
Transcript
Artificial Intelligence Gradient Descent soulchild
Gradient Descent Let computer find the minimum point in a
given graph or equation
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 Minimum point
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 Minimum point How does a computer know that this is a minimum point?
Gradient Descent Ans : By brute-forcing the derivative until a
value equal or near to 0 is found y = x2 dy dx = 2x Then guess x by starting from, eg: -6 to 6
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4 dy dx = 0
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 dy dx = -12 dy dx = —8 dy dx = -4 dy dx = 0 Minimum point found, stop
Gradient Descent A better way to brute force
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent y 0 10 20 30 40 x -6
-4 -2 0 2 4 6 y = x2 In short, it works like this
Gradient Descent How to set a good learning rate (α)
?
Gradient Descent When to stop searching ?
Gradient Descent When to stop searching ? Set a maximum
number of iteration
Gradient Descent When to stop searching ? Set a maximum
number of iteration dy dx < n When n can be 0.1, 0.01,.. etc
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0 then computer stop finding
Gradient Descent Weakness of Gradient Descent 0 3 6 9
12 -8 -6 -4 -2 0 2 4 let say start from here dy dx = 0 then computer stop finding What about this?! smaller than previous point wor
Gradient Descent Weakness of Gradient Descent Gradient Descent may stuck
in a local minima thus can’t find the global minima
Gradient Descent Q&A