Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
130
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
290
Firebase-React-App
izuna385
0
210
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.3k
UseCase of Entity Linking
izuna385
0
400
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
580
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
630
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
850
Entity representation with relational attention
izuna385
0
69
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
450
Other Decks in Science
See All in Science
AI Alignment: A Comprehensive Survey
s_ota
0
200
JSol'Ex : solar image processing in Java
melix
0
260
Cross-Media Information Spaces and Architectures (CISA)
signer
PRO
3
25k
脳とAIは似ているか ― NeuroAI の挑戦
ykamit
9
6.9k
データで課題を解決する -因果関係を調べる統計的因果推論-
sshimizu2006
4
1.4k
Microbiology Labs.
maleehafatima
0
140
救急外来でのめまい診療_中枢性めまいを見逃さない!
psasa
0
210
東大・松尾研主催 LLM Summer 2023 コンペ解法 (11位 – 20位枠での優秀賞)
hayataka88
0
190
[NeurIPS 2023 論文読み会] Wasserstein Quantum Monte Carlo
stakaya
0
360
拡散モデルの原理紹介
brainpadpr
1
1.1k
はじめての「相関と因果とエビデンス」入門:“動機づけられた推論” に抗うために
takehikoihayashi
15
5.5k
バックアップ『しながら』ランサムウェア検出も!? セキュリティ強化が満載 Veeam 12.1
climbteam
0
340
Featured
See All Featured
The Cost Of JavaScript in 2023
addyosmani
21
4k
The Invisible Side of Design
smashingmag
294
49k
A Tale of Four Properties
chriscoyier
153
22k
Optimising Largest Contentful Paint
csswizardry
13
2.4k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
126
32k
Embracing the Ebb and Flow
colly
80
4.2k
Music & Morning Musume
bryan
41
5.6k
The Brand Is Dead. Long Live the Brand.
mthomps
49
30k
The Pragmatic Product Professional
lauravandoore
26
5.9k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
275
13k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
YesSQL, Process and Tooling at Scale
rocio
165
13k
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/