Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
130
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
280
Firebase-React-App
izuna385
0
210
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.3k
UseCase of Entity Linking
izuna385
0
390
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
570
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
620
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
840
Entity representation with relational attention
izuna385
0
69
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
440
Other Decks in Science
See All in Science
【論文紹介】DocTr_ Document Transformer for Structured Information Extraction in Documents / iccv2023-doctr
yuya4
3
560
バックアップ『しながら』ランサムウェア検出も!? セキュリティ強化が満載 Veeam 12.1
climbteam
0
300
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
0
120
『データ可視化学入門』を PythonからRに翻訳した話
bob3bob3
1
360
FIBA W杯の日本代表って組み合わせ次第で2次ラウンド行けたんじゃね?をデータで検証
saltcooky12
0
200
Direct Preference Optimization
zchenry
0
120
勉強会資料 / “Asymptotic Statistics” Section 3.1
asymptotic_minato
0
110
OptimizationNight~機械学習と数理最適化の融合~
hidenari
0
290
文系出身でも「アルゴリズム×数学」はスッキリ理解できた!話
wakamatsu_takumu
0
200
遺伝子発現プロファイルに基づく新しい薬物間相互作用予測法
tagtag
0
100
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
0
260
2023-07-18_Verge_Genomics
lcolladotor
0
100
Featured
See All Featured
The Cult of Friendly URLs
andyhume
74
5.7k
GraphQLの誤解/rethinking-graphql
sonatard
50
9.2k
Scaling GitHub
holman
457
140k
The Invisible Side of Design
smashingmag
294
49k
Producing Creativity
orderedlist
PRO
336
39k
Designing Experiences People Love
moore
136
23k
What the flash - Photography Introduction
edds
64
11k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
1
1.3k
Embracing the Ebb and Flow
colly
79
4.1k
Teambox: Starting and Learning
jrom
128
8.4k
What's new in Ruby 2.0
geeforr
337
31k
Designing for Performance
lara
601
67k
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/