Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
170
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
370
Firebase-React-App
izuna385
0
250
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.7k
UseCase of Entity Linking
izuna385
0
570
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
660
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
860
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
1.1k
Entity representation with relational attention
izuna385
0
83
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
560
Other Decks in Science
See All in Science
機械学習 - DBSCAN
trycycle
PRO
0
930
データベース08: 実体関連モデルとは?
trycycle
PRO
0
750
2025-06-11-ai_belgium
sofievl
1
130
深層学習を用いた根菜類の個数カウントによる収量推定法の開発
kentaitakura
0
170
CV_5_3dVision
hachama
0
140
白金鉱業Meetup Vol.16_数理最適化案件のはじめかた・すすめかた
brainpadpr
4
1.8k
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
mfonobong
2
130
IWASAKI Hideo
genomethica
0
120
サイゼミ用因果推論
lw
1
7.4k
Ignite の1年間の軌跡
ktombow
0
130
機械学習 - pandas入門
trycycle
PRO
0
280
baseballrによるMLBデータの抽出と階層ベイズモデルによる打率の推定 / TokyoR118
dropout009
1
500
Featured
See All Featured
Building an army of robots
kneath
306
45k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
KATA
mclloyd
30
14k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
It's Worth the Effort
3n
185
28k
Site-Speed That Sticks
csswizardry
10
700
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/