Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
150
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
330
Firebase-React-App
izuna385
0
230
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.6k
UseCase of Entity Linking
izuna385
0
530
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
640
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
810
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
1k
Entity representation with relational attention
izuna385
0
75
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
530
Other Decks in Science
See All in Science
いまAI組織が求める企画開発エンジニアとは?
roadroller
2
1.4k
サイゼミ用因果推論
lw
1
3.4k
Tensor Representations in Signal Processing and Machine Learning (Tutorial at APSIPA-ASC 2020)
yokotatsuya
0
120
最適化超入門
tkm2261
14
3.5k
240510 COGNAC LabChat
kazh
0
180
Planted Clique Conjectures are Equivalent
nobushimi
0
120
マテリアルズ・インフォマティクスの先端で起きていること / What's Happening at the Cutting Edge of Materials Informatics
snhryt
1
190
機械学習を支える連続最適化
nearme_tech
PRO
1
260
ウェーブレットおきもち講座
aikiriao
1
820
Celebrate UTIG: Staff and Student Awards 2024
utig
0
590
04_石井クンツ昌子_お茶の水女子大学理事_副学長_D_I社会実現へ向けて.pdf
sip3ristex
0
150
MoveItを使った産業用ロボット向け動作作成方法の紹介 / Introduction to creating motion for industrial robots using MoveIt
ry0_ka
0
310
Featured
See All Featured
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
The Cost Of JavaScript in 2023
addyosmani
47
7.4k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
129
19k
Making the Leap to Tech Lead
cromwellryan
133
9.1k
KATA
mclloyd
29
14k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
10
1.3k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5.2k
Why Our Code Smells
bkeepers
PRO
336
57k
Site-Speed That Sticks
csswizardry
4
410
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3k
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/