Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
180
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
400
Firebase-React-App
izuna385
0
250
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.7k
UseCase of Entity Linking
izuna385
0
590
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
670
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
890
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
1.1k
Entity representation with relational attention
izuna385
0
86
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
570
Other Decks in Science
See All in Science
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
mfonobong
2
180
データマイニング - コミュニティ発見
trycycle
PRO
0
160
AIによる科学の加速: 各領域での革新と共創の未来
masayamoriofficial
0
170
地質研究者が苦労しながら運用する情報公開システムの実例
naito2000
0
290
防災デジタル分野での官民共創の取り組み (1)防災DX官民共創をどう進めるか
ditccsugii
0
290
データマイニング - ウェブとグラフ
trycycle
PRO
0
180
Ignite の1年間の軌跡
ktombow
0
160
mOrganic™ Holdings, LLC.
hyperlocalnetwork
0
120
知能とはなにかーヒトとAIのあいだー
tagtag
0
100
深層学習を用いた根菜類の個数カウントによる収量推定法の開発
kentaitakura
0
190
Explanatory material
yuki1986
0
410
2025-06-11-ai_belgium
sofievl
1
170
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
79
6k
A Modern Web Designer's Workflow
chriscoyier
697
190k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
37
2.6k
Building a Modern Day E-commerce SEO Strategy
aleyda
44
7.8k
Navigating Team Friction
lara
190
15k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
189
55k
For a Future-Friendly Web
brad_frost
180
9.9k
How GitHub (no longer) Works
holman
315
140k
Side Projects
sachag
455
43k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
132
19k
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/