Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DeepNLP_BackPropagation_Rnn_and_Cnn
Search
izuna385
July 02, 2018
Science
0
170
DeepNLP_BackPropagation_Rnn_and_Cnn
深層学習による自然言語処理 2.5から2.9まで
izuna385
July 02, 2018
Tweet
Share
More Decks by izuna385
See All by izuna385
jel: japanese entity linker
izuna385
0
360
Firebase-React-App
izuna385
0
240
React+FastAPIを用いた簡単なWebアプリ作製
izuna385
0
1.7k
UseCase of Entity Linking
izuna385
0
560
Unofficial slides: From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains (ACL 2020)
izuna385
1
650
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
izuna385
0
850
Zero-shot Entity Linking with Dense Entity Retrieval (Unofficial slides) and Entity Linking future directions
izuna385
3
1.1k
Entity representation with relational attention
izuna385
0
83
Zero-Shot Entity Linking by Reading Entity Descriptions
izuna385
0
560
Other Decks in Science
See All in Science
機械学習 - DBSCAN
trycycle
PRO
0
860
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
0
120
02_西村訓弘_プログラムディレクター_人口減少を機にひらく未来社会.pdf
sip3ristex
0
450
データベース01: データベースを使わない世界
trycycle
PRO
1
640
統計学入門講座 第1回スライド
techmathproject
0
330
SpatialBiologyWestCoastUS2024
lcolladotor
0
130
機械学習 - K-means & 階層的クラスタリング
trycycle
PRO
0
820
生成検索エンジン最適化に関する研究の紹介
ynakano
2
1k
白金鉱業Meetup Vol.16_【初学者向け発表】 数理最適化のはじめの一歩 〜身近な問題で学ぶ最適化の面白さ〜
brainpadpr
11
2.2k
点群ライブラリPDALをGoogleColabにて実行する方法の紹介
kentaitakura
1
260
How To Buy, Verified Venmo Accounts in 2025 This year
usaallshop68
2
110
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
konakalab
0
200
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.8k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
106
19k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Product Roadmaps are Hard
iamctodd
PRO
53
11k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
6
680
Adopting Sorbet at Scale
ufuk
77
9.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Why Our Code Smells
bkeepers
PRO
337
57k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
123
52k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.8k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.6k
Transcript
5 . 1 2
• : D !(#) 1 ∇!(#) • L 1 )
) ( 1 S G G 1 & '()*+, = .(/ 012 .(⋯ .(/ 2 '()*+, + 5(2)))) 62 67 68 69 /(:) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ;2 ;7 /(:<2) = − 1 = = + 1
. . ! ℎ($) ℎ(&) ℎ(') (($) ((&) ((') ℎ(')=
( ' ( & ( $ ! *+(,) *- = *+(,) *.(/) 0 *.(/) *.(1) 0 *.(1) *- •
. . ! ℎ($) ℎ(&) '($) '(&)
. .
None
(
.( )
( )
2'+ )NN 2 #) 0%&1&-" (3*, )
.( !/$→ → Residual Connection, Batch Nomalization( ) ! Loss func ! Loss func
: Residual Connection –– F(x) (→-!()) F(x) + x
→ & " - Identity Mapping ' : -*&%,+' ./$ # Identity – [1] He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer, Cham, 2016. . .
. . (2.33) (2.34)
0 1 . 2 C2 2 " 3 2 3
2 ) 2 2 ( 3 23 2 !"#$ !" %"&$ %"#$ %" %"&$ '() '() '() '*+, '*+, -"#$ = /(!"#$ ) -" -"&$ %" !" M I NR 2 '*+, O L '() : input P / L !" = '*+, 2 !"#$ + '() %" ,, L
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )(
2 . 3 !" #$% &% !" #$' !" #$(
!% #$% !% #$' !% #$( !' #$% !' #$' !' #$( !( #$% !( #$' !( #$( &' &( )% )' )( • 23 !* # 1
: !"#$ !%& '( )( … … )*
'* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/
. - •
) (
8 99 2 :9.8 9 9 6 5 3 2
2 5 2 28 79 8 3 9 1 56 2 59 7 /0-
-5 1 02 1 25 58 8 ./ 8 .2
0 8
22/1 444 1 1 - 2 3--.-.3-. 11
http://deeplearning.stanford.edu/wiki/index .php/Feature_extraction_using_convolution
/885 6 0: 6. 5 5
RNN Vanishing/Exploding Gradient : !"#$ !%&
'( )( … … )* '* ………… ………… +( +* !"#$ (-) !%& (-) '% …… '/ )/ +/