Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Chapter 7
Shotaro Ishihara
April 18, 2018
Technology
0
260
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Chapter 7
http://bookclub.kodansha.co.jp/product?isbn=9784061529021
Shotaro Ishihara
April 18, 2018
Tweet
Share
More Decks by Shotaro Ishihara
See All by Shotaro Ishihara
Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning
upura
0
92
データ分析の進め方とニュースメディアでのデータ活用事例 / data-analysis-in-kaggle-and-news-media
upura
0
490
国際会議参加報告 AACL-IJCNLP 2022 / AACL-IJCNLP 2022 Report
upura
0
270
spoana 2022年の活動報告と 来年以降の企画募集 / spoana-2022
upura
0
500
Semantic Shift Stability: Efficient Way to Detect Performance Degradation of Word Embeddings and Pre-trained Language Models
upura
0
830
実践:日本語文章生成 Transformers ライブラリで学ぶ実装の守破離 / Introduction of Japanese Text Generation with Transformers
upura
5
7.6k
Nikkei at SemEval-2022 Task 8: Exploring BERT-based Bi-Encoder Approach for Pairwise Multilingual News Article Similarity
upura
0
440
[Poster] Nikkei at SemEval-2022 Task 8: Exploring BERT-based Bi-Encoder Approach for Pairwise Multilingual News Article Similarity
upura
0
400
新聞記事のクリック率予測に向けたペアワイズ学習用データセットの構築手法の検討 / JSAI2022 Generating Pairwise Dataset for CTR Prediction
upura
0
86
Other Decks in Technology
See All in Technology
SSMパラメーターストアでクロススタック参照の罠を回避する
shuyakinjo
0
7.8k
USB PD で迎える AC アダプター大統一時代
puhitaku
2
2k
OCI DevOps 概要 / OCI DevOps overview
oracle4engineer
PRO
0
510
SignalR を使ったアプリケーション開発をより快適に!
nenonaninu
0
670
目指せCoverage100%! AutoScale環境におけるSavings Plans購入戦略 / JAWS-UG_SRE_Coverage
taishin
0
520
ラズパイとGASで加湿器の消し忘れをLINEでリマインド&操作
minako__ph
0
150
S3とCloudWatch Logsの見直しから始めるコスト削減 / Cost saving S3 and CloudWatch Logs
shonansurvivors
0
260
NGINXENG JP#2 - 1-NGINX-エンジニアリング勉強会-きょうの見どころ
hiropo20
0
120
FlexScan HD2452Wの 後継を探して
tring
0
6.5k
SPA・SSGでSSRのようなOGP対応!
simo123
2
160
IoT から見る AWS re:invent 2022 ― AWSのIoTの歴史を添えて/Point of view the AWS re:invent 2022 with IoT - with a history of IoT in AWS
ma2shita
0
280
Multi-Cloud Gatewayでデータを統治せよ!/ Data Federation with MCG
tutsunom
1
340
Featured
See All Featured
Practical Orchestrator
shlominoach
178
8.9k
Art Directing for the Web. Five minutes with CSS Template Areas
malarkey
196
10k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
32
6.7k
The Illustrated Children's Guide to Kubernetes
chrisshort
22
43k
Web development in the modern age
philhawksworth
197
9.6k
YesSQL, Process and Tooling at Scale
rocio
159
12k
Embracing the Ebb and Flow
colly
75
3.6k
GraphQLとの向き合い方2022年版
quramy
20
9.9k
4 Signs Your Business is Dying
shpigford
171
20k
No one is an island. Learnings from fostering a developers community.
thoeni
12
1.5k
KATA
mclloyd
12
9.7k
Building Applications with DynamoDB
mza
85
5k
Transcript
7 2016/08/20 1
2 l RNN#' l RNN "
" l RNN & !( $%
3
4 We can get
an idea of the quality of the learned feature vectors by displaying them in a 2-D map.
5 $%"! '(Bag of Words ')N-gram
We can get an idea of the quality " #& or
6 l RNN#' l RNN "
" l RNN & !( $%
7 l RNN#' l RNN "
" l RNN & !( $%
RNN 8
RNN 9 x1 z0
RNN 10 z1 y1
RNN 11 x2 z1
RNN 12 z2 y2
13 l RNN#' l RNN "
" l RNN & !( $%
RNN 14 xt zt-1
y t →
RNN 15 xt zt-1 y
t →
16 l RNN#' l RNN "
" l RNN & !( $%
RNN 17 Back Propagation through time
BPTT 18 % x #!% d $ & y
, ... , y ' % δ ( δ ) * " t t 1 t k out, t j t
BPTT 19 δ k out, 1 δ k out, 2
δ k out, 3 δ k out, t
BPTT 20 t1 t δ
j t
BPTT 21
22 l RNN#' l RNN "
" l RNN & !( $%
23 l RNN#' l RNN "
" l RNN & !( $%
RNN 24 #@10+'<3= 0A; ← &91,?7 &9$)+/" ) 4
*58&90 or :( !.2- ← RNN%>264
LSTM 25 '% (Long Short-Term Memory, LSTM) RNN &# →
&# !$ (+) "*
LSTM 26
LSTM 27
LSTM 28
LSTM 29
30 l RNN#' l RNN "
" l RNN & !( $%
RNN 31 “w n” …… ^
(HMM) 32 %! $ "# $ "#
%!
33 $ .)-+ (Connectionist temporal classification, CTC) HMM#
! RNN &, %*"(, ' &,
CTC 34 X = x , ... ,
x l = l , … , l = p( l | X ) 1 t 1 |l|
CTC 35 l = ‘ab’ t = 6
a, b, , , , a, , , b, , , , , a, , b …
CTC 36 = p( l | X ) a, b,
, , , a, a, , b, , , , , a, , b … p( l1 | X ) = p( l2 | X ) = p( l3 | X ) = = p(a)*p(b)*p( )*p( ) *p( )*p( ) = p(a)*p(a)*p( )*p(b) *p( )*p( ) = p( )*p( )*p( )*p(a)*p( )*p(b)
37 • ;&B(2015):5:#3, .<2 • /%) in $"#3 E?!(2015):
http://www.slideshare.net/shotarosano5/chapter7-50542830, 2016A8*
[email protected]
C • Recurrent Neural Networks(2014): http://www.slideshare.net/beam2d/pfi-seminar- 20141030rnn?qid=9e5894c7-f162-4da3-b082-a1e4963689e8&v=&b=&from_search=17, 2016A8*
[email protected]
C • =86 (2013): 7+,4D19+,4D, 2 • LSTM 0(>-'(2016): http://qiita.com/t_Signull/items/21b82be280b46f467d1b, 2016A8*
[email protected]
C • A. Graves(2008): Supervised sequence labelling with Recurrent Neural Networks, PhD thesis, Technische Universität München, https://www.cs.toronto.edu/~graves/preprint.pdf