Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Chapter 7
Search
Shotaro Ishihara
April 18, 2018
Technology
0
280
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Chapter 7
http://bookclub.kodansha.co.jp/product?isbn=9784061529021
Shotaro Ishihara
April 18, 2018
Tweet
Share
More Decks by Shotaro Ishihara
See All by Shotaro Ishihara
「極意本」サンプルコードをクラウド上で動かそう
upura
1
1.7k
論文紹介: Generating News-Centric Crossword Puzzles As A Constraint Satisfaction and Optimization Problem
upura
0
150
関東 Kaggler 会スポンサー資料
upura
0
1.4k
論文紹介 Quantifying attention via dwell time and engagement in a social media browsing environment / web-socialmedia-study-8th
upura
0
170
Quantifying Diachronic Language Change via Word Embeddings: Analysis of Social Events using 11 Years News Articles in Japanese and English
upura
1
340
Training Data Extraction From Pre-trained Language Models: A Survey
upura
0
180
論文紹介 Discovering and Categorising Language Biases in Reddit / web-socialmedia-study-5th
upura
0
280
AMA (Ask me anything) 『Kaggleに挑む深層学習プログラミングの極意』 / Ask me anything
upura
0
240
著者による書籍紹介『Kaggleに挑む深層学習プログラミングの極意』
upura
2
2k
Other Decks in Technology
See All in Technology
コンパウンドスタートアップのためのスケーラブルでセキュアなInfrastructure as Codeパイプラインを考える / Scalable and Secure Infrastructure as Code Pipeline for a Compound Startup
yuyatakeyama
3
1.7k
ユーザーストーリーのレビューを自動化したみたの
bun913
1
240
Signals Unleashed: The Full Guide
rainerhahnekamp
0
350
検証を通して見えてきたTiDBの性能特性
lycorptech_jp
PRO
2
830
強みを伸ばすキャリアデザイン
yug1224
0
200
AIQ株式会社 エンジニア向け会社紹介資料
aiqlab
0
340
Tebiki株式会社 エンジニア採用資料
tebiki
0
4k
「ふりかえりのふりかえり」をふりかえり、実のあるふりかえりにする
naitosatoshi
0
200
〜小さく始めて大きく育てる〜データ分析基盤の開発から活用まで
kniino
0
1.7k
AWS パートナー企業でテクニカルサポートに従事して2年経ったので思うところをまとめてみた
kazzpapa3
3
1.3k
デザインシステム基盤構築実践
leveragestech
0
770
なぜ NOT A HOTEL が Web3 に取り組むのか - NOT A HOTEL TECH TALK
ynunokawa
0
160
Featured
See All Featured
Faster Mobile Websites
deanohume
296
30k
Optimizing for Happiness
mojombo
369
69k
The Straight Up "How To Draw Better" Workshop
denniskardys
227
130k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
34
8.8k
Learning to Love Humans: Emotional Interface Design
aarron
266
39k
Building Your Own Lightsaber
phodgson
97
5.7k
What's new in Ruby 2.0
geeforr
336
31k
Thoughts on Productivity
jonyablonski
57
3.8k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
6
990
Git: the NoSQL Database
bkeepers
PRO
421
63k
Ruby is Unlike a Banana
tanoku
95
10k
The Illustrated Children's Guide to Kubernetes
chrisshort
28
46k
Transcript
7 2016/08/20 1
2 l RNN#' l RNN "
" l RNN & !( $%
3
4 We can get
an idea of the quality of the learned feature vectors by displaying them in a 2-D map.
5 $%"! '(Bag of Words ')N-gram
We can get an idea of the quality " #& or
6 l RNN#' l RNN "
" l RNN & !( $%
7 l RNN#' l RNN "
" l RNN & !( $%
RNN 8
RNN 9 x1 z0
RNN 10 z1 y1
RNN 11 x2 z1
RNN 12 z2 y2
13 l RNN#' l RNN "
" l RNN & !( $%
RNN 14 xt zt-1
y t →
RNN 15 xt zt-1 y
t →
16 l RNN#' l RNN "
" l RNN & !( $%
RNN 17 Back Propagation through time
BPTT 18 % x #!% d $ & y
, ... , y ' % δ ( δ ) * " t t 1 t k out, t j t
BPTT 19 δ k out, 1 δ k out, 2
δ k out, 3 δ k out, t
BPTT 20 t1 t δ
j t
BPTT 21
22 l RNN#' l RNN "
" l RNN & !( $%
23 l RNN#' l RNN "
" l RNN & !( $%
RNN 24 #@10+'<3= 0A; ← &91,?7 &9$)+/" ) 4
*58&90 or :( !.2- ← RNN%>264
LSTM 25 '% (Long Short-Term Memory, LSTM) RNN &# →
&# !$ (+) "*
LSTM 26
LSTM 27
LSTM 28
LSTM 29
30 l RNN#' l RNN "
" l RNN & !( $%
RNN 31 “w n” …… ^
(HMM) 32 %! $ "# $ "#
%!
33 $ .)-+ (Connectionist temporal classification, CTC) HMM#
! RNN &, %*"(, ' &,
CTC 34 X = x , ... ,
x l = l , … , l = p( l | X ) 1 t 1 |l|
CTC 35 l = ‘ab’ t = 6
a, b, , , , a, , , b, , , , , a, , b …
CTC 36 = p( l | X ) a, b,
, , , a, a, , b, , , , , a, , b … p( l1 | X ) = p( l2 | X ) = p( l3 | X ) = = p(a)*p(b)*p( )*p( ) *p( )*p( ) = p(a)*p(a)*p( )*p(b) *p( )*p( ) = p( )*p( )*p( )*p(a)*p( )*p(b)
37 • ;&B(2015):5:#3, .<2 • /%) in $"#3 E?!(2015):
http://www.slideshare.net/shotarosano5/chapter7-50542830, 2016A8*12@C • Recurrent Neural Networks(2014): http://www.slideshare.net/beam2d/pfi-seminar- 20141030rnn?qid=9e5894c7-f162-4da3-b082-a1e4963689e8&v=&b=&from_search=17, 2016A8*12@C • =86 (2013): 7+,4D19+,4D, 2 • LSTM 0(>-'(2016): http://qiita.com/t_Signull/items/21b82be280b46f467d1b, 2016A8*12@C • A. Graves(2008): Supervised sequence labelling with Recurrent Neural Networks, PhD thesis, Technische Universität München, https://www.cs.toronto.edu/~graves/preprint.pdf