Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Cha...
Search
Shotaro Ishihara
April 18, 2018
Technology
0
320
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Chapter 7
http://bookclub.kodansha.co.jp/product?isbn=9784061529021
Shotaro Ishihara
April 18, 2018
Tweet
Share
More Decks by Shotaro Ishihara
See All by Shotaro Ishihara
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
upura
0
180
Quantifying Memorization in Continual Pre-training with Japanese General or Industry-Specific Corpora
upura
1
36
JOAI2025講評 / joai2025-review
upura
0
420
AI エージェントを活用した研究再現性の自動定量評価 / scisci2025
upura
1
150
JSAI2025 企画セッション「人工知能とコンペティション」/ jsai2025-competition
upura
0
44
生成的推薦の人気バイアスの分析:暗記の観点から / JSAI2025
upura
0
260
Semantic Shift Stability: 学習コーパス内の単語の意味変化を用いた事前学習済みモデルの時系列性能劣化の監査
upura
0
70
日本語ニュース記事要約支援に向けたドメイン特化事前学習済みモデルの構築と活用 / t5-news-summarization
upura
0
76
Web からのデータ収集と探究事例の紹介 / no94_jsai_seminar
upura
0
350
Other Decks in Technology
See All in Technology
Autonomous Database - Dedicated 技術詳細 / adb-d_technical_detail_jp
oracle4engineer
PRO
4
10k
開発者を支える Internal Developer Portal のイマとコレカラ / To-day and To-morrow of Internal Developer Portals: Supporting Developers
aoto
PRO
1
440
Snowflakeの生成AI機能を活用したデータ分析アプリの作成 〜Cortex AnalystとCortex Searchの活用とStreamlitアプリでの利用〜
nayuts
1
470
Evolución del razonamiento matemático de GPT-4.1 a GPT-5 - Data Aventura Summit 2025 & VSCode DevDays
lauchacarro
0
160
フルカイテン株式会社 エンジニア向け採用資料
fullkaiten
0
8.7k
RSCの時代にReactとフレームワークの境界を探る
uhyo
10
3.4k
ChatGPTとPlantUML/Mermaidによるソフトウェア設計
gowhich501
1
130
企業の生成AIガバナンスにおけるエージェントとセキュリティ
lycorptech_jp
PRO
2
160
KotlinConf 2025_イベントレポート
sony
1
120
テストを軸にした生き残り術
kworkdev
PRO
0
190
AWSで推進するデータマネジメント
kawanago
1
1.3k
おやつは300円まで!の最適化を模索してみた
techtekt
PRO
0
290
Featured
See All Featured
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Embracing the Ebb and Flow
colly
87
4.8k
Done Done
chrislema
185
16k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.6k
Speed Design
sergeychernyshev
32
1.1k
Fireside Chat
paigeccino
39
3.6k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Reflections from 52 weeks, 52 projects
jeffersonlam
352
21k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
810
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3k
The World Runs on Bad Software
bkeepers
PRO
70
11k
Transcript
7 2016/08/20 1
2 l RNN#' l RNN "
" l RNN & !( $%
3
4 We can get
an idea of the quality of the learned feature vectors by displaying them in a 2-D map.
5 $%"! '(Bag of Words ')N-gram
We can get an idea of the quality " #& or
6 l RNN#' l RNN "
" l RNN & !( $%
7 l RNN#' l RNN "
" l RNN & !( $%
RNN 8
RNN 9 x1 z0
RNN 10 z1 y1
RNN 11 x2 z1
RNN 12 z2 y2
13 l RNN#' l RNN "
" l RNN & !( $%
RNN 14 xt zt-1
y t →
RNN 15 xt zt-1 y
t →
16 l RNN#' l RNN "
" l RNN & !( $%
RNN 17 Back Propagation through time
BPTT 18 % x #!% d $ & y
, ... , y ' % δ ( δ ) * " t t 1 t k out, t j t
BPTT 19 δ k out, 1 δ k out, 2
δ k out, 3 δ k out, t
BPTT 20 t1 t δ
j t
BPTT 21
22 l RNN#' l RNN "
" l RNN & !( $%
23 l RNN#' l RNN "
" l RNN & !( $%
RNN 24 #@10+'<3= 0A; ← &91,?7 &9$)+/" ) 4
*58&90 or :( !.2- ← RNN%>264
LSTM 25 '% (Long Short-Term Memory, LSTM) RNN &# →
&# !$ (+) "*
LSTM 26
LSTM 27
LSTM 28
LSTM 29
30 l RNN#' l RNN "
" l RNN & !( $%
RNN 31 “w n” …… ^
(HMM) 32 %! $ "# $ "#
%!
33 $ .)-+ (Connectionist temporal classification, CTC) HMM#
! RNN &, %*"(, ' &,
CTC 34 X = x , ... ,
x l = l , … , l = p( l | X ) 1 t 1 |l|
CTC 35 l = ‘ab’ t = 6
a, b, , , , a, , , b, , , , , a, , b …
CTC 36 = p( l | X ) a, b,
, , , a, a, , b, , , , , a, , b … p( l1 | X ) = p( l2 | X ) = p( l3 | X ) = = p(a)*p(b)*p( )*p( ) *p( )*p( ) = p(a)*p(a)*p( )*p(b) *p( )*p( ) = p( )*p( )*p( )*p(a)*p( )*p(b)
37 • ;&B(2015):5:#3, .<2 • /%) in $"#3 E?!(2015):
http://www.slideshare.net/shotarosano5/chapter7-50542830, 2016A8*12@C • Recurrent Neural Networks(2014): http://www.slideshare.net/beam2d/pfi-seminar- 20141030rnn?qid=9e5894c7-f162-4da3-b082-a1e4963689e8&v=&b=&from_search=17, 2016A8*12@C • =86 (2013): 7+,4D19+,4D, 2 • LSTM 0(>-'(2016): http://qiita.com/t_Signull/items/21b82be280b46f467d1b, 2016A8*12@C • A. Graves(2008): Supervised sequence labelling with Recurrent Neural Networks, PhD thesis, Technische Universität München, https://www.cs.toronto.edu/~graves/preprint.pdf