Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Cha...
Search
Shotaro Ishihara
April 18, 2018
Technology
340
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
『深層学習』第7章「再帰型ニューラルネット」輪読会資料 / Deep Learning Chapter 7
http://bookclub.kodansha.co.jp/product?isbn=9784061529021
Shotaro Ishihara
April 18, 2018
Transcript
37 slides
7 2016/08/20 1
2 l RNN#' l RNN "
" l RNN & !( $%
3
4 We can get
an idea of the quality of the learned feature vectors by displaying them in a 2-D map.
5 $%"! '(Bag of Words ')N-gram
We can get an idea of the quality " #& or
6 l RNN#' l RNN "
" l RNN & !( $%
7 l RNN#' l RNN "
" l RNN & !( $%
RNN 8
RNN 9 x1 z0
RNN 10 z1 y1
RNN 11 x2 z1
RNN 12 z2 y2
13 l RNN#' l RNN "
" l RNN & !( $%
RNN 14 xt zt-1
y t →
RNN 15 xt zt-1 y
t →
16 l RNN#' l RNN "
" l RNN & !( $%
RNN 17 Back Propagation through time
BPTT 18 % x #!% d $ & y
, ... , y ' % δ ( δ ) * " t t 1 t k out, t j t
BPTT 19 δ k out, 1 δ k out, 2
δ k out, 3 δ k out, t
BPTT 20 t1 t δ
j t
BPTT 21
22 l RNN#' l RNN "
" l RNN & !( $%
23 l RNN#' l RNN "
" l RNN & !( $%
RNN 24 #@10+'<3= 0A; ← &91,?7 &9$)+/" ) 4
*58&90 or :( !.2- ← RNN%>264
LSTM 25 '% (Long Short-Term Memory, LSTM) RNN &# →
&# !$ (+) "*
LSTM 26
LSTM 27
LSTM 28
LSTM 29
30 l RNN#' l RNN "
" l RNN & !( $%
RNN 31 “w n” …… ^
(HMM) 32 %! $ "# $ "#
%!
33 $ .)-+ (Connectionist temporal classification, CTC) HMM#
! RNN &, %*"(, ' &,
CTC 34 X = x , ... ,
x l = l , … , l = p( l | X ) 1 t 1 |l|
CTC 35 l = ‘ab’ t = 6
a, b, , , , a, , , b, , , , , a, , b …
CTC 36 = p( l | X ) a, b,
, , , a, a, , b, , , , , a, , b … p( l1 | X ) = p( l2 | X ) = p( l3 | X ) = = p(a)*p(b)*p( )*p( ) *p( )*p( ) = p(a)*p(a)*p( )*p(b) *p( )*p( ) = p( )*p( )*p( )*p(a)*p( )*p(b)
37 • ;&B(2015):5:#3, .<2 • /%) in $"#3 E?!(2015):
http://www.slideshare.net/shotarosano5/chapter7-50542830, 2016A8*12@C • Recurrent Neural Networks(2014): http://www.slideshare.net/beam2d/pfi-seminar- 20141030rnn?qid=9e5894c7-f162-4da3-b082-a1e4963689e8&v=&b=&from_search=17, 2016A8*12@C • =86 (2013): 7+,4D19+,4D, 2 • LSTM 0(>-'(2016): http://qiita.com/t_Signull/items/21b82be280b46f467d1b, 2016A8*12@C • A. Graves(2008): Supervised sequence labelling with Recurrent Neural Networks, PhD thesis, Technische Universität München, https://www.cs.toronto.edu/~graves/preprint.pdf
More Decks by Shotaro Ishihara
See All by Shotaro Ishihara
Generative News Recommendation Reproduces Popularity: A Memorization-Based Analysis
upura
0
7
Agent 時代の Kaggle 展望 / kaggle-in-the-agentic-era
upura
1
1.5k
大規模言語モデルは誰を覚えているか / Who Do Large Language Models Memorize?
upura
0
240
[ACL 2026 Demo] Fast-MIA: Efficient and Scalable Membership Inference for LLMs
upura
0
160
Fast-MIA: Efficient and Scalable Membership Inference for LLMs
upura
0
120
JAPAN AI CUP Prediction Tutorial
upura
2
1.4k
情報技術の社会実装に向けた応用と課題:ニュースメディアの事例から / appmech-jsce 2025
upura
0
480
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
upura
0
850
Quantifying Memorization in Continual Pre-training with Japanese General or Industry-Specific Corpora
upura
1
170
Other Decks in Technology
See All in Technology
MCPでネットワーク障害調査を自動化する前に直すべき3つのこと
yama3133
1
650
市販IoT機器や自作IoT機器とHomeAssistantを駆使して家の電気代をできる限り見える化する方法の考案
harutiro
0
400
AWS Client VPNの設定でハマったポイントとその過程での気づき
amarelo_n24
1
420
コピーオンライト型ファイルシステムと ランダム書き込み
sat
PRO
2
130
IP重複は"ランディングVPC"で受け止める ― PrivateLink と Transit Gatewayによるマルチテナント接続基盤の設計
hatahata021
3
250
統制されたID・権限をどう利用するか : DPoP/FAPIによるAPIセキュリティ強化
oidfj
1
230
OpenTelemetryで変わった 新時代のCloudWatchメトリクス監視
tsukuboshi
0
470
AIネイティブ組織の作り方。1人のPOとAIから始める、自社での実験
ama_ch
2
1.2k
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
4
950
Datadog の権限統制 〜Org 統合に向けて調べた注意点 / JDDUG-Meetup_24
visional_engineering_and_design
1
200
AI×アジャイルで、石橋を高速で叩きながら渡る~電力会社がゼロから築いた会員サイト改善プロセス~ / 20261112 Ayami Kanazawa & Shigeru Hirai
shift_evolve
PRO
1
110
人の努力に頼らないセキュリティをエンジニアリングする在り方 / An approach to engineering security that does not rely on human effort
carta_engineering
0
230
Featured
See All Featured
A better future with KSS
kneath
240
18k
Thoughts on Productivity
jonyablonski
76
5.5k
Leading Effective Engineering Teams in the AI Era
addyosmani
9
2.9k
Learning to Love Humans: Emotional Interface Design
aarron
275
41k
The Illustrated Guide to Node.js - THAT Conference 2024
reverentgeek
1
610
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
inesmontani
PRO
0
810
Art, The Web, and Tiny UX
lynnandtonic
304
22k
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
soudai
PRO
68
60k
[SF Ruby Conf 2025] Rails X
palkan
3
1.5k
Paper Plane
katiecoart
PRO
6
55k
HDC tutorial
michielstock
2
1k
Beyond borders and beyond the search box: How to win the global "messy middle" with AI-driven SEO
davidcarrasco
3
310