Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Deep Learning Book 10その2 / deep learning book 1...
Search
himkt
January 29, 2018
Research
2
190
Deep Learning Book 10その2 / deep learning book 10 vol2
himkt
January 29, 2018
Tweet
Share
More Decks by himkt
See All by himkt
Linformer: paper reading
himkt
0
520
RoBERTa: paper reading
himkt
1
350
NLP SoTA 勉強会 / ner_2019
himkt
2
1.4k
自然言語処理 @ クックパッド / nlp at cookpad
himkt
1
520
Interpretable Machine Learning 6.3 - Prototypes and Criticisms
himkt
2
160
ニューラル固有表現抽出 / Neural Named Entity Recognition
himkt
3
740
ニューラル固有表現抽出器を実装してみる / PyNER
himkt
6
2.1k
Spacyでお手軽NLP / NLP with spacy
himkt
0
1k
ふわふわ系列ラベリング / ner 2018
himkt
5
850
Other Decks in Research
See All in Research
2025/7/5 応用音響研究会招待講演@北海道大学
takuma_okamoto
1
160
Google Agent Development Kit (ADK) 入門 🚀
mickey_kubo
2
1.5k
一人称視点映像解析の最先端(MIRU2025 チュートリアル)
takumayagi
6
3.1k
Cross-Media Information Spaces and Architectures
signer
PRO
0
230
20250502_ABEJA_論文読み会_スライド
flatton
0
190
Vision and LanguageからのEmbodied AIとAI for Science
yushiku
PRO
1
460
カスタマーサクセスの視点からAWS Summitの展示を考える~製品開発で活用できる勘所~
masakiokuda
2
170
2021年度-基盤研究B-研究計画調書
trycycle
PRO
0
200
公立高校入試等に対する受入保留アルゴリズム(DA)導入の提言
shunyanoda
0
6.6k
引力・斥力を制御可能なランダム部分集合の確率分布
wasyro
0
220
IMC の細かすぎる話 2025
smly
2
540
Mechanistic Interpretability:解釈可能性研究の新たな潮流
koshiro_aoki
1
390
Featured
See All Featured
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
Become a Pro
speakerdeck
PRO
29
5.5k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.4k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
Music & Morning Musume
bryan
46
6.7k
Optimizing for Happiness
mojombo
379
70k
4 Signs Your Business is Dying
shpigford
184
22k
Docker and Python
trallard
45
3.5k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
A designer walks into a library…
pauljervisheath
207
24k
Transcript
&DIP4UBUF/FUXPSLT&YQMJDJU.FNPSZ IJNLU!य़ΤϦΞ DEEP LEARNING BOOK 4FRVFODF.PEFMJOH3FDVSSFOUBOE3FDVSTJWF/FUT
&DIP4UBUF/FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ w ӅΕӅΕ SFDVSSFOUXFJHIUT w ೖྗӅΕ JOQVUXFJHIUT
w &DIP4UBUF/FUXPSL w ӅΕӅΕॏΈΛݻఆ w ֶश͢Δͷʜ w ೖྗӅΕ JOQVUXFJHIUT w ӅΕग़ྗ PVUQVUXFJHIUT
&DIP4UBUF/FUXPSLT IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL
,FSOFMNBDIJOFͱͷྨࣅੑ w Χʔωϧ͕ͬͯΔ͜ͱͬͯʁ w ҙͷ͞ͷܥྻΛݻఆͷϕΫτϧࣸ͢ w ݻఆͷϕΫτϧΛ༻͍ͯྨث͕Λղ͘ w ͜ͷܗͷ߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ w
ग़ྗઢܗճؼͷ߹.4&ͰֶशͰ͖Δ w &4/TೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ w தͷॏΈݻఆ͍ͯ͠Δ ͍͔ʹաڈͷใΛ๛ʹؚΉදݱ͕ಘΒΕΔ ॏΈΛઃఆ͢ΕΑ͍͔ʁ શવҙຯ͕Θ͔Βͣʜ 3//ΛಈతγεςϜͱΈͳ͢ γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ
-FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF w աڈͷใΛ͑ΔͨΊͷ "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF w ޯͷফࣦͷ͕͘ͳΔ w രൃݩͷ3//ͱಉ͡Ͱൃੜ͢Δ
-FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT w աڈͷใΛͲͷఔ͔͢Λ੍ޚ͢Δ 3FNPWJOH$POOFDUJPOT w ͍࣌ࠁͰͷґଘΛ͍࣌ࠁͰͷґଘʹஔ͖͑Δ
-FBLZ6OJUT w աڈͷใΛͲͷ͘Β͍͔͢Λௐ͢Δ w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ w Ћ͕େ͖͍ ʹ͍ۙ աڈͷใΛΑΓอଘ͢Δ w
Ћ͕খ͍͞ ʹ͍ۙ աڈͷใΛ͙͢ʹࣺͯΔ w Ћదʹܾఆ͢ΔϋΠύʔύϥϝʔλ µ(t) ↵µ(t 1) + (1 ↵)v(t)
-POH4IPSU5FSN.FNPSZ w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ͕ফ͑ʹ͘͘͢Δ IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T 3// -45.
(BUFE3FDVSSFOU6OJUT w ٙ-45.ෳࡶ͗͢ΔͷͰͳ͍͔ʁ w (36-45.ΑΓߴɾ-45.ͱಉͷੑೳ w ͲͪΒ͕ྑ͍͔λεΫʹΑΔ -45. (36
IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ
ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ -45. (36 zt = (xtUz + ht 1Wz)
rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱೖྗήʔτͱ٫ήʔτ͕౷߹͞Ε͍ͯΔ
0QUJNJ[BUJPOGPS-POH5FSN%FQFOEFODJFT w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ w ඇৗʹେ͖ͳΛͱΔPS w ඇৗʹখ͞ͳΛͱΔ w ಛʹɼޯ͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Εྑ͍͔ʁ
ޯͷΫϦοϐϯά ޯͷਖ਼نԽ
$MJQQJOH(SBEJFOU w ޯ͕ඇৗʹ େ͖͍cখ͍͞ ͱʁ w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
$MJQQJOH(SBEJFOU w ޯ๏ϕʔεͷख๏ʹΑΔͱʜ w ֑ͷपΓͰ͕ਧ͖ඈΜͰ͠·͏ ޯരൃ w ޯ͕େ͖͘ͳΓ͗ͨ͢ΒޯͷϊϧϜͰׂΔ w
ޯΛHͱͯ͠ʜ w WϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍ g ( gv ||g|| (||g|| > v) g (otherwise)
3FHVMBSJ[JOHUP&ODPVSBHF*OGPSNBUJPO'MPX w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPOqPXʯΛଅਐ w ͜ͷ߲ͷܭࢉ͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ w $MJQQJOHͱΈ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ͕৳ͼΔ ⌦ =
X t ⇣||(rh(t) L) @h(t) @h(t 1) || ||(rh(t) L)|| 1 ⌘2
&YQMJDJU.FNPSZ w χϡʔϥϧωοτϫʔΫʜ w ҉తͳใͷอ࣋ಘҙ w ໌ࣔతͳใ ࣄ࣮ ͷอ࣋ۤख w
໌ࣔతͳใΛอ࣋͠ɼਪʹ׆༻͢Δߏ ʢϫʔΩϯάϝϞϦͷಋೖʣ w .FNPSZ/FUXPSLT w /FVSBM5VSJOH.BDIJOF
"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͍͠ w ଟ͘ͷϝϞϦηϧͷॏΈ͖ฏۉΛͱΔ w ॏΈιϑτϚοΫεͳͲͰ࡞Δ ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ w ϝϞϦηϧεΧϥΑΓϕΫτϧͷํ͕ྑ͍
w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱʁʣ w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ w ʢΞυϨογϯάΞςϯγϣϯͱಉ͡ܗࣜʣ