Tweet
Tweet

Slide 1

Slide 1 text

&DIP4UBUF/FUXPSLT&YQMJDJU.FNPSZ IJNLU!य़೔ΤϦΞ DEEP LEARNING BOOK 4FRVFODF.PEFMJOH3FDVSSFOUBOE3FDVSTJWF/FUT

Slide 2

Slide 2 text

&DIP4UBUF/FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ w ӅΕ૚ӅΕ૚ SFDVSSFOUXFJHIUT w ೖྗӅΕ૚ JOQVUXFJHIUT w &DIP4UBUF/FUXPSL w ӅΕ૚ӅΕ૚͸ॏΈΛݻఆ w ֶश͢Δͷ͸ʜ w ೖྗӅΕ૚ JOQVUXFJHIUT w ӅΕ૚ग़ྗ PVUQVUXFJHIUT

Slide 3

Slide 3 text

&DIP4UBUF/FUXPSLT IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL

Slide 4

Slide 4 text

,FSOFMNBDIJOFͱͷྨࣅੑ w Χʔωϧ͕΍ͬͯΔ͜ͱͬͯʁ w ೚ҙͷ௕͞ͷܥྻΛݻఆ௕ͷϕΫτϧ΁ࣸ͢ w ݻఆ௕ͷϕΫτϧΛ༻͍ͯ෼ྨث͕໰୊Λղ͘ w ͜ͷܗͷ৔߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ w ग़ྗઢܗճؼͷ৔߹͸.4&ͰֶशͰ͖Δ w &4/T΋ೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ w தͷॏΈ͸ݻఆ͍ͯ͠Δ ͍͔ʹաڈͷ৘ใΛ๛෋ʹؚΉදݱ͕ಘΒΕΔ
 ॏΈΛઃఆ͢Ε͹Α͍͔ʁ શવҙຯ͕Θ͔Βͣʜ 3//ΛಈతγεςϜͱΈͳ͢ γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ

Slide 5

Slide 5 text

-FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF w աڈͷ৘ใΛ఻͑ΔͨΊͷ޻෉ "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF w ޯ഑ͷফࣦͷ଎౓͕஗͘ͳΔ w രൃ͸ݩͷ3//ͱಉ͡଎౓Ͱൃੜ͢Δ -FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT w աڈͷ৘ใΛͲͷఔ౓࢒͔͢Λ੍ޚ͢Δ 3FNPWJOH$POOFDUJPOT w ୹͍࣌ࠁͰͷґଘΛ௕͍࣌ࠁͰͷґଘʹஔ͖׵͑Δ

Slide 6

Slide 6 text

-FBLZ6OJUT w աڈͷ৘ใΛͲͷ͘Β͍࢒͔͢Λௐ੔͢Δ w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ w Ћ͕େ͖͍ ʹ͍ۙ աڈͷ৘ใΛΑΓอଘ͢Δ w Ћ͕খ͍͞ ʹ͍ۙ աڈͷ৘ใΛ͙͢ʹࣺͯΔ w Ћ͸ద౰ʹܾఆ͢ΔϋΠύʔύϥϝʔλ µ(t) ↵µ(t 1) + (1 ↵)v(t)

Slide 7

Slide 7 text

-POH4IPSU5FSN.FNPSZ w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ഑͕ফ͑ʹ͘͘͢Δ IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T 3// -45.

Slide 8

Slide 8 text

(BUFE3FDVSSFOU6OJUT w ٙ໰-45.͸ෳࡶ͗͢ΔͷͰ͸ͳ͍͔ʁ w (36-45.ΑΓ΋ߴ଎ɾ-45.ͱಉ౳ͷੑೳ w ͲͪΒ͕ྑ͍͔͸λεΫʹΑΔ -45. (36 IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ

Slide 9

Slide 9 text

਺ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ -45. (36 zt = (xtUz + ht 1Wz) rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱ͸ೖྗήʔτͱ๨٫ήʔτ͕౷߹͞Ε͍ͯΔ

Slide 10

Slide 10 text

0QUJNJ[BUJPOGPS-POH5FSN%FQFOEFODJFT w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ෼͸ w ඇৗʹେ͖ͳ஋ΛͱΔPS w ඇৗʹখ͞ͳ஋ΛͱΔ w ಛʹɼޯ഑͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Ε͹ྑ͍͔ʁ ޯ഑ͷΫϦοϐϯά ޯ഑ͷਖ਼نԽ

Slide 11

Slide 11 text

$MJQQJOH(SBEJFOU w ޯ഑͕ඇৗʹ େ͖͍cখ͍͞ ͱ͸ʁ w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM

Slide 12

Slide 12 text

$MJQQJOH(SBEJFOU w ޯ഑๏ϕʔεͷख๏ʹΑΔͱʜ w ֑ͷपΓͰ఺͕ਧ͖ඈΜͰ͠·͏ ޯ഑രൃ w ޯ഑͕େ͖͘ͳΓ͗ͨ͢Βޯ഑ͷϊϧϜͰׂΔ w ޯ഑ΛHͱͯ͠ʜ w W͸ϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍ g ( gv ||g|| (||g|| > v) g (otherwise)

Slide 13

Slide 13 text

3FHVMBSJ[JOHUP&ODPVSBHF*OGPSNBUJPO'MPX w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPOqPXʯΛଅਐ w ͜ͷ߲ͷ௚઀ܭࢉ͸೉͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ w $MJQQJOHͱ૊Έ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ཭͕৳ͼΔ ⌦ = X t ⇣||(rh(t) L) @h(t) @h(t 1) || ||(rh(t) L)|| 1 ⌘2

Slide 14

Slide 14 text

&YQMJDJU.FNPSZ w χϡʔϥϧωοτϫʔΫ͸ʜ w ҉໧తͳ৘ใͷอ࣋͸ಘҙ w ໌ࣔతͳ৘ใ ࣄ࣮ ͷอ࣋͸ۤख w ໌ࣔతͳ৘ใΛอ࣋͠ɼਪ࿦ʹ׆༻͢Δߏ଄
 ʢϫʔΩϯάϝϞϦͷಋೖʣ w .FNPSZ/FUXPSLT w /FVSBM5VSJOH.BDIJOF

Slide 15

Slide 15 text

"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM

Slide 16

Slide 16 text

"TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͸೉͍͠ w ଟ͘ͷϝϞϦηϧͷॏΈ෇͖ฏۉΛͱΔ w ॏΈ͸ιϑτϚοΫεͳͲͰ࡞Δ
 ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ w ϝϞϦηϧ͸εΧϥΑΓ΋ϕΫτϧͷํ͕ྑ͍ w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱ͸ʁʣ w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ w ʢΞυϨογϯά͸Ξςϯγϣϯͱಉ͡ܗࣜʣ