Deep Learning Book 10その2 / deep learning book 10 vol2

Transcript

1. 
   &DIP
   4UBUF
   /FUXPSLT
   
   &YQMJDJU
   .FNPSZ IJNLU
   !
   य़೔ΤϦΞ
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
    DEEP LEARNING BOOK 4FRVFODF
   .PEFMJOH
   3FDVSSFOU
   BOE
   3FDVSTJWF
   /FUT
2. 
   &DIP
   4UBUF
   /FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ
   w ӅΕ૚
   
   ӅΕ૚
   SFDVSSFOU
   XFJHIUT
   w ೖྗ
   
   ӅΕ૚
   JOQVU
   XFJHIUT
   w &DIP
   4UBUF
   /FUXPSL
   w ӅΕ૚
   
   ӅΕ૚͸ॏΈΛݻఆ
   w ֶश͢Δͷ͸ʜ
   w ೖྗ
   
   ӅΕ૚
   JOQVU
   XFJHIUT
   w ӅΕ૚
   
   ग़ྗ
   PVUQVU
   XFJHIUT 
3. 
   &DIP
   4UBUF
   /FUXPSLT  IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL
4. ,FSOFM
   NBDIJOFͱͷྨࣅੑ w Χʔωϧ͕΍ͬͯΔ͜ͱͬͯʁ
   w ೚ҙͷ௕͞ͷܥྻΛݻఆ௕ͷϕΫτϧ΁ࣸ͢
   w ݻఆ௕ͷϕΫτϧΛ༻͍ͯ෼ྨث͕໰୊Λղ͘
   w ͜ͷܗͷ৔߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ
   w ग़ྗઢܗճؼͷ৔߹͸.4&ͰֶशͰ͖Δ
   w &4/T΋ೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ
   w தͷॏΈ͸ݻఆ͍ͯ͠Δ  ͍͔ʹաڈͷ৘ใΛ๛෋ʹؚΉදݱ͕ಘΒΕΔ
 ॏΈΛઃఆ͢Ε͹Α͍͔ʁ શવҙຯ͕Θ͔Βͣʜ
   
   3//ΛಈతγεςϜͱΈͳ͢
   
   γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ
5. 
   -FBLZ
   6OJUT
   BOE
   0UIFS
   4USBUFHJFT
   GPS
   .VMUJQMF
   5JNF
   4DBMF w աڈͷ৘ใΛ఻͑ΔͨΊͷ޻෉
    "EEJOH
   4LJQ
   $POOFDUJPOT
   UISPVHI
   5JNF
   w ޯ഑ͷফࣦͷ଎౓͕஗͘ͳΔ
   w രൃ͸ݩͷ3//ͱಉ͡଎౓Ͱൃੜ͢Δ
    -FBLZ
   6OJUT
   BOE
   B
   4QFDUSVN
   PG
   %J⒎FSFOU
   5JNF
   4DBMFT
   w աڈͷ৘ใΛͲͷఔ౓࢒͔͢Λ੍ޚ͢Δ
    3FNPWJOH
   $POOFDUJPOT
   w ୹͍࣌ࠁͰͷґଘΛ௕͍࣌ࠁͰͷґଘʹஔ͖׵͑Δ 
6. -FBLZ
   6OJUT w աڈͷ৘ใΛͲͷ͘Β͍࢒͔͢Λௐ੔͢Δ
   w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ
   w Ћ͕େ͖͍
   ʹ͍ۙ
   
   աڈͷ৘ใΛΑΓอଘ͢Δ
   w Ћ͕খ͍͞
   ʹ͍ۙ
   
   աڈͷ৘ใΛ͙͢ʹࣺͯΔ
   w Ћ͸ద౰ʹܾఆ͢ΔϋΠύʔύϥϝʔλ  µ(t) ↵µ(t 1) + (1 ↵)v(t)
7. 
   -POH4IPSU
   5FSN
   .FNPSZ w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ഑͕ফ͑ʹ͘͘͢Δ  IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T 3// -45.
8. 
   (BUFE
   3FDVSSFOU
   6OJUT w ٙ໰
   -45.͸ෳࡶ͗͢ΔͷͰ͸ͳ͍͔ʁ
   w (36
   -45.ΑΓ΋ߴ଎ɾ-45.ͱಉ౳ͷੑೳ
   w ͲͪΒ͕ྑ͍͔͸λεΫʹΑΔ  -45. (36
   IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ
9. ਺ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ  -45. (36 zt = (xtUz + ht 1Wz) rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱ͸ೖྗήʔτͱ๨٫ήʔτ͕౷߹͞Ε͍ͯΔ
10. 
    0QUJNJ[BUJPO
    GPS
    -POH5FSN
    %FQFOEFODJFT w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ෼͸
    w ඇৗʹେ͖ͳ஋ΛͱΔ
    PS
    w ඇৗʹখ͞ͳ஋ΛͱΔ
    w ಛʹɼޯ഑͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Ε͹ྑ͍͔ʁ
     ޯ഑ͷΫϦοϐϯά
     ޯ഑ͷਖ਼نԽ 
11. $MJQQJOH
    (SBEJFOU w ޯ഑͕ඇৗʹ
    େ͖͍cখ͍͞
    ͱ͸ʁ
    w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ  IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
12. $MJQQJOH
    (SBEJFOU w ޯ഑๏ϕʔεͷख๏ʹΑΔͱʜ
    w ֑ͷपΓͰ఺͕ਧ͖ඈΜͰ͠·͏
    ޯ഑രൃ
    w 
    ޯ഑͕େ͖͘ͳΓ͗ͨ͢Βޯ഑ͷϊϧϜͰׂΔ
    w ޯ഑ΛHͱͯ͠ʜ
    w W͸ϋΠύʔύϥϝʔλ
    ࣗવݴޠॲཧͩͱ͕ଟ͍  g ( gv ||g|| (||g|| > v) g (otherwise)
13. 3FHVMBSJ[JOH
    UP
    &ODPVSBHF
    *OGPSNBUJPO
    'MPX w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPO
    qPXʯΛଅਐ
    w ͜ͷ߲ͷ௚઀ܭࢉ͸೉͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ
    w $MJQQJOHͱ૊Έ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ཭͕৳ͼΔ  ⌦ = X t ⇣||(rh(t) L) @h(t) @h(t 1) || ||(rh(t) L)|| 1 ⌘2
14. 
    &YQMJDJU
    .FNPSZ w χϡʔϥϧωοτϫʔΫ͸ʜ
    w ҉໧తͳ৘ใͷอ࣋͸ಘҙ
    w ໌ࣔతͳ৘ใ
    ࣄ࣮
    ͷอ࣋͸ۤख
    w ໌ࣔతͳ৘ใΛอ࣋͠ɼਪ࿦ʹ׆༻͢Δߏ଄
 ʢϫʔΩϯάϝϞϦͷಋೖʣ
    w .FNPSZ
    /FUXPSLT
    w /FVSBM
    5VSJOH
    .BDIJOF 
15. "
    TDIFNBUJD
    PG
    B
    OFUXPSL
    XJUI
    BO
    FYQMJDJU
    NFNPSZ  IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM
16. "
    TDIFNBUJD
    PG
    B
    OFUXPSL
    XJUI
    BO
    FYQMJDJU
    NFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͸೉͍͠
    w 
    ଟ͘ͷϝϞϦηϧͷॏΈ෇͖ฏۉΛͱΔ
    w ॏΈ͸ιϑτϚοΫεͳͲͰ࡞Δ
 ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ
    w ϝϞϦηϧ͸εΧϥΑΓ΋ϕΫτϧͷํ͕ྑ͍
    w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ
    w ʮl8F
    BMM
    MJWF
    JO
    B
    ZFMMPX
    TVCNBSJOFz
    ΛؚΉՎࢺΛݟ͚ͭΔʯ
    w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱ͸ʁʣ
    w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ
    w ʢΞυϨογϯά͸Ξςϯγϣϯͱಉ͡ܗࣜʣ