$30 off During Our Annual Pro Sale. View Details »

Deep Learning Book 10その2 / deep learning book 10 vol2

himkt
January 29, 2018

Deep Learning Book 10その2 / deep learning book 10 vol2

himkt

January 29, 2018
Tweet

More Decks by himkt

Other Decks in Research

Transcript

  1. &DIP4UBUF/FUXPSLT&YQMJDJU.FNPSZ IJNLU!य़೔ΤϦΞ DEEP LEARNING BOOK 4FRVFODF.PEFMJOH3FDVSSFOUBOE3FDVSTJWF/FUT

  2. &DIP4UBUF/FUXPSLT w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ w ӅΕ૚ӅΕ૚ SFDVSSFOUXFJHIUT  w ೖྗӅΕ૚ JOQVUXFJHIUT

     w &DIP4UBUF/FUXPSL w ӅΕ૚ӅΕ૚͸ॏΈΛݻఆ w ֶश͢Δͷ͸ʜ w ೖྗӅΕ૚ JOQVUXFJHIUT  w ӅΕ૚ग़ྗ PVUQVUXFJHIUT 
  3. &DIP4UBUF/FUXPSLT  IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL

  4. ,FSOFMNBDIJOFͱͷྨࣅੑ w Χʔωϧ͕΍ͬͯΔ͜ͱͬͯʁ w ೚ҙͷ௕͞ͷܥྻΛݻఆ௕ͷϕΫτϧ΁ࣸ͢ w ݻఆ௕ͷϕΫτϧΛ༻͍ͯ෼ྨث͕໰୊Λղ͘ w ͜ͷܗͷ৔߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ w

    ग़ྗઢܗճؼͷ৔߹͸.4&ͰֶशͰ͖Δ w &4/T΋ೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ w தͷॏΈ͸ݻఆ͍ͯ͠Δ  ͍͔ʹաڈͷ৘ใΛ๛෋ʹؚΉදݱ͕ಘΒΕΔ
 ॏΈΛઃఆ͢Ε͹Α͍͔ʁ શવҙຯ͕Θ͔Βͣʜ  3//ΛಈతγεςϜͱΈͳ͢ γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ
  5. -FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF w աڈͷ৘ใΛ఻͑ΔͨΊͷ޻෉  "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF w ޯ഑ͷফࣦͷ଎౓͕஗͘ͳΔ w രൃ͸ݩͷ3//ͱಉ͡଎౓Ͱൃੜ͢Δ 

    -FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT w աڈͷ৘ใΛͲͷఔ౓࢒͔͢Λ੍ޚ͢Δ  3FNPWJOH$POOFDUJPOT w ୹͍࣌ࠁͰͷґଘΛ௕͍࣌ࠁͰͷґଘʹஔ͖׵͑Δ 
  6. -FBLZ6OJUT w աڈͷ৘ใΛͲͷ͘Β͍࢒͔͢Λௐ੔͢Δ w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ w Ћ͕େ͖͍ ʹ͍ۙ աڈͷ৘ใΛΑΓอଘ͢Δ w

    Ћ͕খ͍͞ ʹ͍ۙ աڈͷ৘ใΛ͙͢ʹࣺͯΔ w Ћ͸ద౰ʹܾఆ͢ΔϋΠύʔύϥϝʔλ  µ(t) ↵µ(t 1) + (1 ↵)v(t)
  7. -POH4IPSU5FSN.FNPSZ w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ഑͕ফ͑ʹ͘͘͢Δ  IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T 3// -45.

  8. (BUFE3FDVSSFOU6OJUT w ٙ໰-45.͸ෳࡶ͗͢ΔͷͰ͸ͳ͍͔ʁ w (36-45.ΑΓ΋ߴ଎ɾ-45.ͱಉ౳ͷੑೳ w ͲͪΒ͕ྑ͍͔͸λεΫʹΑΔ  -45. (36

    IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ
  9. ਺ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ  -45. (36 zt = (xtUz + ht 1Wz)

    rt = (xtUr + ht 1Wr) ˜ ht = tanh ⇣ xt + Uh + (rt ht 1)Wh ⌘ ht = (1 zt) ht 1 + zt ˜ ht it = (xtUi + ht 1Wi) ft = (xtUf + ht 1Wf ) ot = (xtUo + ht 1Wg) ˜ Ct = tanh (xtUg + ht 1Wg) Ct = (ft Ct 1 + it ˜ Ct) ht = tanh (Ct) ot (36Ͱ͸ೖྗήʔτͱ๨٫ήʔτ͕౷߹͞Ε͍ͯΔ
  10. 0QUJNJ[BUJPOGPS-POH5FSN%FQFOEFODJFT w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ෼͸ w ඇৗʹେ͖ͳ஋ΛͱΔPS w ඇৗʹখ͞ͳ஋ΛͱΔ w ಛʹɼޯ഑͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Ε͹ྑ͍͔ʁ 

    ޯ഑ͷΫϦοϐϯά  ޯ഑ͷਖ਼نԽ 
  11. $MJQQJOH(SBEJFOU w ޯ഑͕ඇৗʹ େ͖͍cখ͍͞ ͱ͸ʁ w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ  IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM

  12. $MJQQJOH(SBEJFOU w ޯ഑๏ϕʔεͷख๏ʹΑΔͱʜ w ֑ͷपΓͰ఺͕ਧ͖ඈΜͰ͠·͏ ޯ഑രൃ  w ޯ഑͕େ͖͘ͳΓ͗ͨ͢Βޯ഑ͷϊϧϜͰׂΔ w

    ޯ഑ΛHͱͯ͠ʜ w W͸ϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍  g ( gv ||g|| (||g|| > v) g (otherwise)
  13. 3FHVMBSJ[JOHUP&ODPVSBHF*OGPSNBUJPO'MPX w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPOqPXʯΛଅਐ w ͜ͷ߲ͷ௚઀ܭࢉ͸೉͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ w $MJQQJOHͱ૊Έ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ཭͕৳ͼΔ  ⌦ =

    X t ⇣||(rh(t) L) @h(t) @h(t 1) || ||(rh(t) L)|| 1 ⌘2
  14. &YQMJDJU.FNPSZ w χϡʔϥϧωοτϫʔΫ͸ʜ w ҉໧తͳ৘ใͷอ࣋͸ಘҙ w ໌ࣔతͳ৘ใ ࣄ࣮ ͷอ࣋͸ۤख w

    ໌ࣔతͳ৘ใΛอ࣋͠ɼਪ࿦ʹ׆༻͢Δߏ଄
 ʢϫʔΩϯάϝϞϦͷಋೖʣ w .FNPSZ/FUXPSLT w /FVSBM5VSJOH.BDIJOF 
  15. "TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ  IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM

  16. "TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͸೉͍͠ w ଟ͘ͷϝϞϦηϧͷॏΈ෇͖ฏۉΛͱΔ w ॏΈ͸ιϑτϚοΫεͳͲͰ࡞Δ
 ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ w ϝϞϦηϧ͸εΧϥΑΓ΋ϕΫτϧͷํ͕ྑ͍

    w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱ͸ʁʣ w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ w ʢΞυϨογϯά͸Ξςϯγϣϯͱಉ͡ܗࣜʣ