Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning Book 10その2 / deep learning book 10 vol2

himkt
January 29, 2018

Deep Learning Book 10その2 / deep learning book 10 vol2

himkt

January 29, 2018
Tweet

More Decks by himkt

Other Decks in Research

Transcript

  1. &DIP4UBUF/FUXPSLT&YQMJDJU.FNPSZ
    IJNLU!य़೔ΤϦΞ
    DEEP LEARNING BOOK
    4FRVFODF.PEFMJOH3FDVSSFOUBOE3FDVSTJWF/FUT

    View full-size slide

  2. &DIP4UBUF/FUXPSLT
    w 3//ʹֶ͓͍ͯश͕େมͳύϥϝʔλ
    w ӅΕ૚ӅΕ૚ SFDVSSFOUXFJHIUT

    w ೖྗӅΕ૚ JOQVUXFJHIUT

    w &DIP4UBUF/FUXPSL
    w ӅΕ૚ӅΕ૚͸ॏΈΛݻఆ
    w ֶश͢Δͷ͸ʜ
    w ೖྗӅΕ૚ JOQVUXFJHIUT

    w ӅΕ૚ग़ྗ PVUQVUXFJHIUT


    View full-size slide

  3. &DIP4UBUF/FUXPSLT

    IUUQXXXTDIPMBSQFEJBPSHBSUJDMF&DIP@TUBUF@OFUXPSL

    View full-size slide

  4. ,FSOFMNBDIJOFͱͷྨࣅੑ
    w Χʔωϧ͕΍ͬͯΔ͜ͱͬͯʁ
    w ೚ҙͷ௕͞ͷܥྻΛݻఆ௕ͷϕΫτϧ΁ࣸ͢
    w ݻఆ௕ͷϕΫτϧΛ༻͍ͯ෼ྨث͕໰୊Λղ͘
    w ͜ͷܗͷ৔߹ɼֶशͷج४ͷઃܭ͕༰қͰ͋Δ
    w ग़ྗઢܗճؼͷ৔߹͸.4&ͰֶशͰ͖Δ
    w &4/T΋ೖྗΛԿΒ͔ͷϕΫτϧʹࣸ͢ૢ࡞Λ͍ͯ͠Δ
    w தͷॏΈ͸ݻఆ͍ͯ͠Δ

    ͍͔ʹաڈͷ৘ใΛ๛෋ʹؚΉදݱ͕ಘΒΕΔ

    ॏΈΛઃఆ͢Ε͹Α͍͔ʁ શવҙຯ͕Θ͔Βͣʜ

    3//ΛಈతγεςϜͱΈͳ͢
    γεςϜ͕҆ఆ͢ΔΑ͏ͳॏΈΛઃఆ͢Δ

    View full-size slide

  5. -FBLZ6OJUTBOE0UIFS4USBUFHJFTGPS.VMUJQMF5JNF4DBMF
    w աڈͷ৘ใΛ఻͑ΔͨΊͷ޻෉
    "EEJOH4LJQ$POOFDUJPOTUISPVHI5JNF
    w ޯ഑ͷফࣦͷ଎౓͕஗͘ͳΔ
    w രൃ͸ݩͷ3//ͱಉ͡଎౓Ͱൃੜ͢Δ
    -FBLZ6OJUTBOEB4QFDUSVNPG%J⒎FSFOU5JNF4DBMFT
    w աڈͷ৘ใΛͲͷఔ౓࢒͔͢Λ੍ޚ͢Δ
    3FNPWJOH$POOFDUJPOT
    w ୹͍࣌ࠁͰͷґଘΛ௕͍࣌ࠁͰͷґଘʹஔ͖׵͑Δ

    View full-size slide

  6. -FBLZ6OJUT
    w աڈͷ৘ใΛͲͷ͘Β͍࢒͔͢Λௐ੔͢Δ
    w ҠಈฏۉͷΑ͏ͳ;Δ·͍Λ͢Δ
    w Ћ͕େ͖͍ ʹ͍ۙ
    աڈͷ৘ใΛΑΓอଘ͢Δ
    w Ћ͕খ͍͞ ʹ͍ۙ
    աڈͷ৘ใΛ͙͢ʹࣺͯΔ
    w Ћ͸ద౰ʹܾఆ͢ΔϋΠύʔύϥϝʔλ

    µ(t) ↵µ(t 1) + (1 ↵)v(t)

    View full-size slide

  7. -POH4IPSU5FSN.FNPSZ
    w ࣗݾϧʔϓΛಋೖ͢Δ͜ͱͰޯ഑͕ফ͑ʹ͘͘͢Δ

    IUUQDPMBIHJUIVCJPQPTUT6OEFSTUBOEJOH-45.T
    3//
    -45.

    View full-size slide

  8. (BUFE3FDVSSFOU6OJUT
    w ٙ໰-45.͸ෳࡶ͗͢ΔͷͰ͸ͳ͍͔ʁ
    w (36-45.ΑΓ΋ߴ଎ɾ-45.ͱಉ౳ͷੑೳ
    w ͲͪΒ͕ྑ͍͔͸λεΫʹΑΔ

    -45. (36
    IUUQTJTBBDDIBOHIBVHJUIVCJP-45.BOE(36'PSNVMB4VNNBSZ

    View full-size slide

  9. ਺ࣜతʹൺֱ͢ΔʢόΠΞεΛແࢹʣ

    -45.
    (36
    zt = (xtUz + ht 1Wz)
    rt = (xtUr + ht 1Wr)
    ˜
    ht = tanh

    xt + Uh + (rt ht 1)Wh

    ht = (1 zt) ht 1 + zt
    ˜
    ht
    it = (xtUi + ht 1Wi)
    ft = (xtUf + ht 1Wf )
    ot = (xtUo + ht 1Wg)
    ˜
    Ct = tanh (xtUg + ht 1Wg)
    Ct = (ft Ct 1 + it
    ˜
    Ct)
    ht = tanh (Ct) ot
    (36Ͱ͸ೖྗήʔτͱ๨٫ήʔτ͕౷߹͞Ε͍ͯΔ

    View full-size slide

  10. 0QUJNJ[BUJPOGPS-POH5FSN%FQFOEFODJFT
    w 3//Λϕʔεͱͨ͠χϡʔϥϧωοτϫʔΫͷඍ෼͸
    w ඇৗʹେ͖ͳ஋ΛͱΔPS
    w ඇৗʹখ͞ͳ஋ΛͱΔ
    w ಛʹɼޯ഑͕ඇৗʹେ͖ͳͱ͖ʹͲ͏͢Ε͹ྑ͍͔ʁ
    ޯ഑ͷΫϦοϐϯά
    ޯ഑ͷਖ਼نԽ

    View full-size slide

  11. $MJQQJOH(SBEJFOU
    w ޯ഑͕ඇৗʹ େ͖͍cখ͍͞
    ͱ͸ʁ
    w ͍͍ͩͨฏΒ͚ͩͲͱ͖Ͳ͖֑͕͋Δ

    IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM

    View full-size slide

  12. $MJQQJOH(SBEJFOU
    w ޯ഑๏ϕʔεͷख๏ʹΑΔͱʜ
    w ֑ͷपΓͰ఺͕ਧ͖ඈΜͰ͠·͏ ޯ഑രൃ

    w ޯ഑͕େ͖͘ͳΓ͗ͨ͢Βޯ഑ͷϊϧϜͰׂΔ
    w ޯ഑ΛHͱͯ͠ʜ
    w W͸ϋΠύʔύϥϝʔλ ࣗવݴޠॲཧͩͱ͕ଟ͍


    g
    (
    gv
    ||g||
    (||g|| > v)
    g (otherwise)

    View full-size slide

  13. 3FHVMBSJ[JOHUP&ODPVSBHF*OGPSNBUJPO'MPX
    w ਖ਼ଇԽ߲Λಋೖ͢Δ͜ͱͰʮJOGPSNBUJPOqPXʯΛଅਐ
    w ͜ͷ߲ͷ௚઀ܭࢉ͸೉͍͕͠ɼۙࣅ͕ఏҊ͞Ε͍ͯΔ
    w $MJQQJOHͱ૊Έ߹ΘͤΔ͜ͱͰهԱͰ͖Δڑ཭͕৳ͼΔ

    ⌦ =
    X
    t
    ⇣||(rh(t)
    L) @h(t)
    @h(t 1)
    ||
    ||(rh(t)
    L)||
    1
    ⌘2

    View full-size slide

  14. &YQMJDJU.FNPSZ
    w χϡʔϥϧωοτϫʔΫ͸ʜ
    w ҉໧తͳ৘ใͷอ࣋͸ಘҙ
    w ໌ࣔతͳ৘ใ ࣄ࣮
    ͷอ࣋͸ۤख
    w ໌ࣔతͳ৘ใΛอ࣋͠ɼਪ࿦ʹ׆༻͢Δߏ଄

    ʢϫʔΩϯάϝϞϦͷಋೖʣ
    w .FNPSZ/FUXPSLT
    w /FVSBM5VSJOH.BDIJOF

    View full-size slide

  15. "TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ

    IUUQXXXEFFQMFBSOJOHCPPLPSHMFDUVSF@TMJEFTIUNM

    View full-size slide

  16. "TDIFNBUJDPGBOFUXPSLXJUIBOFYQMJDJUNFNPSZ
    w ਖ਼֬ͳϝϞϦͷΞυϨεΛग़ྗ͢Δͷ͸೉͍͠
    w ଟ͘ͷϝϞϦηϧͷॏΈ෇͖ฏۉΛͱΔ
    w ॏΈ͸ιϑτϚοΫεͳͲͰ࡞Δ

    ʢͰ͖Δ͚ͩҰՕॴͷϝϞϦΛࢀর͢ΔΑ͏ʹʣ
    w ϝϞϦηϧ͸εΧϥΑΓ΋ϕΫτϧͷํ͕ྑ͍
    w ίϯςϯπϕʔεΞυϨογϯά͕ՄೳʹͳΔ
    w ʮl8FBMMMJWFJOBZFMMPXTVCNBSJOFzΛؚΉՎࢺΛݟ͚ͭΔʯ
    w ʢϩέʔγϣϯϕʔεΞυϨογϯάͱ͸ʁʣ
    w ʮεϩοτ347ʹ֨ೲ͞Ε͍ͯΔՎࢺΛऔಘ͢Δʯ
    w ʢΞυϨογϯά͸Ξςϯγϣϯͱಉ͡ܗࣜʣ

    View full-size slide