Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ベイズ深層学習(4.1)

catla
February 07, 2020

 ベイズ深層学習(4.1)

catla

February 07, 2020
Tweet

More Decks by catla

Other Decks in Science

Transcript

  1. ϕΠζਂ૚ֶश

    αϯϓϦϯάʹجͮ͘ਪ࿦ख๏
    ܡɹঘً

    View Slide

  2. αϯϓϦϯάʹجͮ͘ਪ࿦ख๏
    ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষ΋ࢀߟ

    View Slide

  3. αϯϓϦϯάʹجͮ͘ਪ࿦ख๏
    ɹ؍ଌσʔλΛ ɼඇ؍ଌͷม਺ͷू߹ʢFHύϥϝʔλɼજࡏม਺
    Λ ͱͨ͠ͱ
    ͖ɼϕΠζਪ࿦ʹΑΔ౷ܭղੳͰ͸ɼ֬཰Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ
    ɹ࣮ࡍʹɼ֬཰ϞσϧΛ༻ֶ͍ͯश΍༧ଌ͸ɼࣄޙ෼෍ Λܭࢉͯ͠ߦΘΕΔɽ
    X Z
    p(X, Z)
    p(Z|X)
    ໰୊఺
    ղܾࡦ
    ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣ͸ɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕
    ଟ͍ɽ
    p(Z|X)
    ɹ ΛղੳతʹٻΊΔ୅ΘΓʹɼ͜ͷ෼෍͔Βෳ਺ͷαϯϓϧΛಘΔ͜ͱͰɼ෼෍
    ͷಛੑΛௐ΂Δɽ
    ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճ͸ษڧ͢ΔΑʂ
    p(Z|X)

    View Slide

  4. ຊ೔ͷ಺༰
    ୯७ϞϯςΧϧϩ๏
    غ٫αϯϓϦϯά
    ࣗݾਖ਼نԽॏ఺αϯϓϦϯά
    ʢϚϧίϑ࿈࠯ϞϯςΧϧϩ๏ʣ
    ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏
    ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏
    ΪϒεαϯϓϦϯά

    View Slide

  5. ຊ೔ͷ಺༰
    ୯७ϞϯςΧϧϩ๏
    غ٫αϯϓϦϯά
    ࣗݾਖ਼نԽॏ৺αϯϓϦϯά
    ʢϚϧίϑ࿈࠯ϞϯςΧϧϩ๏ʣ
    ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏
    ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏
    ΪϒεαϯϓϦϯά
    .$.$

    View Slide

  6. ୯७ϞϯςΧϧϩ๏
    ɹ෼෍ ʹର͢Δؔ਺ ͷظ଴஋ɹ ΛٻΊ͍ͨɽɹ
    p(z) f(z) p(z)
    [ f(z)] =

    f(z)p(z)dz
    ໨త
    ঢ়گ
    ɹظ଴஋ ͷղੳతͳੵ෼ܭࢉ͕ࠔ೉ɽ
    ɹ෼෍ ͔ΒͷαϯϓϦϯά͸༰қɽ

    f(z)p(z)dz
    p(z)
    ख๏
    ɹ Λे෼େ͖ͳ஋ͱͨ͠ͱ͖ɼ


    T
    z(1), z(2), …, z(T) ∼ p(z)

    f(z)p(z)dz ≈
    1
    T
    T

    t=1
    f(z(t))
    ͔Β ݸαϯϓϦϯά
    ⟵ p(z) T

    View Slide

  7. ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล໬౓ Λܭࢉ͢Δࡍʹ࢖༻͢
    Δ৔߹ɼ

    θ p(X, θ) = p(X|θ)p(θ) p(X)
    p(X) =

    p(X|θ)p(θ)dθ =

    N

    n=1
    p(xn
    |θ)p(θ)dθ
    = p(θ)
    [p(X|θ)]

    1
    T
    T

    t=1
    N

    n=1
    p(xn
    |θ(t)), (θ(1), …, θ(T) ∼ p(θ))
    ɹظ଴஋ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ͸෯޿͘ͱΔඞཁ͕͋ΓɼҰํ
    Ͱɼ ͸ڱ͍ ͷൣғͰ͔͠େ͖ͳ஋ΛऔΒͳ͍έʔε͕ଟ͍ɽ
    ɹ ൚༻త͚ͩͲɼܭࢉޮ཰͕ѱ͍ɽ
    p(z)
    [ f(z)] p(z) z
    f(z) z

    ୯७ϞϯςΧϧϩ๏
    ໰୊఺

    View Slide

  8. غ٫αϯϓϦϯά
    ɹີ౓ܭࢉ͕ࠔ೉ͳ֬཰෼෍ ͔ΒαϯϓϧΛಘΔɽɹ
    p(z)
    z(1), z(2), … ∼ p(z)
    ໨త
    ঢ়گ
    ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ܭࢉՄೳɽͭ·Γɼ ɽ
    ˜
    p(z)( = Zp
    ⋅ p(z))

    ˜
    p(z)dz ≠ 1
    ख๏
    ɹఏҊ෼෍ Λઃఆ͢Δɽ೚ҙͷ ʹରͯ͠ɼ
    ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ
    ͱͳΔΑ͏ʹɼਖ਼ͷఆ਺ ΛఆΊΔɽ
    q(z) z
    kq(z) > ˜
    p(z)
    k
    ఏҊ෼෍
    ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ
    Ծͷ෼෍ɽ

    View Slide

  9. غ٫αϯϓϦϯά
    ख๏ ͖ͭͮʣ
    ɹఏҊ෼෍ ͔ΒαϯϓϧΛಘΔɽ

    ɹҰ༷෼෍ ͔ΒͷαϯϓϧΛಘΔɽ

    ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબ୒ɽ
    ɹɹɹ
    q(z)
    z(t) ∼ q(z)
    Uni(0,kq(z))
    ˜
    u ∼ Uni (0,kq(z(t)))
    z(t)
    if ˜
    u > ˜
    p(z(t)) then SFKFDU else BDDFQU
    ड༰཰

    q(z)
    ˜
    p(z)
    kq(z)
    dz =
    1
    k ∫
    ˜
    p(z)dz
    ໰୊఺
    ߴ࣍ݩͷม਺ͷαϯϓϦϯά͕ඞཁͳ৔߹ɼड༰཰͕ඇৗʹ௿͘ͳΔɽ

    View Slide

  10. غ٫αϯϓϦϯά
    z
    ˜
    p(z)
    ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ
    ͸ະ஌ɽ ͸ط஌ɽ
    p(z)
    p(z) ˜
    p(z)
    p(z)

    View Slide

  11. غ٫αϯϓϦϯά
    z
    ˜
    p(z)
    αϯϓϧ͕༰қͳఏҊ෼෍ Λઃఆɽ
    p(z)
    q(z)

    View Slide

  12. غ٫αϯϓϦϯά
    z
    ˜
    p(z)
    kq(z)
    Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ
    ˜
    p(z) k
    kq(z) > ˜
    p(z)
    q(z)
    × k

    View Slide

  13. غ٫αϯϓϦϯά
    z
    ˜
    p(z)
    kq(z)
    z(t)
    ఏҊ෼෍ ͔ΒαϯϓϧΛಘΔɽ
    q(z)
    z(t) ∼ q(z)
    q(z)

    View Slide

  14. غ٫αϯϓϦϯά
    z
    ˜
    p(z)
    kq(z)
    z(t)
    kq(z(t))
    ˜
    u
    Ұ༷෼෍ ͔ΒͷαϯϓϧΛಘΔ
    Uni(0,kq(z))
    ˜
    u ∼ Uni (0,kq(z(t)))

    View Slide

  15. غ٫αϯϓϦϯά
    z
    ˜
    p(z)
    kq(z)
    z(t)
    kq(z(t))
    ड༰
    غ٫
    ˜
    u
    αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબ୒ɽ
    z(t)
    if ˜
    u > ˜
    p(z(t)) then SFKFDU else BDDFQU
    ˜
    p(z(t))

    View Slide

  16. ࣗݾਖ਼نԽॏ৺αϯϓϦϯά
    ɹ෼෍ ʹର͢Δؔ਺ ͷظ଴஋ɹ Λ୯७ϞϯςΧϧϩ๏Α
    Γ΋ޮ཰తʹٻΊ͍ͨɽɹ
    p(z) f(z) p(z)
    [ f(z)] =

    f(z)p(z)dz
    ໨త
    ঢ়گ
    ɹظ଴஋ ͷղੳతͳੵ෼ܭࢉ͕ࠔ೉ɽ
    ɹ෼෍ ͔Β௚઀αϯϓϦϯάΛಘΒΕͳ͍ɽ
    ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ܭࢉՄೳɽ

    f(z)p(z)dz
    p(z)
    ˜
    p(z)( = Zp
    ⋅ p(z))
    എܠ
    ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ Λ࢖ΘͣʹαϯϓϧΛऔಘ͠ɼظ଴஋ Λٻ
    ΊΔ͜ͱ΋Ͱ͖Δ͕ɼ ͷ஋͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ
    ୯७ϞϯςΧϧϩ๏ͷܭࢉ΁ͷد༩͕গͳ͍ɽ ͷ஋͕େ͖͘ͳΔΑ͏ͳ
    ྖҬΛॏ఺తʹαϯϓϧͨ͠ํ͕ޮ཰͕͍͍ɽ
    p(z) p(z)
    [ f(z)]
    f(z)
    ⟹ f(z)p(z)

    View Slide

  17. ࣗݾਖ਼نԽॏ৺αϯϓϦϯά
    ɹ·ͣɼఏҊ෼෍ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ɼҎԼͷΑ͏ʹද
    ͞ΕΔɽ

    ɹظ଴஋ͷܭࢉ͸ɼҎԼͷΑ͏ʹมܗͰ͖Δɽ

    q(z) ˜
    p(z), ˜
    q(z)
    p(z) =
    1
    Zp
    ˜
    p(z), q(z) =
    1
    Zq
    ˜
    q(z)

    f(z)p(z)dz =

    f(z)
    p(z)
    q(z)
    q(z)dz = q(z)
    [ f(z)
    p(z)
    q(z)
    ]
    =

    f(z)
    1
    Zp
    ˜
    p(z)
    1
    Zq
    ˜
    q(z)
    q(z)dz =
    Zq
    Zp
    q(z)
    [ f(z)
    ˜
    p(z)
    ˜
    q(z)
    ]

    Zq
    Zp
    1
    T
    T

    t=1
    f(z(t))
    ˜
    p(z(t))
    ˜
    q(z(t))
    =
    Zq
    Zp
    1
    T
    T

    t=1
    f(z(t))w(t),
    w(t) =
    ˜
    p(z(t))
    ˜
    q(z(t))
    ख๏

    View Slide

  18. ࣗݾਖ਼نԽॏ৺αϯϓϦϯά
    ɹ ͸ະ஌ͷ஋ͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ


    ɹΑͬͯɼؔ਺ ͷظ଴஋͕ۙࣅతʹಘΒΕΔɽ
    Zp
    Zp
    Zq
    =

    ˜
    p(z)
    Zq
    dz
    =

    ˜
    p(z)
    ˜
    q(z)
    q(z)dz
    = q(z) [
    ˜
    p(z)
    ˜
    q(z) ]

    1
    T
    T

    t=1
    w(t),
    z(1), …, z(T) ∼ q(z)
    f(z)
    ख๏ ͖ͭͮʣ
    (
    ∵ p(z) =
    1
    Zp
    ˜
    p(z)
    )
    (


    p(z)dz =
    1
    Zp

    ˜
    p(z)dz = 1
    )

    View Slide

  19. ࣗݾਖ਼نԽॏ৺αϯϓϦϯά
    ɹ ͸ະ஌ͷ஋ͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ


    ɹΑͬͯɼؔ਺ ͷظ଴஋͕ۙࣅతʹಘΒΕΔɽ
    Zp
    Zp
    Zq
    =

    ˜
    p(z)
    Zq
    dz
    =

    ˜
    p(z)
    ˜
    q(z)
    q(z)dz
    = q(z) [
    ˜
    p(z)
    ˜
    q(z) ]

    1
    T
    T

    t=1
    w(t),
    z(1), …, z(T) ∼ q(z)
    f(z)
    ख๏ ͖ͭͮʣ
    (
    ∵ p(z) =
    1
    Zp
    ˜
    p(z)
    )
    (


    p(z)dz =
    1
    Zp

    ˜
    p(z)dz = 1
    )
    ͳΜͷͨΊʹ Λ
    ͖࣋ͬͯͨΜͩΖ͏ʜ
    ˜
    q(z)

    View Slide

  20. ࣗݾਖ਼نԽॏ఺αϯϓϦϯάʢޡهͷՄೳੑʣ
    ʮϕΠζਂ૚ֶशʯQ
    ɹࣗݾਖ਼نԽॏ఺αϯϓϦϯάͷஈམ໨ͷ࠷ॳ
    ޡΓ
    ɹغ٫αϯϓϦϯάͱҟͳΔ఺͸ɼ ͔Βͷαϯϓϧʜʜ
    p(z)
    ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔ఺͸ɼ ͔Βͷαϯϓϧʜʜ
    p(z)

    View Slide

  21. Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ
    ɹغ٫αϯϓϦϯάͷ໰୊఺͸ɼߴ࣍ݩʹͳΔͱड༰཰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ
    ʹ͸ɼ࣍ݩఔ౓ͷ؆୯ͳੵ෼ۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ
    ɹͰ͸ɼߴ࣍ݩۭؒͰޮ཰తʹαϯϓϦϯά͢Δʹ͸ʜʜ
    ɹɹɹ Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ

    ࣍Ϛϧίϑ࿈࠯
    ɹ֬཰ม਺ͷܥྻ ʹରͯ͠

    ͕੒Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ
    z(1), z(2), …
    p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1))
    z(1), z(2), …
    άϥϑΟΧϧϞσϧ
    z(1) z(2) z(t−1) z(t)

    ɹભҠ֬཰ɹΛ ͱ͓͍ͨͱ͖ɼ

    ͕੒Γཱͭͱ͖ɼ Λɹఆৗ෼෍ɹͱ͍͏ɽ
    (z(t−1), z(t)) = p(z(t) |z(t−1))
    p*
    (z) =

    (z′ , z)p*
    (z′ )dz′
    p*
    (z)

    View Slide

  22. Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ
    ɹఆৗ෼෍ ʹऩଋ͢ΔΑ͏ͳભҠ֬཰ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ
    ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗ෼෍ʹີ౓ܭࢉ͕ࠔ೉ͳ֬཰෼෍Λ͓͘ɽ
    p*
    (z) (z(t−1), z(t)) p*
    (z)

    ख๏ͷΩϞ
    ৄࡉ௼Γ߹͍৚݅
    p*
    (z)(z, z′ ) = p*
    (z′ )(z′ , z)
    ʲे෼৚݅ʳৄࡉ௼Γ߹͍৚͕݅੒Γཱͭ ͸ఆৗ෼෍ͱͳΔɽ
    ⟹ p*
    (z)
    p*
    (z)(z, z′ ) = p*
    (z′ )(z′ , z)


    p*
    (z)(z, z′ )dz′ =

    p*
    (z′ )(z′ , z)dz′
    ⇔ p*
    (z)

    p(z′ |z)dz′ =

    p*
    (z′ )(z′ , z)dz′
    ⇔ p*
    (z) =

    p*
    (z′ )(z′ , z)dz′

    View Slide

  23. Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ
    ɹ௼Γ߹͍৚݅ʹՃ͑ͯɼαϯϓϧ਺͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ
    ͣɼ ͕ఆৗ෼෍ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ
    t → ∞ p(z(1))
    p(z(t)) p*
    (z) ⟹
    Τϧΰʔυੑ
    w ط໿ੑɹɿ೚ҙͷঢ়ଶ͔Β೚ҙͷঢ়ଶ΁༗ݶճ਺ͰભҠՄೳɽ
    w ඇपظੑɿ͢΂ͯͷঢ়ଶ͕ݻఆͷपظੑΛ΋ͨͳ͍ɽ
    w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰ໭Δ͜ͱ͕Մೳɽ

    View Slide

  24. ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ
    ໨త
    ɹະ஌ͷ֬཰෼෍ ͔ΒαϯϓϦϯάΛಘΔɽ
    p(z)
    લఏ
    ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ط஌Ͱ͋Δɽ
    p(z) ∝ ˜
    p(z) ˜
    p(z)
    ख๏
    ɹભҠ֬཰ ͕௚઀ઃܭ͕೉͍͠৔߹͸ɼભҠͷఏҊ෼෍ Λ࢖͑Δɽ
    (z′ , z) q(z|z′ )
    ɽఏҊ෼෍ ͔Β࣍ͷαϯϓϧ఺ͷީิ ΛαϯϓϦϯά͢Δɽ
    ɽ࣍ͷൺ཰ Λܭࢉ͢Δɽ

    ɽ Λ֬཰ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍৔߹
    ͸ɼ ͱ͢Δɽ
    q( ⋅ |z(t)) z*
    r
    r =
    ˜
    p(z*
    )q(z(t) |z*
    )
    ˜
    p(z(t))q(z*
    |z(t))
    z*
    min(1,r) z(t+1) ⟵ z*
    z(t+1) ⟵ z(t)
    ΞϧΰϦζϜͷྲྀΕ

    View Slide

  25. ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ
    ৄࡉ௼Γ߹͍৚݅ͷূ໌
    ɹભҠ֬཰͸ɼҎԼͷΑ͏ʹͳΔɽ
    (z, z′ ) = q(z′ |z) min (1,
    ˜
    p(z′ )q(z|z′ )
    ˜
    p(z)q(z′ |z) )
    p(z)(z, z′ ) = p(z)q(z′ |z) min (1,
    ˜
    p(z′ )q(z|z′ )
    ˜
    p(z)q(z′ |z) )
    = p(z)q(z′ |z) min (1,
    p(z′ )q(z|z′ )
    p(z)q(z′ |z) )
    = min (p(z)q(z′ |z), p(z′ )q(z|z′ ))
    = min (p(z′ )q(z|z′ ), p(z)q(z′ |z))
    = p(z′ )q(z|z′ ) min (1,
    p(z)q(z′ |z)
    p(z′ )q(z|z′ ))
    = p(z′ )q(z|z′ ) min (1,
    ˜
    p(z)q(z′ |z)
    ˜
    p(z′ )q(z|z′ ))
    = p(z′ )(z′ , z)
    ɹ ͷ৔߹͸ɼϝτϩϙϦε๏ͱݺ͹ΕΔɽ
    q(z′ |z) = q(z|z′ )

    View Slide

  26. ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ
    ʮϕΠζਂ૚ֶशʯQࣜʢʣ
    ޡΓ
    (z, z′ ) = q(z′ |z) min (1,
    ˜
    p(z′ )q(z|z′ )
    ˜
    p(z)q(z′ |z) )
    (z, z′ ) = q(z|z′ ) min (1,
    ˜
    p(z′ )q(z|z′ )
    ˜
    p(z)q(z′ |z) )

    View Slide

  27. ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ
    ۩ମྫͰֶͿ
    ໨ඪ෼෍ʢະ஌ʣ
    ɹɹɹɹɹɹɹ
    ਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ʢط஌ʣ

    ఏҊ෼෍
    ͭ·Γ

    p(z) = (z|μ, Σ) =
    1
    (2π)D |Σ|
    exp {−
    1
    2
    (z − μ)TΣ−1(z − μ)}
    ˜
    p(z) = exp {−
    1
    2
    (z − μ)TΣ−1(z − μ)}
    q(z*
    |z) = (z′ |z, I) z*
    ∼ (z, I)
    https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3

    View Slide

  28. ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳ਺஋γϛϡϨʔγϣϯΛղ
    આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯ͸ҎԼͷΑ͏ʹ
    ද͞ΕΔɽ
    ℋ(z, p) = (z) + (p),
    (p) =
    1
    2m
    pTp,
    z ∈ ℝD: ෺ମͷҐஔϕΫτϧ,
    p ∈ ℝD: ෺ମͷӡಈྔϕΫτϧ,
    m ∈ ℝ: ෺ମͷ࣭ྔ,
    ℋ(z, p): ϋϛϧτχΞϯ,
    (z): ϙςϯγϟϧΤωϧΪʔ,
    (p): ӡಈΤωϧΪʔ .
    ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹ෺ମͷيಓͷझຯϨʔγϣϯ .)๏
    ×
    ɹ).$๏͸ɼϥϯμϜ΢ΥʔΫతͳ.)ͱൺ΂ͯɼޮ཰తʹۭؒΛ୳ࡧՄೳɽ
    ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ

    View Slide

  29. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢Δڍಈ͸ɼϋϛϧτχΞϯͷภඍ෼ʹΑܾͬͯఆɽ

    ͜ͷඍ෼ํఔ͕ࣜղੳతʹղ͚ͳ͍΋ͷͱ͠ɼ਺஋γϛϡϨʔγϣϯʹΑͬͯيಓΛܭ
    ࢉ͢Δɽ
    z p τ
    dpi

    = −
    dℋ
    dzi
    = −
    d
    dzi
    ,
    dzi

    =
    dℋ
    dpi
    =
    d
    dpi
    .
    ΦΠϥʔ๏
    ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ
    ϵ > 0
    pi
    (τ + ϵ) = pi
    (τ) + ϵ
    dpi

    τ
    = pi
    (τ) − ϵ
    d
    dzi
    zi
    (τ)
    ,
    zi
    (τ + ϵ) = zi
    (τ) + ϵ
    dzi

    τ
    = zi
    (τ) + ϵpi
    (τ)
    ໰୊఺
    ཭ࢄԽʹΑΔ਺஋ޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏

    View Slide

  30. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    Ϧʔϓϑϩοά๏
    pi (τ +
    ϵ
    2 ) = pi
    (τ) −
    ϵ
    2
    d
    dzi
    zi
    (τ)
    ,
    zi
    (τ + ϵ) = zi
    (τ) + ϵpi (τ +
    ϵ
    2 ),
    pi
    (τ + ϵ) = pi (τ +
    ϵ
    2 ) −
    ϵ
    2
    d
    dzi
    zi
    (τ + ϵ)
    .
    ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷ෺ମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ
    L ϵL z*
    p*
    ϋϛϧτχΞϯͷੑ࣭
    ɽ ͸࣌ؒ ʹΑͬͯෆมɽ
    ɽՄٯੑɿ ͔Β ͷભҠ͸ҰରҰɽ
    ɽମੵอଘ
    ℋ τ
    (z, p) (z*
    , p*
    )

    View Slide

  31. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    αϯϓϦϯάΞϧΰϦζϜ΁ͷద༻
    ໨త
    ɹະ஌ͷ֬཰෼෍ ͔ΒαϯϓϦϯάΛಘΔɽ
    p(z)
    લఏ
    ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ط஌Ͱ͋Δɽ
    ɹ ͱ֦ு͢Δͱɼ ͸पล෼෍ ͔Βαϯϓϧ͕ಘΒΕΔɽ
    ɹ

    p(z) ∝ ˜
    p(z) ˜
    p(z)
    p(z, p) = p(z)p(p) z p(z)
    p(p) = (p|0, I)
    (z) = − log (˜
    p(z))
    (p) =
    1
    2
    pTp

    View Slide

  32. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    ɹಉ࣌෼෍Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ

    ϝτϩϙϦε๏Ͱ࢖ΘΕΔൺ཰ ͸ɼҎԼͷΑ͏ʹͳΔɽ
    p(z, p) = p(z)p(p)
    = exp (log p(z) + log p(p))
    ∝ exp (log ˜
    p(z) −
    1
    2
    pTIp
    )
    = exp (−(z) − (p))
    = exp (−ℋ(z, p))
    r
    r =
    p(z*
    , p*
    )
    p(z, p)
    = exp (−ℋ(z*
    , p*
    ) + ℋ(z, p))

    View Slide

  33. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    ख๏
    ɽӡಈྔΛαϯϓϦϯά

    ɽϦʔϓϑϩοά๏Ͱݱࡏͷ఺ ͔Βީิ఺ ΛಘΔɽ
    ɽ࣍ͷൺ཰ Λܭࢉ͢Δɽ

    ɽ Λ֬཰ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍৔߹
    ͸ɼ ͱ͢Δɽ
    p ∼ (0, I)
    (z(t), p) (z*
    , p*
    )
    r
    r =
    p(z*
    , p*
    )
    p(z, p)
    z*
    min(1,r) z(t+1) ⟵ z*
    z(t+1) ⟵ z(t)
    ΞϧΰϦζϜͷྲྀΕ

    View Slide

  34. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ
    ɹ).$๏͸ɼࣄޙ෼෍ͷඍ෼͑͞ܭࢉͰ͖Ε͹ద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ
    χϡʔϥϧωοτϫʔΫ͸࿈ଓͳજࡏม਺ͷΈͰ੒Γཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$
    ๏͸χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹ࢖ΘΕ͖ͯͨɽ
    େ͖͍
    εςοϓαΠζ
    ϵ
    εςοϓ਺
    L
    খ͍͞ খ͍͞
    େ͖͍
    ड༰཰ ड༰཰
    ୳ࡧޮ཰ ܭࢉྔ
    ߴ͍ ߴ͍
    ௿͍ ௿͍
    େ͖͍ খ͍͞
    ߴ͍
    ௿͍

    View Slide

  35. ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    ϥϯδϡόϯಈྗֶ๏
    ɹ ͱͨ͠৔߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨ͸ɹϥϯδϡόϯಈྗֶ๏ɹͱ
    ݺ͹ΕΔɽ

    ɹਂ૚ֶश޲͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬཰తޯ഑ϥϯδϡόϯಈྗֶ
    ๏ɹʹల։͞ΕΔɽ
    L = 1
    z*i
    = zi
    (τ + ϵ)
    = zi
    (τ) + ϵ pi
    (τ) −
    ϵ
    2
    d
    dzi
    zi
    (τ)
    = zi
    (τ) −
    ϵ2
    2
    d
    dzi
    zi
    (τ)
    + ϵpi
    (τ)

    View Slide

  36. ໨ඪ෼෍ʢະ஌ʣ
    ɹɹɹɹɹɹɹ
    ਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ʢط஌ʣ

    ӡಈྔͷαϯϓϦϯάɿ
    ӡಈΤωϧΪʔɿ
    ҐஔΤωϧΪʔɿ
    ҐஔΤωϧΪʔͷภඍ෼ɿ
    p(z) = (z|μ, Σ) =
    1
    (2π)D |Σ|
    exp {−
    1
    2
    (z − μ)TΣ−1(z − μ)}
    ˜
    p(z) = exp {−
    1
    2
    (z − μ)TΣ−1(z − μ)}
    p ∼ (0, I)
    (p) =
    1
    2
    pTp
    (z) = − log (˜
    p(z)) = −
    1
    2
    (z − μ)TΣ−1(z − μ)

    ∂z
    = − (z − μ)TΣ−1
    ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ
    ۩ମྫͰֶͿ
    https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ

    View Slide

  37. ΪϒεαϯϓϦϯά
    ໨త
    ɹ֬཰෼෍ ͔Β௚઀ શମΛαϯϓϦϯά͢Δ͜ͱ͕೉͍͠ͱ͖ͷ୳ࡧɽ
    p(Z) Z
    લఏ
    ɹ֬཰෼෍ ͸ط஌ɽ
    p(Z)
    ख๏
    ɹɽม਺ Λ ݸͷ෦෼ू߹ʹ෼͚Δɽ

    ɹɽ෦෼ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ
    ɹ
    Z M
    Z = {Z1
    , Z2
    , …, ZM
    }
    Z1
    ∼ p(Z1
    |Z2
    , Z3
    , …, ZM−1
    , ZM
    )
    Z2
    ∼ p(Z2
    |Z1
    , Z3
    , …, ZM−1
    , ZM
    )

    ZM
    ∼ p(ZM
    |Z1
    , Z2
    , …, ZM−2
    , ZM−1
    )

    View Slide

  38. ΪϒεαϯϓϦϯά
    ɹΪϒεαϯϓϦϯάͷଥ౰ੑ͸ɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ
    ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ෼͚ɼ Λ৚݅෇͚ͨ͠΋ͱͰ ͷα
    ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ৔߹ɼ ͔ͭ
    ɹൺ཰ Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ

    Αͬͯɼશͯड༰͞ΕΔɽ ΋ಉ༷ɽ
    Z = {Z1
    , Z2
    } Z2
    Z1
    q(Z*
    |Z) = p(Z1*
    |Z2*
    ) Z2
    = Z2*
    r
    r =
    p(Z*
    )q(Z|Z*
    )
    p(Z)q(Z*
    |Z)
    =
    p(Z1*
    , Z2*
    )p(Z1
    |Z2*
    )
    p(Z1
    , Z2
    )p(Z1*
    |Z2
    )
    =
    p(Z1*
    |Z2*
    )p(Z2*
    )p(Z1
    |Z2*
    )
    p(Z1
    |Z2
    )p(Z2
    )p(Z1*
    |Z2
    )
    = 1
    Z2

    View Slide

  39. ΪϒεαϯϓϦϯά
    ۩ମྫͰֶͿ
    ໨ඪ෼෍ʢط஌ʣ
    ɹɹɹɹ
    ͱ͢Δɽ

    p(z) = (z|μ, Σ) =
    1
    (2π)D |Σ|
    exp {−
    1
    2
    (z − μ)TΣ−1(z − μ)}
    z = (
    z1
    z2
    ), μ = (
    μ1
    μ2
    ), Σ = (
    Σ11
    Σ12
    Σ21
    Σ22
    ), Λ = Σ−1 = (
    Λ11
    Λ12
    Λ21
    Λ22
    )
    log p(z) = log p(z1
    , z2
    )
    =
    − 1
    2
    (z1
    − μ1
    )TΛ11
    (z1
    − μ1
    ) + (z1
    − μ1
    )TΛ12
    (z2
    − μ2
    ))
    − 1
    2
    (z2
    − μ2
    )TΛ22
    (z2
    − μ2
    ) + (z2
    − μ2
    )TΛ21
    (z1
    − μ1
    ))
    =
    − 1
    2 (
    zT
    1
    Λ11
    z1
    − 2z1 {Λ11
    μ1
    − 1
    2
    Λ12
    (z2
    − μ2
    )}) + C1
    − 1
    2 (
    zT
    2
    Λ22
    z2
    − 2z2 {Λ22
    μ2
    − 1
    2
    Λ21
    (z1
    − μ1
    )}) + C2

    View Slide

  40. ΪϒεαϯϓϦϯά
    ۩ମྫͰֶͿ
    Αͬͯɼ৚݅෇͖֬཰ͷର਺͸ɼ

    ͱͳΔͷͰɼ৚݅෇͖֬཰͸Ψ΢ε෼෍Ͱ͋Δɼ


    log p(zi
    |zj
    ) = −
    1
    2 (
    zT
    i
    Λii
    zi
    − 2zi {
    Λii
    μi

    1
    2
    Λij
    (zj
    − μj
    )})
    + C
    p(zi
    |zj
    ) = (zi
    |μi
    , Σi
    ),
    Σ−1
    i
    = Λii
    ,
    Σ−1
    i
    μi
    = Λii
    μi

    1
    2
    Λij
    (zj
    − μj
    ),
    ⇔ μi
    = Σi (
    Λii
    μi

    1
    2
    Λij
    (zj
    − μj
    )) .
    https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0

    View Slide