ベイズ深層学習(4.1)

ϕΠζਂ૚ֶश αϯϓϦϯάʹجͮ͘ਪ࿦ख๏ ܡɹঘً

αϯϓϦϯάʹجͮ͘ਪ࿦ख๏ ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষ΋ࢀߟ

αϯϓϦϯάʹجͮ͘ਪ࿦ख๏ ɹ؍ଌσʔλΛ ɼඇ؍ଌͷม਺ͷू߹ʢFHύϥϝʔλɼજࡏม਺ Λ ͱͨ͠ͱ ͖ɼϕΠζਪ࿦ʹΑΔ౷ܭղੳͰ͸ɼ֬཰Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ ɹ࣮ࡍʹɼ֬཰ϞσϧΛ༻ֶ͍ͯश΍༧ଌ͸ɼࣄޙ෼෍ Λܭࢉͯ͠ߦΘΕΔɽ X
Z p(X, Z) p(Z|X) ໰୊఺ ղܾࡦ ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣ͸ɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕ ଟ͍ɽ p(Z|X) ɹ ΛղੳతʹٻΊΔ୅ΘΓʹɼ͜ͷ෼෍͔Βෳ਺ͷαϯϓϧΛಘΔ͜ͱͰɼ෼෍ ͷಛੑΛௐ΂Δɽ ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճ͸ษڧ͢ΔΑʂ p(Z|X)

ຊ೔ͷ಺༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ఺αϯϓϦϯά ʢϚϧίϑ࿈࠯ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά

ຊ೔ͷ಺༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ʢϚϧίϑ࿈࠯ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
.$.$

୯७ϞϯςΧϧϩ๏ ɹ෼෍ ʹର͢Δؔ਺ ͷظ଴஋ɹ ΛٻΊ͍ͨɽɹ p(z) f(z) p(z) [ f(z)]
= ∫ f(z)p(z)dz ໨త ঢ়گ ɹظ଴஋ ͷղੳతͳੵ෼ܭࢉ͕ࠔ೉ɽ ɹ෼෍ ͔ΒͷαϯϓϦϯά͸༰қɽ ∫ f(z)p(z)dz p(z) ख๏ ɹ Λे෼େ͖ͳ஋ͱͨ͠ͱ͖ɼ T z(1), z(2), …, z(T) ∼ p(z) ∫ f(z)p(z)dz ≈ 1 T T ∑ t=1 f(z(t)) ͔Β ݸαϯϓϦϯά ⟵ p(z) T

ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล໬౓ Λܭࢉ͢Δࡍʹ࢖༻͢ Δ৔߹ɼ θ p(X, θ) =
p(X|θ)p(θ) p(X) p(X) = ∫ p(X|θ)p(θ)dθ = ∫ N ∏ n=1 p(xn |θ)p(θ)dθ = p(θ) [p(X|θ)] ≈ 1 T T ∑ t=1 N ∏ n=1 p(xn |θ(t)), (θ(1), …, θ(T) ∼ p(θ)) ɹظ଴஋ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ͸෯޿͘ͱΔඞཁ͕͋ΓɼҰํ Ͱɼ ͸ڱ͍ ͷൣғͰ͔͠େ͖ͳ஋ΛऔΒͳ͍έʔε͕ଟ͍ɽ ɹ ൚༻త͚ͩͲɼܭࢉޮ཰͕ѱ͍ɽ p(z) [ f(z)] p(z) z f(z) z ⟹ ୯७ϞϯςΧϧϩ๏ ໰୊఺

غ٫αϯϓϦϯά ɹີ౓ܭࢉ͕ࠔ೉ͳ֬཰෼෍ ͔ΒαϯϓϧΛಘΔɽɹ p(z) z(1), z(2), … ∼ p(z) ໨త
ঢ়گ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ܭࢉՄೳɽͭ·Γɼ ɽ ˜ p(z)( = Zp ⋅ p(z)) ∫ ˜ p(z)dz ≠ 1 ख๏ ɹఏҊ෼෍ Λઃఆ͢Δɽ೚ҙͷ ʹରͯ͠ɼ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ͱͳΔΑ͏ʹɼਖ਼ͷఆ਺ ΛఆΊΔɽ q(z) z kq(z) > ˜ p(z) k ఏҊ෼෍ ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ Ծͷ෼෍ɽ

غ٫αϯϓϦϯά ख๏ ͖ͭͮʣ ɹఏҊ෼෍ ͔ΒαϯϓϧΛಘΔɽ ɹҰ༷෼෍ ͔ΒͷαϯϓϧΛಘΔɽ
ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબ୒ɽ ɹɹɹ q(z) z(t) ∼ q(z) Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t))) z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ड༰཰ ∫ q(z) ˜ p(z) kq(z) dz = 1 k ∫ ˜ p(z)dz ໰୊఺ ߴ࣍ݩͷม਺ͷαϯϓϦϯά͕ඞཁͳ৔߹ɼड༰཰͕ඇৗʹ௿͘ͳΔɽ

غ٫αϯϓϦϯά z ˜ p(z) ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ ͸ະ஌ɽ ͸ط஌ɽ p(z) p(z) ˜
p(z) p(z)

غ٫αϯϓϦϯά z ˜ p(z) αϯϓϧ͕༰қͳఏҊ෼෍ Λઃఆɽ p(z) q(z)

غ٫αϯϓϦϯά z ˜ p(z) kq(z) Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ ˜ p(z) k
kq(z) > ˜ p(z) q(z) × k

غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) ఏҊ෼෍ ͔ΒαϯϓϧΛಘΔɽ q(z) z(t)
∼ q(z) q(z)

غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ˜ u Ұ༷෼෍
͔ΒͷαϯϓϧΛಘΔ Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t)))

غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ड༰ غ٫ ˜
u αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબ୒ɽ z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ˜ p(z(t))

ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ෼෍ ʹର͢Δؔ਺ ͷظ଴஋ɹ Λ୯७ϞϯςΧϧϩ๏Α Γ΋ޮ཰తʹٻΊ͍ͨɽɹ p(z) f(z) p(z) [
f(z)] = ∫ f(z)p(z)dz ໨త ঢ়گ ɹظ଴஋ ͷղੳతͳੵ෼ܭࢉ͕ࠔ೉ɽ ɹ෼෍ ͔Β௚઀αϯϓϦϯάΛಘΒΕͳ͍ɽ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ܭࢉՄೳɽ ∫ f(z)p(z)dz p(z) ˜ p(z)( = Zp ⋅ p(z)) എܠ ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ Λ࢖ΘͣʹαϯϓϧΛऔಘ͠ɼظ଴஋ Λٻ ΊΔ͜ͱ΋Ͱ͖Δ͕ɼ ͷ஋͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ ୯७ϞϯςΧϧϩ๏ͷܭࢉ΁ͷد༩͕গͳ͍ɽ ͷ஋͕େ͖͘ͳΔΑ͏ͳ ྖҬΛॏ఺తʹαϯϓϧͨ͠ํ͕ޮ཰͕͍͍ɽ p(z) p(z) [ f(z)] f(z) ⟹ f(z)p(z)

ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ·ͣɼఏҊ෼෍ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ɼҎԼͷΑ͏ʹද ͞ΕΔɽ ɹظ଴஋ͷܭࢉ͸ɼҎԼͷΑ͏ʹมܗͰ͖Δɽ q(z)
˜ p(z), ˜ q(z) p(z) = 1 Zp ˜ p(z), q(z) = 1 Zq ˜ q(z) ∫ f(z)p(z)dz = ∫ f(z) p(z) q(z) q(z)dz = q(z) [ f(z) p(z) q(z) ] = ∫ f(z) 1 Zp ˜ p(z) 1 Zq ˜ q(z) q(z)dz = Zq Zp q(z) [ f(z) ˜ p(z) ˜ q(z) ] ≈ Zq Zp 1 T T ∑ t=1 f(z(t)) ˜ p(z(t)) ˜ q(z(t)) = Zq Zp 1 T T ∑ t=1 f(z(t))w(t), w(t) = ˜ p(z(t)) ˜ q(z(t)) ख๏

ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ͸ະ஌ͷ஋ͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ਺ ͷظ଴஋͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 )

ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ͸ະ஌ͷ஋ͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ਺ ͷظ଴஋͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 ) ͳΜͷͨΊʹ Λ ͖࣋ͬͯͨΜͩΖ͏ʜ ˜ q(z)

ࣗݾਖ਼نԽॏ఺αϯϓϦϯάʢޡهͷՄೳੑʣ ʮϕΠζਂ૚ֶशʯQ ɹࣗݾਖ਼نԽॏ఺αϯϓϦϯάͷஈམ໨ͷ࠷ॳ ޡΓ ɹغ٫αϯϓϦϯάͱҟͳΔ఺͸ɼ ͔Βͷαϯϓϧʜʜ p(z) ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔ఺͸ɼ ͔Βͷαϯϓϧʜʜ p(z)

Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹغ٫αϯϓϦϯάͷ໰୊఺͸ɼߴ࣍ݩʹͳΔͱड༰཰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ ʹ͸ɼ࣍ݩఔ౓ͷ؆୯ͳੵ෼ۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ ɹͰ͸ɼߴ࣍ݩۭؒͰޮ཰తʹαϯϓϦϯά͢Δʹ͸ʜʜ ɹɹɹ Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ ⟹ ࣍Ϛϧίϑ࿈࠯ ɹ֬཰ม਺ͷܥྻ
ʹରͯ͠ ͕੒Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ z(1), z(2), … p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1)) z(1), z(2), … άϥϑΟΧϧϞσϧ z(1) z(2) z(t−1) z(t) ⋯ ɹભҠ֬཰ɹΛ ͱ͓͍ͨͱ͖ɼ ͕੒Γཱͭͱ͖ɼ Λɹఆৗ෼෍ɹͱ͍͏ɽ (z(t−1), z(t)) = p(z(t) |z(t−1)) p* (z) = ∫ (z′ , z)p* (z′ )dz′ p* (z)

Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹఆৗ෼෍ ʹऩଋ͢ΔΑ͏ͳભҠ֬཰ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗ෼෍ʹີ౓ܭࢉ͕ࠔ೉ͳ֬཰෼෍Λ͓͘ɽ p* (z) (z(t−1),
z(t)) p* (z) ⟹ ख๏ͷΩϞ ৄࡉ௼Γ߹͍৚݅ p* (z)(z, z′ ) = p* (z′ )(z′ , z) ʲे෼৚݅ʳৄࡉ௼Γ߹͍৚͕݅੒Γཱͭ ͸ఆৗ෼෍ͱͳΔɽ ⟹ p* (z) p* (z)(z, z′ ) = p* (z′ )(z′ , z) ⇒ ∫ p* (z)(z, z′ )dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) ∫ p(z′ |z)dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) = ∫ p* (z′ )(z′ , z)dz′

Ϛϧίϑ࿈࠯ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹ௼Γ߹͍৚݅ʹՃ͑ͯɼαϯϓϧ਺͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ ͣɼ ͕ఆৗ෼෍ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ t →
∞ p(z(1)) p(z(t)) p* (z) ⟹ Τϧΰʔυੑ w ط໿ੑɹɿ೚ҙͷঢ়ଶ͔Β೚ҙͷঢ়ଶ΁༗ݶճ਺ͰભҠՄೳɽ w ඇपظੑɿ͢΂ͯͷঢ়ଶ͕ݻఆͷपظੑΛ΋ͨͳ͍ɽ w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰ໭Δ͜ͱ͕Մೳɽ

ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ໨త ɹະ஌ͷ֬཰෼෍ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ط஌Ͱ͋Δɽ p(z)
∝ ˜ p(z) ˜ p(z) ख๏ ɹભҠ֬཰ ͕௚઀ઃܭ͕೉͍͠৔߹͸ɼભҠͷఏҊ෼෍ Λ࢖͑Δɽ (z′ , z) q(z|z′ ) ɽఏҊ෼෍ ͔Β࣍ͷαϯϓϧ఺ͷީิ ΛαϯϓϦϯά͢Δɽ ɽ࣍ͷൺ཰ Λܭࢉ͢Δɽ ɽ Λ֬཰ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍৔߹ ͸ɼ ͱ͢Δɽ q( ⋅ |z(t)) z* r r = ˜ p(z* )q(z(t) |z* ) ˜ p(z(t))q(z* |z(t)) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ

ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ۩ମྫͰֶͿ ໨ඪ෼෍ʢະ஌ʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ʢط஌ʣ ఏҊ෼෍
ͭ·Γ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} q(z* |z) = (z′ |z, I) z* ∼ (z, I) https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3

ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳ਺஋γϛϡϨʔγϣϯΛղ આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯ͸ҎԼͷΑ͏ʹ ද͞ΕΔɽ ℋ(z, p) = (z) + (p), (p)
= 1 2m pTp, z ∈ ℝD: ෺ମͷҐஔϕΫτϧ, p ∈ ℝD: ෺ମͷӡಈྔϕΫτϧ, m ∈ ℝ: ෺ମͷ࣭ྔ, ℋ(z, p): ϋϛϧτχΞϯ, (z): ϙςϯγϟϧΤωϧΪʔ, (p): ӡಈΤωϧΪʔ . ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹ෺ମͷيಓͷझຯϨʔγϣϯ .)๏ × ɹ).$๏͸ɼϥϯμϜ΢ΥʔΫతͳ.)ͱൺ΂ͯɼޮ཰తʹۭؒΛ୳ࡧՄೳɽ ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢Δڍಈ͸ɼϋϛϧτχΞϯͷภඍ෼ʹΑܾͬͯఆɽ ͜ͷඍ෼ํఔ͕ࣜղੳతʹղ͚ͳ͍΋ͷͱ͠ɼ਺஋γϛϡϨʔγϣϯʹΑͬͯيಓΛܭ ࢉ͢Δɽ z
p τ dpi dτ = − dℋ dzi = − d dzi , dzi dτ = dℋ dpi = d dpi . ΦΠϥʔ๏ ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ ϵ > 0 pi (τ + ϵ) = pi (τ) + ϵ dpi dτ τ = pi (τ) − ϵ d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵ dzi dτ τ = zi (τ) + ϵpi (τ) ໰୊఺ ཭ࢄԽʹΑΔ਺஋ޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏ ⟹

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοά๏ pi (τ + ϵ 2 ) = pi
(τ) − ϵ 2 d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵpi (τ + ϵ 2 ), pi (τ + ϵ) = pi (τ + ϵ 2 ) − ϵ 2 d dzi zi (τ + ϵ) . ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷ෺ମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ L ϵL z* p* ϋϛϧτχΞϯͷੑ࣭ ɽ ͸࣌ؒ ʹΑͬͯෆมɽ ɽՄٯੑɿ ͔Β ͷભҠ͸ҰରҰɽ ɽମੵอଘ ℋ τ (z, p) (z* , p* )

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ αϯϓϦϯάΞϧΰϦζϜ΁ͷద༻ ໨త ɹະ஌ͷ֬཰෼෍ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ ͸ط஌Ͱ͋Δɽ
ɹ ͱ֦ு͢Δͱɼ ͸पล෼෍ ͔Βαϯϓϧ͕ಘΒΕΔɽ ɹ p(z) ∝ ˜ p(z) ˜ p(z) p(z, p) = p(z)p(p) z p(z) p(p) = (p|0, I) (z) = − log (˜ p(z)) (p) = 1 2 pTp

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹಉ࣌෼෍Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ ϝτϩϙϦε๏Ͱ࢖ΘΕΔൺ཰ ͸ɼҎԼͷΑ͏ʹͳΔɽ p(z, p) = p(z)p(p) =
exp (log p(z) + log p(p)) ∝ exp (log ˜ p(z) − 1 2 pTIp ) = exp (−(z) − (p)) = exp (−ℋ(z, p)) r r = p(z* , p* ) p(z, p) = exp (−ℋ(z* , p* ) + ℋ(z, p))

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ख๏ ɽӡಈྔΛαϯϓϦϯά ɽϦʔϓϑϩοά๏Ͱݱࡏͷ఺ ͔Βީิ఺ ΛಘΔɽ ɽ࣍ͷൺ཰ Λܭࢉ͢Δɽ
ɽ Λ֬཰ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍৔߹ ͸ɼ ͱ͢Δɽ p ∼ (0, I) (z(t), p) (z* , p* ) r r = p(z* , p* ) p(z, p) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ ɹ).$๏͸ɼࣄޙ෼෍ͷඍ෼͑͞ܭࢉͰ͖Ε͹ద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ χϡʔϥϧωοτϫʔΫ͸࿈ଓͳજࡏม਺ͷΈͰ੒Γཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$ ๏͸χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹ࢖ΘΕ͖ͯͨɽ େ͖͍ εςοϓαΠζ ϵ εςοϓ਺ L
খ͍͞ খ͍͞ େ͖͍ ड༰཰ ड༰཰ ୳ࡧޮ཰ ܭࢉྔ ߴ͍ ߴ͍ ௿͍ ௿͍ େ͖͍ খ͍͞ ߴ͍ ௿͍

ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ϥϯδϡόϯಈྗֶ๏ ɹ ͱͨ͠৔߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨ͸ɹϥϯδϡόϯಈྗֶ๏ɹͱ ݺ͹ΕΔɽ ɹਂ૚ֶश޲͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬཰తޯ഑ϥϯδϡόϯಈྗֶ ๏ɹʹల։͞ΕΔɽ L =
1 z*i = zi (τ + ϵ) = zi (τ) + ϵ pi (τ) − ϵ 2 d dzi zi (τ) = zi (τ) − ϵ2 2 d dzi zi (τ) + ϵpi (τ)

໨ඪ෼෍ʢະ஌ʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔ਺ʢط஌ʣ ӡಈྔͷαϯϓϦϯάɿ ӡಈΤωϧΪʔɿ
ҐஔΤωϧΪʔɿ ҐஔΤωϧΪʔͷภඍ෼ɿ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} p ∼ (0, I) (p) = 1 2 pTp (z) = − log (˜ p(z)) = − 1 2 (z − μ)TΣ−1(z − μ) ∂ ∂z = − (z − μ)TΣ−1 ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ۩ମྫͰֶͿ https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ

ΪϒεαϯϓϦϯά ໨త ɹ֬཰෼෍ ͔Β௚઀ શମΛαϯϓϦϯά͢Δ͜ͱ͕೉͍͠ͱ͖ͷ୳ࡧɽ p(Z) Z લఏ ɹ֬཰෼෍ ͸ط஌ɽ
p(Z) ख๏ ɹɽม਺ Λ ݸͷ෦෼ू߹ʹ෼͚Δɽ ɹɽ෦෼ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ ɹ Z M Z = {Z1 , Z2 , …, ZM } Z1 ∼ p(Z1 |Z2 , Z3 , …, ZM−1 , ZM ) Z2 ∼ p(Z2 |Z1 , Z3 , …, ZM−1 , ZM ) ⋮ ZM ∼ p(ZM |Z1 , Z2 , …, ZM−2 , ZM−1 )

ΪϒεαϯϓϦϯά ɹΪϒεαϯϓϦϯάͷଥ౰ੑ͸ɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ෼͚ɼ Λ৚݅෇͚ͨ͠΋ͱͰ ͷα ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ৔߹ɼ ͔ͭ ɹൺ཰
Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ Αͬͯɼશͯड༰͞ΕΔɽ ΋ಉ༷ɽ Z = {Z1 , Z2 } Z2 Z1 q(Z* |Z) = p(Z1* |Z2* ) Z2 = Z2* r r = p(Z* )q(Z|Z* ) p(Z)q(Z* |Z) = p(Z1* , Z2* )p(Z1 |Z2* ) p(Z1 , Z2 )p(Z1* |Z2 ) = p(Z1* |Z2* )p(Z2* )p(Z1 |Z2* ) p(Z1 |Z2 )p(Z2 )p(Z1* |Z2 ) = 1 Z2

ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ ໨ඪ෼෍ʢط஌ʣ ɹɹɹɹ ͱ͢Δɽ p(z) = (z|μ,
Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} z = ( z1 z2 ), μ = ( μ1 μ2 ), Σ = ( Σ11 Σ12 Σ21 Σ22 ), Λ = Σ−1 = ( Λ11 Λ12 Λ21 Λ22 ) log p(z) = log p(z1 , z2 ) = − 1 2 (z1 − μ1 )TΛ11 (z1 − μ1 ) + (z1 − μ1 )TΛ12 (z2 − μ2 )) − 1 2 (z2 − μ2 )TΛ22 (z2 − μ2 ) + (z2 − μ2 )TΛ21 (z1 − μ1 )) = − 1 2 ( zT 1 Λ11 z1 − 2z1 {Λ11 μ1 − 1 2 Λ12 (z2 − μ2 )}) + C1 − 1 2 ( zT 2 Λ22 z2 − 2z2 {Λ22 μ2 − 1 2 Λ21 (z1 − μ1 )}) + C2

ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ Αͬͯɼ৚݅෇͖֬཰ͷର਺͸ɼ ͱͳΔͷͰɼ৚݅෇͖֬཰͸Ψ΢ε෼෍Ͱ͋Δɼ log p(zi |zj
) = − 1 2 ( zT i Λii zi − 2zi { Λii μi − 1 2 Λij (zj − μj )}) + C p(zi |zj ) = (zi |μi , Σi ), Σ−1 i = Λii , Σ−1 i μi = Λii μi − 1 2 Λij (zj − μj ), ⇔ μi = Σi ( Λii μi − 1 2 Λij (zj − μj )) . https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0

ベイズ深層学習(4.1)

ベイズ深層学習(4.1)

More Decks by catla

Other Decks in Science

Featured

Transcript