Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ベイズ深層学習(4.1)
Search
catla
February 07, 2020
Science
0
360
ベイズ深層学習(4.1)
catla
February 07, 2020
Tweet
Share
More Decks by catla
See All by catla
ベイズ深層学習(6.3)
catla
2
190
ベイズ深層学習(6.2)
catla
3
200
[読み会資料] Federated Learning for Vision-and-Language Grounding Problems
catla
0
190
ベイズ深層学習(5.1~5.2)
catla
0
210
ベイズ深層学習(3.3~3.4)
catla
18
10k
ベイズ深層学習(2.2~2.4)
catla
6
1.2k
23回アルゴリズムコンテスト 1位解法
catla
6
640
Learning Lightweight Lane Detection CNNs by Self Attention Distillation(ICCV2019)の紹介
catla
0
490
TGS Salt Identification Challenge 12th place solution
catla
3
11k
Other Decks in Science
See All in Science
勉強会資料 / “Asymptotic Statistics” Section 3.1
asymptotic_minato
0
110
Design of three-dimensional binary manipulators based on the KS statistic and maximum empty circles (IECON2023)
konakalab
0
230
B-Cubed: Leveraging analysis-ready biodiversity datasets and cloud computing for timely and actionable biodiversity monitoring
peterdesmet
0
160
ベクトル型スーパーコンピュータ「AOBA-S」の性能評価
keichi
0
250
Yasuke
drawsbygba
0
610
Non-Gaussian methods for causal discovery
sshimizu2006
0
170
20240420 Global Azure 2024 | Azure Migrate でデータセンターのサーバーを評価&移行してみる
olivia_0707
2
440
早わかり W3C Community Group
takanorip
0
260
LCG20
lcolladotor
0
200
救急外来でのめまい診療_中枢性めまいを見逃さない!
psasa
0
150
JSol'Ex : solar image processing in Java
melix
0
240
脳とAIは似ているか ― NeuroAI の挑戦
ykamit
9
6.8k
Featured
See All Featured
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
321
20k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
658
120k
The World Runs on Bad Software
bkeepers
PRO
61
6.7k
Making Projects Easy
brettharned
108
5.5k
How to name files
jennybc
64
93k
Documentation Writing (for coders)
carmenintech
59
3.9k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
273
13k
What’s in a name? Adding method to the madness
productmarketing
PRO
15
2.6k
Become a Pro
speakerdeck
PRO
10
4.5k
Designing Experiences People Love
moore
136
23k
Making the Leap to Tech Lead
cromwellryan
123
8.5k
Bootstrapping a Software Product
garrettdimon
PRO
301
110k
Transcript
ϕΠζਂֶश αϯϓϦϯάʹجͮ͘ਪख๏ ܡɹঘً
αϯϓϦϯάʹجͮ͘ਪख๏ ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষࢀߟ
αϯϓϦϯάʹجͮ͘ਪख๏ ɹ؍ଌσʔλΛ ɼඇ؍ଌͷมͷू߹ʢFHύϥϝʔλɼજࡏม Λ ͱͨ͠ͱ ͖ɼϕΠζਪʹΑΔ౷ܭղੳͰɼ֬Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ ɹ࣮ࡍʹɼ֬ϞσϧΛ༻ֶ͍ͯश༧ଌɼࣄޙ Λܭࢉͯ͠ߦΘΕΔɽ X
Z p(X, Z) p(Z|X) ղܾࡦ ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕ ଟ͍ɽ p(Z|X) ɹ ΛղੳతʹٻΊΔΘΓʹɼ͜ͷ͔ΒෳͷαϯϓϧΛಘΔ͜ͱͰɼ ͷಛੑΛௐΔɽ ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճษڧ͢ΔΑʂ p(Z|X)
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏαϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
.$.$
୯७ϞϯςΧϧϩ๏ ɹ ʹର͢Δؔ ͷظɹ ΛٻΊ͍ͨɽɹ p(z) f(z) p(z) [ f(z)]
= ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒͷαϯϓϦϯά༰қɽ ∫ f(z)p(z)dz p(z) ख๏ ɹ Λेେ͖ͳͱͨ͠ͱ͖ɼ T z(1), z(2), …, z(T) ∼ p(z) ∫ f(z)p(z)dz ≈ 1 T T ∑ t=1 f(z(t)) ͔Β ݸαϯϓϦϯά ⟵ p(z) T
ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล Λܭࢉ͢Δࡍʹ༻͢ Δ߹ɼ θ p(X, θ) =
p(X|θ)p(θ) p(X) p(X) = ∫ p(X|θ)p(θ)dθ = ∫ N ∏ n=1 p(xn |θ)p(θ)dθ = p(θ) [p(X|θ)] ≈ 1 T T ∑ t=1 N ∏ n=1 p(xn |θ(t)), (θ(1), …, θ(T) ∼ p(θ)) ɹظ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ෯͘ͱΔඞཁ͕͋ΓɼҰํ Ͱɼ ڱ͍ ͷൣғͰ͔͠େ͖ͳΛऔΒͳ͍έʔε͕ଟ͍ɽ ɹ ൚༻త͚ͩͲɼܭࢉޮ͕ѱ͍ɽ p(z) [ f(z)] p(z) z f(z) z ⟹ ୯७ϞϯςΧϧϩ๏
غ٫αϯϓϦϯά ɹີܭࢉ͕ࠔͳ֬ ͔ΒαϯϓϧΛಘΔɽɹ p(z) z(1), z(2), … ∼ p(z) త
ঢ়گ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽͭ·Γɼ ɽ ˜ p(z)( = Zp ⋅ p(z)) ∫ ˜ p(z)dz ≠ 1 ख๏ ɹఏҊ Λઃఆ͢Δɽҙͷ ʹରͯ͠ɼ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ͱͳΔΑ͏ʹɼਖ਼ͷఆ ΛఆΊΔɽ q(z) z kq(z) > ˜ p(z) k ఏҊ ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ Ծͷɽ
غ٫αϯϓϦϯά ख๏ ͖ͭͮʣ ɹఏҊ ͔ΒαϯϓϧΛಘΔɽ ɹҰ༷ ͔ΒͷαϯϓϧΛಘΔɽ
ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ ɹɹɹ q(z) z(t) ∼ q(z) Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t))) z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ड༰ ∫ q(z) ˜ p(z) kq(z) dz = 1 k ∫ ˜ p(z)dz ߴ࣍ݩͷมͷαϯϓϦϯά͕ඞཁͳ߹ɼड༰͕ඇৗʹ͘ͳΔɽ
غ٫αϯϓϦϯά z ˜ p(z) ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ ະɽ طɽ p(z) p(z) ˜
p(z) p(z)
غ٫αϯϓϦϯά z ˜ p(z) αϯϓϧ͕༰қͳఏҊ Λઃఆɽ p(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ ˜ p(z) k
kq(z) > ˜ p(z) q(z) × k
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) ఏҊ ͔ΒαϯϓϧΛಘΔɽ q(z) z(t)
∼ q(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ˜ u Ұ༷
͔ΒͷαϯϓϧΛಘΔ Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t)))
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ड༰ غ٫ ˜
u αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ˜ p(z(t))
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ʹର͢Δؔ ͷظɹ Λ୯७ϞϯςΧϧϩ๏Α ΓޮతʹٻΊ͍ͨɽɹ p(z) f(z) p(z) [
f(z)] = ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒαϯϓϦϯάΛಘΒΕͳ͍ɽ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽ ∫ f(z)p(z)dz p(z) ˜ p(z)( = Zp ⋅ p(z)) എܠ ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ ΛΘͣʹαϯϓϧΛऔಘ͠ɼظ Λٻ ΊΔ͜ͱͰ͖Δ͕ɼ ͷ͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ ୯७ϞϯςΧϧϩ๏ͷܭࢉͷد༩͕গͳ͍ɽ ͷ͕େ͖͘ͳΔΑ͏ͳ ྖҬΛॏతʹαϯϓϧͨ͠ํ͕ޮ͕͍͍ɽ p(z) p(z) [ f(z)] f(z) ⟹ f(z)p(z)
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ·ͣɼఏҊ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ ɼҎԼͷΑ͏ʹද ͞ΕΔɽ ɹظͷܭࢉɼҎԼͷΑ͏ʹมܗͰ͖Δɽ q(z)
˜ p(z), ˜ q(z) p(z) = 1 Zp ˜ p(z), q(z) = 1 Zq ˜ q(z) ∫ f(z)p(z)dz = ∫ f(z) p(z) q(z) q(z)dz = q(z) [ f(z) p(z) q(z) ] = ∫ f(z) 1 Zp ˜ p(z) 1 Zq ˜ q(z) q(z)dz = Zq Zp q(z) [ f(z) ˜ p(z) ˜ q(z) ] ≈ Zq Zp 1 T T ∑ t=1 f(z(t)) ˜ p(z(t)) ˜ q(z(t)) = Zq Zp 1 T T ∑ t=1 f(z(t))w(t), w(t) = ˜ p(z(t)) ˜ q(z(t)) ख๏
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 )
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 ) ͳΜͷͨΊʹ Λ ͖࣋ͬͯͨΜͩΖ͏ʜ ˜ q(z)
ࣗݾਖ਼نԽॏαϯϓϦϯάʢޡهͷՄೳੑʣ ʮϕΠζਂֶशʯQ ɹࣗݾਖ਼نԽॏαϯϓϦϯάͷஈམͷ࠷ॳ ޡΓ ɹغ٫αϯϓϦϯάͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z) ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹغ٫αϯϓϦϯάͷɼߴ࣍ݩʹͳΔͱड༰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ ʹɼ࣍ݩఔͷ؆୯ͳੵۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ ɹͰɼߴ࣍ݩۭؒͰޮతʹαϯϓϦϯά͢Δʹʜʜ ɹɹɹ Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ ⟹ ࣍Ϛϧίϑ࿈ ɹ֬มͷܥྻ
ʹରͯ͠ ͕Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ z(1), z(2), … p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1)) z(1), z(2), … άϥϑΟΧϧϞσϧ z(1) z(2) z(t−1) z(t) ⋯ ɹભҠ֬ɹΛ ͱ͓͍ͨͱ͖ɼ ͕Γཱͭͱ͖ɼ Λɹఆৗɹͱ͍͏ɽ (z(t−1), z(t)) = p(z(t) |z(t−1)) p* (z) = ∫ (z′ , z)p* (z′ )dz′ p* (z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹఆৗ ʹऩଋ͢ΔΑ͏ͳભҠ֬ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗʹີܭࢉ͕ࠔͳ֬Λ͓͘ɽ p* (z) (z(t−1),
z(t)) p* (z) ⟹ ख๏ͷΩϞ ৄࡉΓ߹͍݅ p* (z)(z, z′ ) = p* (z′ )(z′ , z) ʲे݅ʳৄࡉΓ߹͍͕݅Γཱͭ ఆৗͱͳΔɽ ⟹ p* (z) p* (z)(z, z′ ) = p* (z′ )(z′ , z) ⇒ ∫ p* (z)(z, z′ )dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) ∫ p(z′ |z)dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) = ∫ p* (z′ )(z′ , z)dz′
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹΓ߹͍݅ʹՃ͑ͯɼαϯϓϧ͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ ͣɼ ͕ఆৗ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ t →
∞ p(z(1)) p(z(t)) p* (z) ⟹ Τϧΰʔυੑ w طੑɹɿҙͷঢ়ଶ͔Βҙͷঢ়ଶ༗ݶճͰભҠՄೳɽ w ඇपظੑɿͯ͢ͷঢ়ଶ͕ݻఆͷपظੑΛͨͳ͍ɽ w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰΔ͜ͱ͕Մೳɽ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ p(z)
∝ ˜ p(z) ˜ p(z) ख๏ ɹભҠ֬ ͕ઃܭ͕͍͠߹ɼભҠͷఏҊ Λ͑Δɽ (z′ , z) q(z|z′ ) ɽఏҊ ͔Β࣍ͷαϯϓϧͷީิ ΛαϯϓϦϯά͢Δɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ q( ⋅ |z(t)) z* r r = ˜ p(z* )q(z(t) |z* ) ˜ p(z(t))q(z* |z(t)) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ৄࡉΓ߹͍݅ͷূ໌ ɹભҠ֬ɼҎԼͷΑ͏ʹͳΔɽ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) p(z)(z, z′ ) = p(z)q(z′ |z) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) = p(z)q(z′ |z) min (1, p(z′ )q(z|z′ ) p(z)q(z′ |z) ) = min (p(z)q(z′ |z), p(z′ )q(z|z′ )) = min (p(z′ )q(z|z′ ), p(z)q(z′ |z)) = p(z′ )q(z|z′ ) min (1, p(z)q(z′ |z) p(z′ )q(z|z′ )) = p(z′ )q(z|z′ ) min (1, ˜ p(z)q(z′ |z) ˜ p(z′ )q(z|z′ )) = p(z′ )(z′ , z) ɹ ͷ߹ɼϝτϩϙϦε๏ͱݺΕΔɽ q(z′ |z) = q(z|z′ )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ʮϕΠζਂֶशʯQࣜʢʣ ޡΓ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) (z, z′ ) = q(z|z′ ) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ۩ମྫͰֶͿ ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ఏҊ
ͭ·Γ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} q(z* |z) = (z′ |z, I) z* ∼ (z, I) https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3
ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳγϛϡϨʔγϣϯΛղ આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯҎԼͷΑ͏ʹ ද͞ΕΔɽ ℋ(z, p) = (z) + (p), (p)
= 1 2m pTp, z ∈ ℝD: ମͷҐஔϕΫτϧ, p ∈ ℝD: ମͷӡಈྔϕΫτϧ, m ∈ ℝ: ମͷ࣭ྔ, ℋ(z, p): ϋϛϧτχΞϯ, (z): ϙςϯγϟϧΤωϧΪʔ, (p): ӡಈΤωϧΪʔ . ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹମͷيಓͷझຯϨʔγϣϯ .)๏ × ɹ).$๏ɼϥϯμϜΥʔΫతͳ.)ͱൺͯɼޮతʹۭؒΛ୳ࡧՄೳɽ ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢ΔڍಈɼϋϛϧτχΞϯͷภඍʹΑܾͬͯఆɽ ͜ͷඍํఔ͕ࣜղੳతʹղ͚ͳ͍ͷͱ͠ɼγϛϡϨʔγϣϯʹΑͬͯيಓΛܭ ࢉ͢Δɽ z
p τ dpi dτ = − dℋ dzi = − d dzi , dzi dτ = dℋ dpi = d dpi . ΦΠϥʔ๏ ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ ϵ > 0 pi (τ + ϵ) = pi (τ) + ϵ dpi dτ τ = pi (τ) − ϵ d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵ dzi dτ τ = zi (τ) + ϵpi (τ) ࢄԽʹΑΔޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏ ⟹
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοά๏ pi (τ + ϵ 2 ) = pi
(τ) − ϵ 2 d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵpi (τ + ϵ 2 ), pi (τ + ϵ) = pi (τ + ϵ 2 ) − ϵ 2 d dzi zi (τ + ϵ) . ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ L ϵL z* p* ϋϛϧτχΞϯͷੑ࣭ ɽ ࣌ؒ ʹΑͬͯෆมɽ ɽՄٯੑɿ ͔Β ͷભҠҰରҰɽ ɽମੵอଘ ℋ τ (z, p) (z* , p* )
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ αϯϓϦϯάΞϧΰϦζϜͷద༻ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ
ɹ ͱ֦ு͢Δͱɼ पล ͔Βαϯϓϧ͕ಘΒΕΔɽ ɹ p(z) ∝ ˜ p(z) ˜ p(z) p(z, p) = p(z)p(p) z p(z) p(p) = (p|0, I) (z) = − log (˜ p(z)) (p) = 1 2 pTp
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹಉ࣌Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ ϝτϩϙϦε๏ͰΘΕΔൺ ɼҎԼͷΑ͏ʹͳΔɽ p(z, p) = p(z)p(p) =
exp (log p(z) + log p(p)) ∝ exp (log ˜ p(z) − 1 2 pTIp ) = exp (−(z) − (p)) = exp (−ℋ(z, p)) r r = p(z* , p* ) p(z, p) = exp (−ℋ(z* , p* ) + ℋ(z, p))
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ख๏ ɽӡಈྔΛαϯϓϦϯά ɽϦʔϓϑϩοά๏Ͱݱࡏͷ ͔Βީิ ΛಘΔɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ
ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ p ∼ (0, I) (z(t), p) (z* , p* ) r r = p(z* , p* ) p(z, p) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ ɹ).$๏ɼࣄޙͷඍ͑͞ܭࢉͰ͖Εద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ χϡʔϥϧωοτϫʔΫ࿈ଓͳજࡏมͷΈͰΓཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$ ๏χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹΘΕ͖ͯͨɽ େ͖͍ εςοϓαΠζ ϵ εςοϓ L
খ͍͞ খ͍͞ େ͖͍ ड༰ ड༰ ୳ࡧޮ ܭࢉྔ ߴ͍ ߴ͍ ͍ ͍ େ͖͍ খ͍͞ ߴ͍ ͍
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ϥϯδϡόϯಈྗֶ๏ ɹ ͱͨ͠߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨɹϥϯδϡόϯಈྗֶ๏ɹͱ ݺΕΔɽ ɹਂֶश͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬తޯϥϯδϡόϯಈྗֶ ๏ɹʹల։͞ΕΔɽ L =
1 z*i = zi (τ + ϵ) = zi (τ) + ϵ pi (τ) − ϵ 2 d dzi zi (τ) = zi (τ) − ϵ2 2 d dzi zi (τ) + ϵpi (τ)
ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ӡಈྔͷαϯϓϦϯάɿ ӡಈΤωϧΪʔɿ
ҐஔΤωϧΪʔɿ ҐஔΤωϧΪʔͷภඍɿ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} p ∼ (0, I) (p) = 1 2 pTp (z) = − log (˜ p(z)) = − 1 2 (z − μ)TΣ−1(z − μ) ∂ ∂z = − (z − μ)TΣ−1 ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ۩ମྫͰֶͿ https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ
ΪϒεαϯϓϦϯά త ɹ֬ ͔Β શମΛαϯϓϦϯά͢Δ͜ͱ͕͍͠ͱ͖ͷ୳ࡧɽ p(Z) Z લఏ ɹ֬ طɽ
p(Z) ख๏ ɹɽม Λ ݸͷ෦ू߹ʹ͚Δɽ ɹɽ෦ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ ɹ Z M Z = {Z1 , Z2 , …, ZM } Z1 ∼ p(Z1 |Z2 , Z3 , …, ZM−1 , ZM ) Z2 ∼ p(Z2 |Z1 , Z3 , …, ZM−1 , ZM ) ⋮ ZM ∼ p(ZM |Z1 , Z2 , …, ZM−2 , ZM−1 )
ΪϒεαϯϓϦϯά ɹΪϒεαϯϓϦϯάͷଥੑɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ͚ɼ Λ͚݅ͨ͠ͱͰ ͷα ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ߹ɼ ͔ͭ ɹൺ
Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ Αͬͯɼશͯड༰͞ΕΔɽ ಉ༷ɽ Z = {Z1 , Z2 } Z2 Z1 q(Z* |Z) = p(Z1* |Z2* ) Z2 = Z2* r r = p(Z* )q(Z|Z* ) p(Z)q(Z* |Z) = p(Z1* , Z2* )p(Z1 |Z2* ) p(Z1 , Z2 )p(Z1* |Z2 ) = p(Z1* |Z2* )p(Z2* )p(Z1 |Z2* ) p(Z1 |Z2 )p(Z2 )p(Z1* |Z2 ) = 1 Z2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ ඪʢطʣ ɹɹɹɹ ͱ͢Δɽ p(z) = (z|μ,
Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} z = ( z1 z2 ), μ = ( μ1 μ2 ), Σ = ( Σ11 Σ12 Σ21 Σ22 ), Λ = Σ−1 = ( Λ11 Λ12 Λ21 Λ22 ) log p(z) = log p(z1 , z2 ) = − 1 2 (z1 − μ1 )TΛ11 (z1 − μ1 ) + (z1 − μ1 )TΛ12 (z2 − μ2 )) − 1 2 (z2 − μ2 )TΛ22 (z2 − μ2 ) + (z2 − μ2 )TΛ21 (z1 − μ1 )) = − 1 2 ( zT 1 Λ11 z1 − 2z1 {Λ11 μ1 − 1 2 Λ12 (z2 − μ2 )}) + C1 − 1 2 ( zT 2 Λ22 z2 − 2z2 {Λ22 μ2 − 1 2 Λ21 (z1 − μ1 )}) + C2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ Αͬͯɼ͖݅֬ͷରɼ ͱͳΔͷͰɼ͖݅֬ΨεͰ͋Δɼ log p(zi |zj
) = − 1 2 ( zT i Λii zi − 2zi { Λii μi − 1 2 Λij (zj − μj )}) + C p(zi |zj ) = (zi |μi , Σi ), Σ−1 i = Λii , Σ−1 i μi = Λii μi − 1 2 Λij (zj − μj ), ⇔ μi = Σi ( Λii μi − 1 2 Λij (zj − μj )) . https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0