Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ベイズ深層学習(4.1)
Search
catla
February 07, 2020
Science
0
440
ベイズ深層学習(4.1)
catla
February 07, 2020
Tweet
Share
More Decks by catla
See All by catla
ベイズ深層学習(6.3)
catla
2
220
ベイズ深層学習(6.2)
catla
3
230
[読み会資料] Federated Learning for Vision-and-Language Grounding Problems
catla
0
280
ベイズ深層学習(5.1~5.2)
catla
0
220
ベイズ深層学習(3.3~3.4)
catla
18
11k
ベイズ深層学習(2.2~2.4)
catla
6
1.3k
23回アルゴリズムコンテスト 1位解法
catla
6
670
Learning Lightweight Lane Detection CNNs by Self Attention Distillation(ICCV2019)の紹介
catla
0
560
TGS Salt Identification Challenge 12th place solution
catla
3
11k
Other Decks in Science
See All in Science
機械学習 - DBSCAN
trycycle
PRO
0
890
Symfony Console Facelift
chalasr
2
450
オンプレミス環境にKubernetesを構築する
koukimiura
0
260
局所保存性・相似変換対称性を満たす機械学習モデルによる数値流体力学
yellowshippo
1
270
動的トリートメント・レジームを推定するDynTxRegimeパッケージ
saltcooky12
0
140
証明支援系LEANに入門しよう
unaoya
0
1.1k
A Guide to Academic Writing Using Generative AI - A Workshop
ks91
PRO
0
110
SpatialBiologyWestCoastUS2024
lcolladotor
0
140
MoveItを使った産業用ロボット向け動作作成方法の紹介 / Introduction to creating motion for industrial robots using MoveIt
ry0_ka
0
490
点群ライブラリPDALをGoogleColabにて実行する方法の紹介
kentaitakura
1
290
Transport information Geometry: Current and Future II
lwc2017
0
150
学術講演会中央大学学員会府中支部
tagtag
0
270
Featured
See All Featured
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
Embracing the Ebb and Flow
colly
86
4.7k
Site-Speed That Sticks
csswizardry
10
660
Unsuck your backbone
ammeep
671
58k
Writing Fast Ruby
sferik
628
61k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Gamification - CAS2011
davidbonilla
81
5.3k
Designing for humans not robots
tammielis
253
25k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
It's Worth the Effort
3n
185
28k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Transcript
ϕΠζਂֶश αϯϓϦϯάʹجͮ͘ਪख๏ ܡɹঘً
αϯϓϦϯάʹجͮ͘ਪख๏ ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষࢀߟ
αϯϓϦϯάʹجͮ͘ਪख๏ ɹ؍ଌσʔλΛ ɼඇ؍ଌͷมͷू߹ʢFHύϥϝʔλɼજࡏม Λ ͱͨ͠ͱ ͖ɼϕΠζਪʹΑΔ౷ܭղੳͰɼ֬Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ ɹ࣮ࡍʹɼ֬ϞσϧΛ༻ֶ͍ͯश༧ଌɼࣄޙ Λܭࢉͯ͠ߦΘΕΔɽ X
Z p(X, Z) p(Z|X) ղܾࡦ ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕ ଟ͍ɽ p(Z|X) ɹ ΛղੳతʹٻΊΔΘΓʹɼ͜ͷ͔ΒෳͷαϯϓϧΛಘΔ͜ͱͰɼ ͷಛੑΛௐΔɽ ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճษڧ͢ΔΑʂ p(Z|X)
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏαϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
.$.$
୯७ϞϯςΧϧϩ๏ ɹ ʹର͢Δؔ ͷظɹ ΛٻΊ͍ͨɽɹ p(z) f(z) p(z) [ f(z)]
= ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒͷαϯϓϦϯά༰қɽ ∫ f(z)p(z)dz p(z) ख๏ ɹ Λेେ͖ͳͱͨ͠ͱ͖ɼ T z(1), z(2), …, z(T) ∼ p(z) ∫ f(z)p(z)dz ≈ 1 T T ∑ t=1 f(z(t)) ͔Β ݸαϯϓϦϯά ⟵ p(z) T
ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล Λܭࢉ͢Δࡍʹ༻͢ Δ߹ɼ θ p(X, θ) =
p(X|θ)p(θ) p(X) p(X) = ∫ p(X|θ)p(θ)dθ = ∫ N ∏ n=1 p(xn |θ)p(θ)dθ = p(θ) [p(X|θ)] ≈ 1 T T ∑ t=1 N ∏ n=1 p(xn |θ(t)), (θ(1), …, θ(T) ∼ p(θ)) ɹظ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ෯͘ͱΔඞཁ͕͋ΓɼҰํ Ͱɼ ڱ͍ ͷൣғͰ͔͠େ͖ͳΛऔΒͳ͍έʔε͕ଟ͍ɽ ɹ ൚༻త͚ͩͲɼܭࢉޮ͕ѱ͍ɽ p(z) [ f(z)] p(z) z f(z) z ⟹ ୯७ϞϯςΧϧϩ๏
غ٫αϯϓϦϯά ɹີܭࢉ͕ࠔͳ֬ ͔ΒαϯϓϧΛಘΔɽɹ p(z) z(1), z(2), … ∼ p(z) త
ঢ়گ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽͭ·Γɼ ɽ ˜ p(z)( = Zp ⋅ p(z)) ∫ ˜ p(z)dz ≠ 1 ख๏ ɹఏҊ Λઃఆ͢Δɽҙͷ ʹରͯ͠ɼ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ͱͳΔΑ͏ʹɼਖ਼ͷఆ ΛఆΊΔɽ q(z) z kq(z) > ˜ p(z) k ఏҊ ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ Ծͷɽ
غ٫αϯϓϦϯά ख๏ ͖ͭͮʣ ɹఏҊ ͔ΒαϯϓϧΛಘΔɽ ɹҰ༷ ͔ΒͷαϯϓϧΛಘΔɽ
ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ ɹɹɹ q(z) z(t) ∼ q(z) Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t))) z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ड༰ ∫ q(z) ˜ p(z) kq(z) dz = 1 k ∫ ˜ p(z)dz ߴ࣍ݩͷมͷαϯϓϦϯά͕ඞཁͳ߹ɼड༰͕ඇৗʹ͘ͳΔɽ
غ٫αϯϓϦϯά z ˜ p(z) ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ ະɽ طɽ p(z) p(z) ˜
p(z) p(z)
غ٫αϯϓϦϯά z ˜ p(z) αϯϓϧ͕༰қͳఏҊ Λઃఆɽ p(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ ˜ p(z) k
kq(z) > ˜ p(z) q(z) × k
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) ఏҊ ͔ΒαϯϓϧΛಘΔɽ q(z) z(t)
∼ q(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ˜ u Ұ༷
͔ΒͷαϯϓϧΛಘΔ Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t)))
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ड༰ غ٫ ˜
u αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ˜ p(z(t))
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ʹର͢Δؔ ͷظɹ Λ୯७ϞϯςΧϧϩ๏Α ΓޮతʹٻΊ͍ͨɽɹ p(z) f(z) p(z) [
f(z)] = ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒαϯϓϦϯάΛಘΒΕͳ͍ɽ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽ ∫ f(z)p(z)dz p(z) ˜ p(z)( = Zp ⋅ p(z)) എܠ ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ ΛΘͣʹαϯϓϧΛऔಘ͠ɼظ Λٻ ΊΔ͜ͱͰ͖Δ͕ɼ ͷ͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ ୯७ϞϯςΧϧϩ๏ͷܭࢉͷد༩͕গͳ͍ɽ ͷ͕େ͖͘ͳΔΑ͏ͳ ྖҬΛॏతʹαϯϓϧͨ͠ํ͕ޮ͕͍͍ɽ p(z) p(z) [ f(z)] f(z) ⟹ f(z)p(z)
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ·ͣɼఏҊ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ ɼҎԼͷΑ͏ʹද ͞ΕΔɽ ɹظͷܭࢉɼҎԼͷΑ͏ʹมܗͰ͖Δɽ q(z)
˜ p(z), ˜ q(z) p(z) = 1 Zp ˜ p(z), q(z) = 1 Zq ˜ q(z) ∫ f(z)p(z)dz = ∫ f(z) p(z) q(z) q(z)dz = q(z) [ f(z) p(z) q(z) ] = ∫ f(z) 1 Zp ˜ p(z) 1 Zq ˜ q(z) q(z)dz = Zq Zp q(z) [ f(z) ˜ p(z) ˜ q(z) ] ≈ Zq Zp 1 T T ∑ t=1 f(z(t)) ˜ p(z(t)) ˜ q(z(t)) = Zq Zp 1 T T ∑ t=1 f(z(t))w(t), w(t) = ˜ p(z(t)) ˜ q(z(t)) ख๏
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 )
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 ) ͳΜͷͨΊʹ Λ ͖࣋ͬͯͨΜͩΖ͏ʜ ˜ q(z)
ࣗݾਖ਼نԽॏαϯϓϦϯάʢޡهͷՄೳੑʣ ʮϕΠζਂֶशʯQ ɹࣗݾਖ਼نԽॏαϯϓϦϯάͷஈམͷ࠷ॳ ޡΓ ɹغ٫αϯϓϦϯάͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z) ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹغ٫αϯϓϦϯάͷɼߴ࣍ݩʹͳΔͱड༰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ ʹɼ࣍ݩఔͷ؆୯ͳੵۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ ɹͰɼߴ࣍ݩۭؒͰޮతʹαϯϓϦϯά͢Δʹʜʜ ɹɹɹ Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ ⟹ ࣍Ϛϧίϑ࿈ ɹ֬มͷܥྻ
ʹରͯ͠ ͕Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ z(1), z(2), … p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1)) z(1), z(2), … άϥϑΟΧϧϞσϧ z(1) z(2) z(t−1) z(t) ⋯ ɹભҠ֬ɹΛ ͱ͓͍ͨͱ͖ɼ ͕Γཱͭͱ͖ɼ Λɹఆৗɹͱ͍͏ɽ (z(t−1), z(t)) = p(z(t) |z(t−1)) p* (z) = ∫ (z′ , z)p* (z′ )dz′ p* (z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹఆৗ ʹऩଋ͢ΔΑ͏ͳભҠ֬ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗʹີܭࢉ͕ࠔͳ֬Λ͓͘ɽ p* (z) (z(t−1),
z(t)) p* (z) ⟹ ख๏ͷΩϞ ৄࡉΓ߹͍݅ p* (z)(z, z′ ) = p* (z′ )(z′ , z) ʲे݅ʳৄࡉΓ߹͍͕݅Γཱͭ ఆৗͱͳΔɽ ⟹ p* (z) p* (z)(z, z′ ) = p* (z′ )(z′ , z) ⇒ ∫ p* (z)(z, z′ )dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) ∫ p(z′ |z)dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) = ∫ p* (z′ )(z′ , z)dz′
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹΓ߹͍݅ʹՃ͑ͯɼαϯϓϧ͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ ͣɼ ͕ఆৗ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ t →
∞ p(z(1)) p(z(t)) p* (z) ⟹ Τϧΰʔυੑ w طੑɹɿҙͷঢ়ଶ͔Βҙͷঢ়ଶ༗ݶճͰભҠՄೳɽ w ඇपظੑɿͯ͢ͷঢ়ଶ͕ݻఆͷपظੑΛͨͳ͍ɽ w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰΔ͜ͱ͕Մೳɽ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ p(z)
∝ ˜ p(z) ˜ p(z) ख๏ ɹભҠ֬ ͕ઃܭ͕͍͠߹ɼભҠͷఏҊ Λ͑Δɽ (z′ , z) q(z|z′ ) ɽఏҊ ͔Β࣍ͷαϯϓϧͷީิ ΛαϯϓϦϯά͢Δɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ q( ⋅ |z(t)) z* r r = ˜ p(z* )q(z(t) |z* ) ˜ p(z(t))q(z* |z(t)) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ৄࡉΓ߹͍݅ͷূ໌ ɹભҠ֬ɼҎԼͷΑ͏ʹͳΔɽ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) p(z)(z, z′ ) = p(z)q(z′ |z) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) = p(z)q(z′ |z) min (1, p(z′ )q(z|z′ ) p(z)q(z′ |z) ) = min (p(z)q(z′ |z), p(z′ )q(z|z′ )) = min (p(z′ )q(z|z′ ), p(z)q(z′ |z)) = p(z′ )q(z|z′ ) min (1, p(z)q(z′ |z) p(z′ )q(z|z′ )) = p(z′ )q(z|z′ ) min (1, ˜ p(z)q(z′ |z) ˜ p(z′ )q(z|z′ )) = p(z′ )(z′ , z) ɹ ͷ߹ɼϝτϩϙϦε๏ͱݺΕΔɽ q(z′ |z) = q(z|z′ )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ʮϕΠζਂֶशʯQࣜʢʣ ޡΓ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) (z, z′ ) = q(z|z′ ) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ۩ମྫͰֶͿ ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ఏҊ
ͭ·Γ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} q(z* |z) = (z′ |z, I) z* ∼ (z, I) https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3
ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳγϛϡϨʔγϣϯΛղ આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯҎԼͷΑ͏ʹ ද͞ΕΔɽ ℋ(z, p) = (z) + (p), (p)
= 1 2m pTp, z ∈ ℝD: ମͷҐஔϕΫτϧ, p ∈ ℝD: ମͷӡಈྔϕΫτϧ, m ∈ ℝ: ମͷ࣭ྔ, ℋ(z, p): ϋϛϧτχΞϯ, (z): ϙςϯγϟϧΤωϧΪʔ, (p): ӡಈΤωϧΪʔ . ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹମͷيಓͷझຯϨʔγϣϯ .)๏ × ɹ).$๏ɼϥϯμϜΥʔΫతͳ.)ͱൺͯɼޮతʹۭؒΛ୳ࡧՄೳɽ ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢ΔڍಈɼϋϛϧτχΞϯͷภඍʹΑܾͬͯఆɽ ͜ͷඍํఔ͕ࣜղੳతʹղ͚ͳ͍ͷͱ͠ɼγϛϡϨʔγϣϯʹΑͬͯيಓΛܭ ࢉ͢Δɽ z
p τ dpi dτ = − dℋ dzi = − d dzi , dzi dτ = dℋ dpi = d dpi . ΦΠϥʔ๏ ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ ϵ > 0 pi (τ + ϵ) = pi (τ) + ϵ dpi dτ τ = pi (τ) − ϵ d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵ dzi dτ τ = zi (τ) + ϵpi (τ) ࢄԽʹΑΔޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏ ⟹
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοά๏ pi (τ + ϵ 2 ) = pi
(τ) − ϵ 2 d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵpi (τ + ϵ 2 ), pi (τ + ϵ) = pi (τ + ϵ 2 ) − ϵ 2 d dzi zi (τ + ϵ) . ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ L ϵL z* p* ϋϛϧτχΞϯͷੑ࣭ ɽ ࣌ؒ ʹΑͬͯෆมɽ ɽՄٯੑɿ ͔Β ͷભҠҰରҰɽ ɽମੵอଘ ℋ τ (z, p) (z* , p* )
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ αϯϓϦϯάΞϧΰϦζϜͷద༻ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ
ɹ ͱ֦ு͢Δͱɼ पล ͔Βαϯϓϧ͕ಘΒΕΔɽ ɹ p(z) ∝ ˜ p(z) ˜ p(z) p(z, p) = p(z)p(p) z p(z) p(p) = (p|0, I) (z) = − log (˜ p(z)) (p) = 1 2 pTp
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹಉ࣌Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ ϝτϩϙϦε๏ͰΘΕΔൺ ɼҎԼͷΑ͏ʹͳΔɽ p(z, p) = p(z)p(p) =
exp (log p(z) + log p(p)) ∝ exp (log ˜ p(z) − 1 2 pTIp ) = exp (−(z) − (p)) = exp (−ℋ(z, p)) r r = p(z* , p* ) p(z, p) = exp (−ℋ(z* , p* ) + ℋ(z, p))
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ख๏ ɽӡಈྔΛαϯϓϦϯά ɽϦʔϓϑϩοά๏Ͱݱࡏͷ ͔Βީิ ΛಘΔɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ
ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ p ∼ (0, I) (z(t), p) (z* , p* ) r r = p(z* , p* ) p(z, p) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ ɹ).$๏ɼࣄޙͷඍ͑͞ܭࢉͰ͖Εద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ χϡʔϥϧωοτϫʔΫ࿈ଓͳજࡏมͷΈͰΓཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$ ๏χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹΘΕ͖ͯͨɽ େ͖͍ εςοϓαΠζ ϵ εςοϓ L
খ͍͞ খ͍͞ େ͖͍ ड༰ ड༰ ୳ࡧޮ ܭࢉྔ ߴ͍ ߴ͍ ͍ ͍ େ͖͍ খ͍͞ ߴ͍ ͍
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ϥϯδϡόϯಈྗֶ๏ ɹ ͱͨ͠߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨɹϥϯδϡόϯಈྗֶ๏ɹͱ ݺΕΔɽ ɹਂֶश͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬తޯϥϯδϡόϯಈྗֶ ๏ɹʹల։͞ΕΔɽ L =
1 z*i = zi (τ + ϵ) = zi (τ) + ϵ pi (τ) − ϵ 2 d dzi zi (τ) = zi (τ) − ϵ2 2 d dzi zi (τ) + ϵpi (τ)
ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ӡಈྔͷαϯϓϦϯάɿ ӡಈΤωϧΪʔɿ
ҐஔΤωϧΪʔɿ ҐஔΤωϧΪʔͷภඍɿ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} p ∼ (0, I) (p) = 1 2 pTp (z) = − log (˜ p(z)) = − 1 2 (z − μ)TΣ−1(z − μ) ∂ ∂z = − (z − μ)TΣ−1 ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ۩ମྫͰֶͿ https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ
ΪϒεαϯϓϦϯά త ɹ֬ ͔Β શମΛαϯϓϦϯά͢Δ͜ͱ͕͍͠ͱ͖ͷ୳ࡧɽ p(Z) Z લఏ ɹ֬ طɽ
p(Z) ख๏ ɹɽม Λ ݸͷ෦ू߹ʹ͚Δɽ ɹɽ෦ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ ɹ Z M Z = {Z1 , Z2 , …, ZM } Z1 ∼ p(Z1 |Z2 , Z3 , …, ZM−1 , ZM ) Z2 ∼ p(Z2 |Z1 , Z3 , …, ZM−1 , ZM ) ⋮ ZM ∼ p(ZM |Z1 , Z2 , …, ZM−2 , ZM−1 )
ΪϒεαϯϓϦϯά ɹΪϒεαϯϓϦϯάͷଥੑɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ͚ɼ Λ͚݅ͨ͠ͱͰ ͷα ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ߹ɼ ͔ͭ ɹൺ
Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ Αͬͯɼશͯड༰͞ΕΔɽ ಉ༷ɽ Z = {Z1 , Z2 } Z2 Z1 q(Z* |Z) = p(Z1* |Z2* ) Z2 = Z2* r r = p(Z* )q(Z|Z* ) p(Z)q(Z* |Z) = p(Z1* , Z2* )p(Z1 |Z2* ) p(Z1 , Z2 )p(Z1* |Z2 ) = p(Z1* |Z2* )p(Z2* )p(Z1 |Z2* ) p(Z1 |Z2 )p(Z2 )p(Z1* |Z2 ) = 1 Z2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ ඪʢطʣ ɹɹɹɹ ͱ͢Δɽ p(z) = (z|μ,
Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} z = ( z1 z2 ), μ = ( μ1 μ2 ), Σ = ( Σ11 Σ12 Σ21 Σ22 ), Λ = Σ−1 = ( Λ11 Λ12 Λ21 Λ22 ) log p(z) = log p(z1 , z2 ) = − 1 2 (z1 − μ1 )TΛ11 (z1 − μ1 ) + (z1 − μ1 )TΛ12 (z2 − μ2 )) − 1 2 (z2 − μ2 )TΛ22 (z2 − μ2 ) + (z2 − μ2 )TΛ21 (z1 − μ1 )) = − 1 2 ( zT 1 Λ11 z1 − 2z1 {Λ11 μ1 − 1 2 Λ12 (z2 − μ2 )}) + C1 − 1 2 ( zT 2 Λ22 z2 − 2z2 {Λ22 μ2 − 1 2 Λ21 (z1 − μ1 )}) + C2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ Αͬͯɼ͖݅֬ͷରɼ ͱͳΔͷͰɼ͖݅֬ΨεͰ͋Δɼ log p(zi |zj
) = − 1 2 ( zT i Λii zi − 2zi { Λii μi − 1 2 Λij (zj − μj )}) + C p(zi |zj ) = (zi |μi , Σi ), Σ−1 i = Λii , Σ−1 i μi = Λii μi − 1 2 Λij (zj − μj ), ⇔ μi = Σi ( Λii μi − 1 2 Λij (zj − μj )) . https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0