Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ベイズ深層学習(4.1)
Search
catla
February 07, 2020
Science
0
410
ベイズ深層学習(4.1)
catla
February 07, 2020
Tweet
Share
More Decks by catla
See All by catla
ベイズ深層学習(6.3)
catla
2
200
ベイズ深層学習(6.2)
catla
3
210
[読み会資料] Federated Learning for Vision-and-Language Grounding Problems
catla
0
230
ベイズ深層学習(5.1~5.2)
catla
0
210
ベイズ深層学習(3.3~3.4)
catla
18
11k
ベイズ深層学習(2.2~2.4)
catla
6
1.2k
23回アルゴリズムコンテスト 1位解法
catla
6
650
Learning Lightweight Lane Detection CNNs by Self Attention Distillation(ICCV2019)の紹介
catla
0
520
TGS Salt Identification Challenge 12th place solution
catla
3
11k
Other Decks in Science
See All in Science
早わかり W3C Community Group
takanorip
0
360
第4回ナレッジグラフ勉強会 Knowledge Graph Embedding
maruru0090
0
230
ICRA2024 速報
rpc
3
4.8k
LIMEを用いた判断根拠の可視化
kentaitakura
0
250
How were Quaternion discovered
kinakomoti321
2
910
Boil Order
uni_of_nomi
0
100
教師なしテンソル分解に基づく、有糸分裂後の転写再活性化におけるヒストン修飾ブックマークとしての転写因子候補の抽出法
tagtag
0
110
プロダクト開発を通して学んだナレッジマネジメントの哲学
sonod
0
130
qeMLパッケージの紹介
bob3bob3
0
1.3k
機械学習による確率推定とカリブレーション/probabilistic-calibration-on-classification-model
ktgrstsh
2
140
Xpenologyなるアングラプロジェクト周りについて語るやつ
sushi514
0
980
いまAI組織が求める企画開発エンジニアとは?
roadroller
2
1.2k
Featured
See All Featured
4 Signs Your Business is Dying
shpigford
179
21k
Imperfection Machines: The Place of Print at Facebook
scottboms
263
13k
Building Flexible Design Systems
yeseniaperezcruz
324
37k
The Power of CSS Pseudo Elements
geoffreycrofte
71
5.2k
It's Worth the Effort
3n
182
27k
Atom: Resistance is Futile
akmur
261
25k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
363
22k
A Tale of Four Properties
chriscoyier
155
22k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
24
2k
The Illustrated Children's Guide to Kubernetes
chrisshort
46
48k
Being A Developer After 40
akosma
82
580k
What’s in a name? Adding method to the madness
productmarketing
PRO
21
3k
Transcript
ϕΠζਂֶश αϯϓϦϯάʹجͮ͘ਪख๏ ܡɹঘً
αϯϓϦϯάʹجͮ͘ਪख๏ ˞ʮύλʔϯೝࣝͱػցֶशɹԼרʯষࢀߟ
αϯϓϦϯάʹجͮ͘ਪख๏ ɹ؍ଌσʔλΛ ɼඇ؍ଌͷมͷू߹ʢFHύϥϝʔλɼજࡏม Λ ͱͨ͠ͱ ͖ɼϕΠζਪʹΑΔ౷ܭղੳͰɼ֬Ϟσϧ Λઃܭ͢Δඞཁ͕͋Δɽ ɹ࣮ࡍʹɼ֬ϞσϧΛ༻ֶ͍ͯश༧ଌɼࣄޙ Λܭࢉͯ͠ߦΘΕΔɽ X
Z p(X, Z) p(Z|X) ղܾࡦ ɹෳࡶͳϞσϧʢFHχϡʔϥϧωοτʣɼ ͕ղੳతʹٻΊΒΕͳ͍͜ͱ͕ ଟ͍ɽ p(Z|X) ɹ ΛղੳతʹٻΊΔΘΓʹɼ͜ͷ͔ΒෳͷαϯϓϧΛಘΔ͜ͱͰɼ ͷಛੑΛௐΔɽ ɹͱ͍͏͜ͱ͔ΒɼαϯϓϦϯά͢Δํ๏Λࠓճษڧ͢ΔΑʂ p(Z|X)
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏαϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
ຊͷ༰ ୯७ϞϯςΧϧϩ๏ غ٫αϯϓϦϯά ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ʢϚϧίϑ࿈ϞϯςΧϧϩ๏ʣ ϝτϩϙϦεɾϔΠεςΟϯάε๏ ϝτϩϙϦε๏ ϋϛϧτχΞϯϞϯςΧϧϩ๏ ϥϯδϡόϯϞϯςΧϧϩ๏ ΪϒεαϯϓϦϯά
.$.$
୯७ϞϯςΧϧϩ๏ ɹ ʹର͢Δؔ ͷظɹ ΛٻΊ͍ͨɽɹ p(z) f(z) p(z) [ f(z)]
= ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒͷαϯϓϦϯά༰қɽ ∫ f(z)p(z)dz p(z) ख๏ ɹ Λेେ͖ͳͱͨ͠ͱ͖ɼ T z(1), z(2), …, z(T) ∼ p(z) ∫ f(z)p(z)dz ≈ 1 T T ∑ t=1 f(z(t)) ͔Β ݸαϯϓϦϯά ⟵ p(z) T
ɹύϥϝʔλ Λ࣋ͭϞσϧ ͷपล Λܭࢉ͢Δࡍʹ༻͢ Δ߹ɼ θ p(X, θ) =
p(X|θ)p(θ) p(X) p(X) = ∫ p(X|θ)p(θ)dθ = ∫ N ∏ n=1 p(xn |θ)p(θ)dθ = p(θ) [p(X|θ)] ≈ 1 T T ∑ t=1 N ∏ n=1 p(xn |θ(t)), (θ(1), …, θ(T) ∼ p(θ)) ɹظ ʹ͓͍ͯɼ ͔Βͷαϯϓϧ ͷൣғ෯͘ͱΔඞཁ͕͋ΓɼҰํ Ͱɼ ڱ͍ ͷൣғͰ͔͠େ͖ͳΛऔΒͳ͍έʔε͕ଟ͍ɽ ɹ ൚༻త͚ͩͲɼܭࢉޮ͕ѱ͍ɽ p(z) [ f(z)] p(z) z f(z) z ⟹ ୯७ϞϯςΧϧϩ๏
غ٫αϯϓϦϯά ɹີܭࢉ͕ࠔͳ֬ ͔ΒαϯϓϧΛಘΔɽɹ p(z) z(1), z(2), … ∼ p(z) త
ঢ়گ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽͭ·Γɼ ɽ ˜ p(z)( = Zp ⋅ p(z)) ∫ ˜ p(z)dz ≠ 1 ख๏ ɹఏҊ Λઃఆ͢Δɽҙͷ ʹରͯ͠ɼ ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ͱͳΔΑ͏ʹɼਖ਼ͷఆ ΛఆΊΔɽ q(z) z kq(z) > ˜ p(z) k ఏҊ ɹαϯϓϦϯά͕؆୯ʹߦ͑ΔΑ͏ͳ Ծͷɽ
غ٫αϯϓϦϯά ख๏ ͖ͭͮʣ ɹఏҊ ͔ΒαϯϓϧΛಘΔɽ ɹҰ༷ ͔ΒͷαϯϓϧΛಘΔɽ
ɹαϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ ɹɹɹ q(z) z(t) ∼ q(z) Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t))) z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ड༰ ∫ q(z) ˜ p(z) kq(z) dz = 1 k ∫ ˜ p(z)dz ߴ࣍ݩͷมͷαϯϓϦϯά͕ඞཁͳ߹ɼड༰͕ඇৗʹ͘ͳΔɽ
غ٫αϯϓϦϯά z ˜ p(z) ͷαϯϓϧΛغ٫αϯϓϧϦϯάͰ֫ಘ͢Δɽ ະɽ طɽ p(z) p(z) ˜
p(z) p(z)
غ٫αϯϓϦϯά z ˜ p(z) αϯϓϧ͕༰қͳఏҊ Λઃఆɽ p(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) Λ෴͍͔Ϳ͞ΔΑ͏ʹ Λઃఆɽ ˜ p(z) k
kq(z) > ˜ p(z) q(z) × k
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) ఏҊ ͔ΒαϯϓϧΛಘΔɽ q(z) z(t)
∼ q(z) q(z)
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ˜ u Ұ༷
͔ΒͷαϯϓϧΛಘΔ Uni(0,kq(z)) ˜ u ∼ Uni (0,kq(z(t)))
غ٫αϯϓϦϯά z ˜ p(z) kq(z) z(t) kq(z(t)) ड༰ غ٫ ˜
u αϯϓϧ ͷड༰ʢBDDFQUʣغ٫ʢSFKFDUʣબɽ z(t) if ˜ u > ˜ p(z(t)) then SFKFDU else BDDFQU ˜ p(z(t))
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ʹର͢Δؔ ͷظɹ Λ୯७ϞϯςΧϧϩ๏Α ΓޮతʹٻΊ͍ͨɽɹ p(z) f(z) p(z) [
f(z)] = ∫ f(z)p(z)dz త ঢ়گ ɹظ ͷղੳతͳੵܭࢉ͕ࠔɽ ɹ ͔ΒαϯϓϦϯάΛಘΒΕͳ͍ɽ ɹਖ਼نԽ͞Ε͍ͯͳ͍ؔ ܭࢉՄೳɽ ∫ f(z)p(z)dz p(z) ˜ p(z)( = Zp ⋅ p(z)) എܠ ɹغ٫αϯϓϦϯάΛ༻͍ͯɼ ΛΘͣʹαϯϓϧΛऔಘ͠ɼظ Λٻ ΊΔ͜ͱͰ͖Δ͕ɼ ͷ͕খ͞ͳྖҬʹαϯϓϧ͕ूத͢ΔՄೳੑ͕͋Δɽ ୯७ϞϯςΧϧϩ๏ͷܭࢉͷد༩͕গͳ͍ɽ ͷ͕େ͖͘ͳΔΑ͏ͳ ྖҬΛॏతʹαϯϓϧͨ͠ํ͕ޮ͕͍͍ɽ p(z) p(z) [ f(z)] f(z) ⟹ f(z)p(z)
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ·ͣɼఏҊ Λઃఆ͢Δɽਖ਼نԽ͞Ε͍ͯͳ͍ؔ ɼҎԼͷΑ͏ʹද ͞ΕΔɽ ɹظͷܭࢉɼҎԼͷΑ͏ʹมܗͰ͖Δɽ q(z)
˜ p(z), ˜ q(z) p(z) = 1 Zp ˜ p(z), q(z) = 1 Zq ˜ q(z) ∫ f(z)p(z)dz = ∫ f(z) p(z) q(z) q(z)dz = q(z) [ f(z) p(z) q(z) ] = ∫ f(z) 1 Zp ˜ p(z) 1 Zq ˜ q(z) q(z)dz = Zq Zp q(z) [ f(z) ˜ p(z) ˜ q(z) ] ≈ Zq Zp 1 T T ∑ t=1 f(z(t)) ˜ p(z(t)) ˜ q(z(t)) = Zq Zp 1 T T ∑ t=1 f(z(t))w(t), w(t) = ˜ p(z(t)) ˜ q(z(t)) ख๏
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 )
ࣗݾਖ਼نԽॏ৺αϯϓϦϯά ɹ ະͷͳͷͰɼҎԼͷΑ͏ʹਖ਼نԽ߲ͷൺΛۙࣅ͢Δɽ ɹΑͬͯɼؔ ͷظ͕ۙࣅతʹಘΒΕΔɽ Zp
Zp Zq = ∫ ˜ p(z) Zq dz = ∫ ˜ p(z) ˜ q(z) q(z)dz = q(z) [ ˜ p(z) ˜ q(z) ] ≈ 1 T T ∑ t=1 w(t), z(1), …, z(T) ∼ q(z) f(z) ख๏ ͖ͭͮʣ ( ∵ p(z) = 1 Zp ˜ p(z) ) ( ∵ ∫ p(z)dz = 1 Zp ∫ ˜ p(z)dz = 1 ) ͳΜͷͨΊʹ Λ ͖࣋ͬͯͨΜͩΖ͏ʜ ˜ q(z)
ࣗݾਖ਼نԽॏαϯϓϦϯάʢޡهͷՄೳੑʣ ʮϕΠζਂֶशʯQ ɹࣗݾਖ਼نԽॏαϯϓϦϯάͷஈམͷ࠷ॳ ޡΓ ɹغ٫αϯϓϦϯάͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z) ɹ୯७ϞϯςΧϧϩ๏ͱҟͳΔɼ ͔Βͷαϯϓϧʜʜ p(z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹغ٫αϯϓϦϯάͷɼߴ࣍ݩʹͳΔͱड༰͕ඇৗʹখ͘͞ͳΔ͜ͱɽ࣮ࡍ ʹɼ࣍ݩఔͷ؆୯ͳੵۙࣅʹ͔͠ద༻Ͱ͖ͳ͍ɽ ɹͰɼߴ࣍ݩۭؒͰޮతʹαϯϓϦϯά͢Δʹʜʜ ɹɹɹ Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ͕ఏҊ͞Ε͍ͯΔɽ ⟹ ࣍Ϛϧίϑ࿈ ɹ֬มͷܥྻ
ʹରͯ͠ ͕Γཱͭͱ͖ͷܥྻ ͷ͜ͱɽ z(1), z(2), … p(z(t) |z(1), z(2), …, z(t−1)) = p(z(t) |z(t−1)) z(1), z(2), … άϥϑΟΧϧϞσϧ z(1) z(2) z(t−1) z(t) ⋯ ɹભҠ֬ɹΛ ͱ͓͍ͨͱ͖ɼ ͕Γཱͭͱ͖ɼ Λɹఆৗɹͱ͍͏ɽ (z(t−1), z(t)) = p(z(t) |z(t−1)) p* (z) = ∫ (z′ , z)p* (z′ )dz′ p* (z)
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹఆৗ ʹऩଋ͢ΔΑ͏ͳભҠ֬ Λઃܭ͢Δͱɼ ͔Βαϯϓϧ ΛಘΔ͜ͱ͕Ͱ͖Δɽ ఆৗʹີܭࢉ͕ࠔͳ֬Λ͓͘ɽ p* (z) (z(t−1),
z(t)) p* (z) ⟹ ख๏ͷΩϞ ৄࡉΓ߹͍݅ p* (z)(z, z′ ) = p* (z′ )(z′ , z) ʲे݅ʳৄࡉΓ߹͍͕݅Γཱͭ ఆৗͱͳΔɽ ⟹ p* (z) p* (z)(z, z′ ) = p* (z′ )(z′ , z) ⇒ ∫ p* (z)(z, z′ )dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) ∫ p(z′ |z)dz′ = ∫ p* (z′ )(z′ , z)dz′ ⇔ p* (z) = ∫ p* (z′ )(z′ , z)dz′
Ϛϧίϑ࿈ϞϯςΧϧϩ๏ʢ.$.$ʣ ɹΓ߹͍݅ʹՃ͑ͯɼαϯϓϧ͕ ͱͨ͠ͱ͖ɼॳظঢ়ଶ ʹ͔͔ΘΒ ͣɼ ͕ఆৗ ʹऩଋ͢Δඞཁ͕͋Δɽ Τϧΰʔυੑ t →
∞ p(z(1)) p(z(t)) p* (z) ⟹ Τϧΰʔυੑ w طੑɹɿҙͷঢ়ଶ͔Βҙͷঢ়ଶ༗ݶճͰભҠՄೳɽ w ඇपظੑɿͯ͢ͷঢ়ଶ͕ݻఆͷपظੑΛͨͳ͍ɽ w ਖ਼࠶ؼੑɿಉ͡ঢ়ଶ͕༗ݶճͰΔ͜ͱ͕Մೳɽ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ p(z)
∝ ˜ p(z) ˜ p(z) ख๏ ɹભҠ֬ ͕ઃܭ͕͍͠߹ɼભҠͷఏҊ Λ͑Δɽ (z′ , z) q(z|z′ ) ɽఏҊ ͔Β࣍ͷαϯϓϧͷީิ ΛαϯϓϦϯά͢Δɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ q( ⋅ |z(t)) z* r r = ˜ p(z* )q(z(t) |z* ) ˜ p(z(t))q(z* |z(t)) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ৄࡉΓ߹͍݅ͷূ໌ ɹભҠ֬ɼҎԼͷΑ͏ʹͳΔɽ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) p(z)(z, z′ ) = p(z)q(z′ |z) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) = p(z)q(z′ |z) min (1, p(z′ )q(z|z′ ) p(z)q(z′ |z) ) = min (p(z)q(z′ |z), p(z′ )q(z|z′ )) = min (p(z′ )q(z|z′ ), p(z)q(z′ |z)) = p(z′ )q(z|z′ ) min (1, p(z)q(z′ |z) p(z′ )q(z|z′ )) = p(z′ )q(z|z′ ) min (1, ˜ p(z)q(z′ |z) ˜ p(z′ )q(z|z′ )) = p(z′ )(z′ , z) ɹ ͷ߹ɼϝτϩϙϦε๏ͱݺΕΔɽ q(z′ |z) = q(z|z′ )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ʮϕΠζਂֶशʯQࣜʢʣ ޡΓ (z, z′ ) = q(z′ |z) min
(1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) ) (z, z′ ) = q(z|z′ ) min (1, ˜ p(z′ )q(z|z′ ) ˜ p(z)q(z′ |z) )
ϝτϩϙϦεɾϔΠεςΟϯάε๏ʢ.)๏ʣ ۩ମྫͰֶͿ ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ఏҊ
ͭ·Γ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} q(z* |z) = (z′ |z, I) z* ∼ (z, I) https://drive.google.com/open?id=1vcBZWp9HPzfCzBjj2INdkH_CJUDL-JA3
ɹαϯϓϦϯάͷલʹϋϛϧτχΞϯΛར༻ͨ͠ղੳֶతͳγϛϡϨʔγϣϯΛղ આɽຎࡲʹΑΔΤωϧΪʔͷݮগ͕ͳ͍ͱԾఆ͢ΔͱɼϋϛϧτχΞϯҎԼͷΑ͏ʹ ද͞ΕΔɽ ℋ(z, p) = (z) + (p), (p)
= 1 2m pTp, z ∈ ℝD: ମͷҐஔϕΫτϧ, p ∈ ℝD: ମͷӡಈྔϕΫτϧ, m ∈ ℝ: ମͷ࣭ྔ, ℋ(z, p): ϋϛϧτχΞϯ, (z): ϙςϯγϟϧΤωϧΪʔ, (p): ӡಈΤωϧΪʔ . ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ).$๏ʢϋΠϒϦοτϞϯςΧϧϩ๏ʣʹମͷيಓͷझຯϨʔγϣϯ .)๏ × ɹ).$๏ɼϥϯμϜΥʔΫతͳ.)ͱൺͯɼޮతʹۭؒΛ୳ࡧՄೳɽ ϋϛϧτχΞϯͷγϛϡϨʔγϣϯ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹ࣭ྔΛͱ͠ɼ ͱ ͷ࣌ؒ ʹؔ͢ΔڍಈɼϋϛϧτχΞϯͷภඍʹΑܾͬͯఆɽ ͜ͷඍํఔ͕ࣜղੳతʹղ͚ͳ͍ͷͱ͠ɼγϛϡϨʔγϣϯʹΑͬͯيಓΛܭ ࢉ͢Δɽ z
p τ dpi dτ = − dℋ dzi = − d dzi , dzi dτ = dℋ dpi = d dpi . ΦΠϥʔ๏ ࣌ࠁ ઌͷڍಈΛۙࣅతʹ༧ଌɽ ϵ > 0 pi (τ + ϵ) = pi (τ) + ϵ dpi dτ τ = pi (τ) − ϵ d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵ dzi dτ τ = zi (τ) + ϵpi (τ) ࢄԽʹΑΔޡ͕ࠩେ͖͍ɽ Ϧʔϓϑϩοά๏ ⟹
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοά๏ pi (τ + ϵ 2 ) = pi
(τ) − ϵ 2 d dzi zi (τ) , zi (τ + ϵ) = zi (τ) + ϵpi (τ + ϵ 2 ), pi (τ + ϵ) = pi (τ + ϵ 2 ) − ϵ 2 d dzi zi (τ + ϵ) . ͜ΕΛ ճ܁Γฦ͢͜ͱͰ࣌ࠁ ઌͷମͷҐஔ ͱӡಈྔ ΛۙࣅతʹܭࢉͰ͖Δɽ L ϵL z* p* ϋϛϧτχΞϯͷੑ࣭ ɽ ࣌ؒ ʹΑͬͯෆมɽ ɽՄٯੑɿ ͔Β ͷભҠҰରҰɽ ɽମੵอଘ ℋ τ (z, p) (z* , p* )
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ αϯϓϦϯάΞϧΰϦζϜͷద༻ త ɹະͷ֬ ͔ΒαϯϓϦϯάΛಘΔɽ p(z) લఏ ɹ ͱͳΔਖ਼نԽ͞Ε͍ͯͳ͍ؔ طͰ͋Δɽ
ɹ ͱ֦ு͢Δͱɼ पล ͔Βαϯϓϧ͕ಘΒΕΔɽ ɹ p(z) ∝ ˜ p(z) ˜ p(z) p(z, p) = p(z)p(p) z p(z) p(p) = (p|0, I) (z) = − log (˜ p(z)) (p) = 1 2 pTp
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ɹಉ࣌Λܭࢉ͢ΔͱɼҎԼͷΑ͏ʹͳΔɽ ϝτϩϙϦε๏ͰΘΕΔൺ ɼҎԼͷΑ͏ʹͳΔɽ p(z, p) = p(z)p(p) =
exp (log p(z) + log p(p)) ∝ exp (log ˜ p(z) − 1 2 pTIp ) = exp (−(z) − (p)) = exp (−ℋ(z, p)) r r = p(z* , p* ) p(z, p) = exp (−ℋ(z* , p* ) + ℋ(z, p))
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ख๏ ɽӡಈྔΛαϯϓϦϯά ɽϦʔϓϑϩοά๏Ͱݱࡏͷ ͔Βީิ ΛಘΔɽ ɽ࣍ͷൺ Λܭࢉ͢Δɽ
ɽ Λ֬ ʹΑͬͯ ͱͯ͠ड༰͠ɼͦ͏Ͱͳ͍߹ ɼ ͱ͢Δɽ p ∼ (0, I) (z(t), p) (z* , p* ) r r = p(z* , p* ) p(z, p) z* min(1,r) z(t+1) ⟵ z* z(t+1) ⟵ z(t) ΞϧΰϦζϜͷྲྀΕ
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ Ϧʔϓϑϩοάͷύϥϝʔλʹؔͯ͠ɼҎԼͷΑ͏ͳτϨʔυΦϑ͕͋Δɽ ɹ).$๏ɼࣄޙͷඍ͑͞ܭࢉͰ͖Εద༻Ͱ͖ɼඇৗʹ൚༻తɽҰൠతͳ χϡʔϥϧωοτϫʔΫ࿈ଓͳજࡏมͷΈͰΓཱ͍ͬͯΔ͜ͱ͕ଟ͍ͷͰɼ).$ ๏χϡʔϥϧωοτϫʔΫͷϕΠζֶशʹΘΕ͖ͯͨɽ େ͖͍ εςοϓαΠζ ϵ εςοϓ L
খ͍͞ খ͍͞ େ͖͍ ड༰ ड༰ ୳ࡧޮ ܭࢉྔ ߴ͍ ߴ͍ ͍ ͍ େ͖͍ খ͍͞ ߴ͍ ͍
ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ϥϯδϡόϯಈྗֶ๏ ɹ ͱͨ͠߹ɼϥϯδϡόϯϞϯςΧϧϩ๏ɹ·ͨɹϥϯδϡόϯಈྗֶ๏ɹͱ ݺΕΔɽ ɹਂֶश͚ʹϛχόονֶश͕ߦ͑ΔΑ͏ʹͨ͠ɹ֬తޯϥϯδϡόϯಈྗֶ ๏ɹʹల։͞ΕΔɽ L =
1 z*i = zi (τ + ϵ) = zi (τ) + ϵ pi (τ) − ϵ 2 d dzi zi (τ) = zi (τ) − ϵ2 2 d dzi zi (τ) + ϵpi (τ)
ඪʢະʣ ɹɹɹɹɹɹɹ ਖ਼نԽ͞Ε͍ͯͳ͍ؔʢطʣ ӡಈྔͷαϯϓϦϯάɿ ӡಈΤωϧΪʔɿ
ҐஔΤωϧΪʔɿ ҐஔΤωϧΪʔͷภඍɿ p(z) = (z|μ, Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} ˜ p(z) = exp {− 1 2 (z − μ)TΣ−1(z − μ)} p ∼ (0, I) (p) = 1 2 pTp (z) = − log (˜ p(z)) = − 1 2 (z − μ)TΣ−1(z − μ) ∂ ∂z = − (z − μ)TΣ−1 ϋϛϧτχΞϯϞϯςΧϧϩ๏ʢ).$๏ʣ ۩ମྫͰֶͿ https://drive.google.com/open?id=11zWctTbECXEhlHm7AqPiXC_MErAYl7hJ
ΪϒεαϯϓϦϯά త ɹ֬ ͔Β શମΛαϯϓϦϯά͢Δ͜ͱ͕͍͠ͱ͖ͷ୳ࡧɽ p(Z) Z લఏ ɹ֬ طɽ
p(Z) ख๏ ɹɽม Λ ݸͷ෦ू߹ʹ͚Δɽ ɹɽ෦ू߹Λஞ࣍తʹ୳ࡧ͢Δɽ ɹ Z M Z = {Z1 , Z2 , …, ZM } Z1 ∼ p(Z1 |Z2 , Z3 , …, ZM−1 , ZM ) Z2 ∼ p(Z2 |Z1 , Z3 , …, ZM−1 , ZM ) ⋮ ZM ∼ p(ZM |Z1 , Z2 , …, ZM−2 , ZM−1 )
ΪϒεαϯϓϦϯά ɹΪϒεαϯϓϦϯάͷଥੑɼαϯϓϦϯάͷखଓ͖͕.)๏ͷҰछͱͯ͠ղऍͰ ͖Δ͜ͱ͕อূ͞Ε͍ͯΔɽ ͷΑ͏ʹ͚ɼ Λ͚݅ͨ͠ͱͰ ͷα ϯϓϦϯάΛ͢Δ͜ͱΛߟ͑ͨ߹ɼ ͔ͭ ɹൺ
Λܭࢉ͢ΔͱҎԼͷΑ͏ʹͳΔɽ Αͬͯɼશͯड༰͞ΕΔɽ ಉ༷ɽ Z = {Z1 , Z2 } Z2 Z1 q(Z* |Z) = p(Z1* |Z2* ) Z2 = Z2* r r = p(Z* )q(Z|Z* ) p(Z)q(Z* |Z) = p(Z1* , Z2* )p(Z1 |Z2* ) p(Z1 , Z2 )p(Z1* |Z2 ) = p(Z1* |Z2* )p(Z2* )p(Z1 |Z2* ) p(Z1 |Z2 )p(Z2 )p(Z1* |Z2 ) = 1 Z2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ ඪʢطʣ ɹɹɹɹ ͱ͢Δɽ p(z) = (z|μ,
Σ) = 1 (2π)D |Σ| exp {− 1 2 (z − μ)TΣ−1(z − μ)} z = ( z1 z2 ), μ = ( μ1 μ2 ), Σ = ( Σ11 Σ12 Σ21 Σ22 ), Λ = Σ−1 = ( Λ11 Λ12 Λ21 Λ22 ) log p(z) = log p(z1 , z2 ) = − 1 2 (z1 − μ1 )TΛ11 (z1 − μ1 ) + (z1 − μ1 )TΛ12 (z2 − μ2 )) − 1 2 (z2 − μ2 )TΛ22 (z2 − μ2 ) + (z2 − μ2 )TΛ21 (z1 − μ1 )) = − 1 2 ( zT 1 Λ11 z1 − 2z1 {Λ11 μ1 − 1 2 Λ12 (z2 − μ2 )}) + C1 − 1 2 ( zT 2 Λ22 z2 − 2z2 {Λ22 μ2 − 1 2 Λ21 (z1 − μ1 )}) + C2
ΪϒεαϯϓϦϯά ۩ମྫͰֶͿ Αͬͯɼ͖݅֬ͷରɼ ͱͳΔͷͰɼ͖݅֬ΨεͰ͋Δɼ log p(zi |zj
) = − 1 2 ( zT i Λii zi − 2zi { Λii μi − 1 2 Λij (zj − μj )}) + C p(zi |zj ) = (zi |μi , Σi ), Σ−1 i = Λii , Σ−1 i μi = Λii μi − 1 2 Λij (zj − μj ), ⇔ μi = Σi ( Λii μi − 1 2 Λij (zj − μj )) . https://drive.google.com/open?id=1ReYNvvH-NgtsuRiDDV-lz1779sps2pT0