これでわかる GAN

生成モデルや情報量についても軽く触れるGAN論文まとめ

Transcript

1. ("/(FOFSBUJWF
   "EWFSTBSJBM
   /FUT :PTVLF)PTIJ 1
2. ("/ͱ͸ ɹ("/ ఢରతੜ੒ωοτϫʔΫ
   ɾ(FOFSBUJWF
   NPEFM
   
   ੜ੒Ϟσϧ
   ɾ%JTDSJNJOBUPS
   NPEFM
   
   ൑ผϞσϧ ̎ͭͷఢରతϞσϧΛ௨ͯ͠ੜ੒ϞσϧΛධՁ͢ΔϑϨʔϜϫʔΫ 2
3. ("/ͱ͸ 3
4. ("/ͱ͸ 4
5. ੜ੒ͱೝࣝ જࡏม਺ O࣍ݩͷಛ௃
   ੑผɺਓछɺ൅ͷ৭ʜFUD ը૾ ೝࣝ ੜ੒ 5
6. ੜ੒ͱࣝผϞσϧ ੜ੒Ϟσϧ
   σʔλͷ෼෍ঢ়ଶΛ֬཰తʹଊ͑ɺͦͷσʔλ෼෍ʹैͬͯσʔλΛੜ੒͢Δ
   ֬཰Ϟσϧ
   ࣝผϞσϧ
   ༩͑ΒΕͨσʔλΛԿͰ͋Δ͔ࣝผ͢ΔϞσϧ 47.ͱ͔ 6
7. "EWFSTBSJBM
   /FUT 
   ه߸ͷఆٛ
   
   ೖྗϊΠζ[ͷࣄલ෼෍
   
   ೖྗ[ʹର͢Δ(FOFSBUPSͷग़ྗ ͷ෼෍
   
   ೖྗ[ʹର͢ΔσʔλۭؒΛද͢.-1
   
   Ϟσϧͷύϥϝʔλ
   
   %JTDSJNJOBUPSͷ.-1 ग़ྗ͸୯ҰεΧϥ
   
   ೖྗY͕ Ͱ͸ͳ͘αϯϓϧσʔλ༝དྷͩͱ͢Δ֬཰ Pz (z) : Pg (z) : G(Z; θg ) : θg : D(Z; θg ) : D(x) : Pg 7
8. "EWFSTBSJBM
   /FUT Pz (z) G Pg (z) Pdata (x) D
   0 1 G(Z; θg ) D(Z; θg ) 8
9. "EWFSTBSJBM
   /FUT %JTDSJNJOBUPSͷֶश
   
   
   αϯϓϧͱ(ʹରͯͦ͠ΕͧΕਖ਼͍͠ϥϕϧΛׂΓ౰ͯΔ֬཰ Λ࠷େԽ
   Λ࠷খԽ͍ͤͨ͞
   (FOFSBUPSͷֶश
   
   
   αϯϓϧͱ(ʹରͯͦ͠ΕͧΕਖ਼͍͠ϥϕϧΛׂΓ౰ͯΔ֬཰ Λ࠷খԽ
   Λ࠷େԽ͍ͤͨ͞
   ͜ΕΛఆࣜԽ͢Δͱʜ D(x) 1 − D(x) D(x) 1 − D(x) 9
10. "EWFSTBSJBM
    /FUT ਓϓϨΠϠʔϛχϚοΫεήʔϜʹ
    
    min ⏟ G max ⏟ D V(D, G) = x∼Pdata(x) [log D(x)] + z∼Pz(z) [log(1 − D(G(z)))] 10
11. "EWFSTBSJBM
    /FUT θϩ࿨ήʔϜ ;FSPTVN
    HBNF
    
    ૬ޓʹӨڹ͠߹͏ঢ়گͷதͰɺશһͷརಘͷ߹ܭ͕ৗʹʹͳΔήʔϜͷ͜ͱ
    ࠓճ͸(FOFSBUPSͱ%JTDSJNJOBUPSͷਓθϩ࿨ήʔϜʹ֘౰͢Δ
    ্هͷඇڠྗήʔϜΛղͨ͘ΊʹϛχϚοΫεఆཧ ෆ޾தͷ޾͍Λ୳͢ ૝ఆ͞ΕΔ
    ࠷େͷଛ֐͕࠷খʹͳΔΑ͏ʹܾஅΛߦ͏ Λ༻͍Δ 11
12. "EWFSTBSJBM
    /FUT
    ࠨ୺
    
    (͸%ʹؒҧ͍͑ͤͨ͞ͷͰ࠷খԽ
    
    ਖ਼ࣝ͘͠ผͤ͞Α͏ͱ͢ΔͷͰ࠷େԽ min ⏟ G max ⏟ D V(D, G) = x∼Pdata(x) [log D(x)] + z∼Pz(z) [log(1 − D(G(z)))] min ⏟ G : max ⏟ D : 12
13. "EWFSTBSJBM
    /FUT
    ӈลୈ߲
    
    %͕ຊ෺Λຊ෺ͱ൑ผ͜͜Ͱ͸αϯϓϧσʔλYΛ%ʹ౉͍ͯ͠ΔͷͰ
    ͭ·Γ Αͬͯظ଴஋΋ʹ͍ۙ΄͏͕ྑ͍ min ⏟ G max ⏟ D V(D, G) = x∼Pdata(x) [log D(x)]+z∼Pz(z) [log(1 − D(G(z)))] x∼¶data(x) [log D(x)] : D(x) = 1 log 1 = 0 13
14. "EWFSTBSJBM
    /FUT
    ӈลୈ߲
    
    %ʹِ෺ͷੜ੒ͨ͠σʔλΛ౉͍ͯ͠Δ
    %ʹ͍ͭͯ͸ɺ
    
    ͱͳΕ͹ྑ͍
    (ʹ͍ͭͯ͸ɺ
    ͱͳΕ͹ྑ͍ min ⏟ G max ⏟ D V(D, G) = x∼Pdata(x) [log D(x)] + z∼Pz(z) [log(1 − D(G(z)))] z∼¶z(z) [log(1 − D(G(z)))] : D(G(z)) → 0 D(G(z)) → 1 14
15. ֶशͷྲྀΕ 0WFSpUΛආ͚ΔͨΊʹϞσϧ%ͷߋ৽ΛLεςοϓߦͳ͔ͬͯΒ
    Ϟσϧ(ͷߋ৽Λɺεςοϓߦ͏
    D * (x) = Pdata (x) Pdata (x) + Pg (x) 15
16. ֶशͷྲྀΕ
    %ͷֶश ͔ΒNίͷϊΠζΛαϯϓϦϯάِͯ͠෺ΛNί࡞Δ࣍ʹ ͔Β΋
    NίͷσʔλΛαϯϓϦϯά͠ɺ֬཰తޯ഑๏ʹΑͬͯϞσϧ%Λֶशͤ͞Δ
    Pz (z) Pdata (x) ▽θd 1 m m ∑ i=1 [logD(x(i)) + log(D(G(z(i))))] 16
17. ֶशͷྲྀΕ
    (ͷֶश %ͷޯ഑Λ΋ͱʹ ͕ݩσʔλ෼෍ͱࣅΔΑ͏ʹֶशͤ͞Δ
    ͔ΒNίͷϊΠζΛαϯϓϦϯάِͯ͠෺ΛNί࡞ΔͦΕΒΛ༻͍ͯϞσϧ(Λߋ৽ G(z) Pz (z) ▽θd 1 m m ∑ i=1 log(1 − D(G(z(i)))) 17
18. ֶशͷྲྀΕ ͜ΕΛ܁Γฦ͢ͱ ͱ ͕͖ۙͮ ͷঢ়ଶʹͳΔ
    Ϟσϧ%͕ػೳ͠ͳ͍ঢ়ଶ Pg Pdata D(x) = 1 2 18
19. ϑϨʔϜϫʔΫͷਖ਼౰ੑͷূ໌
    Λݻఆͨ࣌͠ͷ ͷ࠷దղ ΛٻΊΔ
    
    
    
    ͜͜Ͱɺੵ෼ͷத਎Λ࠷େԽ͍ͨ͠ʂ
    ˠ
    % Y Ͱඍ෼͢Δ min ⏟ G max ⏟ D V(D, G) = x∼Pdata(x) [log D(x)] + z∼Pz(z) [log(1 − D(G(z)))] G D DG (x)* V(D, G) = ∫ x [Pdata (x)log(D(x))]dx + ∫ z [Pz (z)log(1 − D(G(z)))]dz = ∫ x [Pdata (x)log(D(x))]dx + ∫ x [Pg (x)log(1 − D(x))]dx = ∫ x [Pdata (x)log(D(x)) + Pg (x)log(1 − D(x))]dx 19
20. ϑϨʔϜϫʔΫͷਖ਼౰ੑͷূ໌ લࣜ
    ʜᶃ
    ᶃࣜͷத਎Λ࠷େԽ͢Δ
    
    '0$ΑΓ
    
    
    
    
    = ∫ x [Pdata (x)log(D(x)) + Pg (x)log(1 − D(x))]dx f(x) = Pdata (x)log(D(x)) + Pg (x)log(1 − D(x)) df(x) dD(x) = 0 Pdata (x) D(x) − Pg (x) 1 − D(x) = 0 Pdata (x) D(x) = Pg (x) 1 − D(x) Pdata (x) − Pdata (x)D(x) = Pg (x)D(x) DG (x)* = Pdata (x) Pdata (x) + Pg (x) 20
21. ϑϨʔϜϫʔΫͷਖ਼౰ੑͷূ໌
    ࣍ʹɺ Λ༻͍ͯ ͷ࠷దղΛٻΊΔ
    
    
    
    
    
    min ⏟ G max ⏟ D V(D, G) = x∼Pdata(x) [log D(x)] + z∼Pz(z) [log(1 − D(G(z)))] DG (x)* G C(G) = max ⏟ D V(G, D) = V(G, D* G (x)) = ∫ x [Pdata (x)log(D* G (x)) + Pg (x)log(1 − D* G (x))]dx = ∫ x [Pdata (x)log( Pdata (x) Pdata (x) + Pg (x) ) + Pg (x)log( Pg (x) Pdata (x) + Pg (x) )]dx = ∫ x [Pdata (x)log( 1 2 Pdata (x) Pdata (x) + Pg (x) 2 ) + Pg (x)log( 1 2 Pg (x) Pdata (x) + Pg (x) 2 )]dx 21
22. ϑϨʔϜϫʔΫͷਖ਼౰ੑͷূ໌
    
    
    ͜͜Ͱɺ ͷͱ͖
    
    = ∫ x [Pdata (x)log( 1 2 Pdata (x) Pdata (x) + Pg (x) 2 ) + Pg (x)log( 1 2 Pg (x) Pdata (x) + Pg (x) 2 )]dx = ∫ x [Pdata (x)log( Pdata (x) Pdata (x) + Pg (x) 2 )]dx + ∫ x [Pg (x)log( Pg (x) Pdata (x) + Pg (x) 2 )]dx − ∫ x [(Pdata + Pg )log2]dx = DKL (Pdata || Pdara + Pg 2 ) + DKL (Pg || Pdata + Pg 2 ) − ∫ x [(Pdata + Pg )log2]dx Pdata = Pg = 0 − 2log2 = − log4 22
23. ϑϨʔϜϫʔΫͷਖ਼౰ੑͷূ໌ Ҏ্ΑΓ ͸ ͷάϩʔόϧ࠷খ஋ʹͳΔ
    ͜ͷ࣌ͷ།Ұղ͸ Ͱ͋Γɺଈͪੜ੒Ϟσϧ͕
    ׬ᘳʹෳ੡Λੜ੒͢ΔϞσϧͰ͋Δɺͱ͍͏͜ͱͰ͋Δ C* = − log4 C(G) Pdata = Pg 23
24. ࢀߟจݙ (FOFSBUJWF
    "EWFSTBSJBM
    /FUXPSL
    IUUQTBSYJWPSHQEGQEG
    ύλʔϯೝࣝͱػցֶश
    ؙળग़൛
    ύλʔϯೝࣝͱػցֶशͷֶश
    IUUQTIFSVNJHJUIVCJPQSNM
    ৘ใྔ
    %JWFSHFODF
    IUUQTRJJUBDPNLJMPNFUFSJUFNTCFFEFGFBEBDB
    24
25. "11&/%*9 25
26. ੜ੒Ϟσϧ
    
    
    Y͕༩͑ΒΕͨͱ͖ͷ ͷࣄޙ֬཰͜Εͷ࠷େ஋Ͱϥϕϧ෼͚Λ͢Δ
    
    
    ύϥϝʔλ ΛܾΊͨͱ͖ͷσʔλYͷ໬౓
    
    
    Yͷੜى֬཰
    
    
    ύϥϝʔλ ͷࣄલ֬཰ P(Ck |x) = P(x|Ck ) P(x) × P(Ck ) P(Ck |x) Ck P(x|Ck ) Ck P(x) P(Ck ) Ck 26
27. ੜ੒Ϟσϧ
    
    ˠ
    पลԽपล֬཰ͱ΋͍͏
    ·ͨ͸
    
    ͭ·Γɺ ͕͋Δͱ֤ ʹରͯ͠YΛੜ੒Ͱ͖ΔͷͰੜ੒Ϟσϧͱ͍͏ P(Ck |x) = P(x|Ck ) P(x) × P(Ck ) P(x) = k ∑ i=1 P(Ci , x) P(x) = k ∑ i=1 P(x|Ck )P(Ck ) P(x|Ck ) Ck 27
28. ৘ใྔ ৘ใྔ
    ͋Δग़དྷࣄ͕ى͖ͨ࣌ɺͦΕ͕ͲΕ͚ͩ௝͍͔͠Λද͢ई౓ͷ͜ͱ
    ˠͭ·Γ௝͍͠ঢ়ଶͰ͋Ε͹ͦΕ͕࣋ͭ৘ใྔ͸େ͖͍ͱ͍͏͜ͱʹͳΔɻ ྫɿԯਓ͍ΔੈքͰࠃࡍࢦ໊ख഑൜Λࠃ੶͔Β୳͢
     ͍͋ͭ͸தࠃਓͩ
    ˠதࠃࠃຽ͕ԯਓͩͱ͢Δͱ൜ਓ͸શਓޱͷ
    
     ͍͋ͭ͸όνΧϯࢢࠃਓͩ
    ˠόνΧϯࢢࠃຽ͕ਓͩͱ͢Δͱ൜ਓ͸શਓޱͷ
    
    ͭ·Γ൜ਓͷࠃ੶͕όνΧϯࢢࠃຽͩͱΘ͔ͬͨ࣌ͷํ͕৘ใྔ͕େ͖͍
    1 7 1 7000000 28
29. ৘ใྔ ྫɿਖ਼ղ͕̍ͭͷܻͷ਺ࣈΛ౰ͯΔๅ͘͡ શͯͰ௨Γ
     ॳΊͷ਺ࣈΛڭ͑ͯ΋Β͏
    ˠ࢒ΓͷܻΛ౰ͯͳ͍ͱ͍͚ͳ͍
    
     ܻ໨·Ͱڭ͑ͯ΋Β͏
    ˠ࢒Γܻ౰ͯͳ͍ͱ͍͚ͳ͍
    
    ͭ·Γܻڭ͑ͯ΋Β͏ํ͕৘ใྔ͕େ͖͍
    1 × 105 106 14 × 102 106 29
30. ࣗݾ৘ใྔ ࣄ৅"͕ى͜Δ֬཰Λ ͱ͢Δ࣌ɺࣄ৅"ͷ৘ใྔ Λ Ͱද͢
    ͜ͷ࣌৘ใྔ ͸ҎԼͷ৚݅Λຬͨ͢
     ʹରͯ͠୯ௐݮগ
    ˠ֬཰͕େ͖͘ͳΕ͹৘ใྔ͸খ͘͞ͳΔ
     Ճ๏ੑ
    ˠ"ͱ#͕ಠཱͷ࣌
     ͸CJU
     ࿈ଓੑ P(A) I(A) P(A) I(A) P(A) P(AB) = P(A) + P(B) P(A) = 1 2 30
31. ࣗݾ৘ใྔ ઌ΄Ͳͷ৚݅Λຬͨ͢Α͏ʹ͢Δͱ৘ใྔ͸ର਺ؔ਺Ͱද͞ΕΔ
    
    ·ͨɺ৘ใྔ ͷظ଴஋ΛΤϯτϩϐʔͱݺͼ ͱද͢ͱ
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    I(A) = log 1 P(A) = − logP(A) I(A) H(A) A = [a1 , a2 , . . . , an ] H(A) = E[P(A)] = n ∑ i=1 P(ai )I(ai ) = − n ∑ i=1 P(ai )logP(ai ) 31
32. ΧϧόοΫŋϥΠϒϥʔڑ཭ ҟͳΔ̎ͭͷ෼෍ͷڑ཭ ࠩҟ Λද͢৘ใྔͷ͜ͱ
    ͦΕͧΕΛ ͱ ͱ͢Δͱ̎ͭͷ෼෍ͷࠩ෼͸࣍ͷΑ͏ʹͳΔ
    
    Τϯτϩϐʔͱಉ͡Α͏ʹظ଴஋Λͱͬͨ΋ͷ͕ΧϧόοΫϥΠϒϥʔڑ཭
    
    ͨͩ͠1ͱ2͸ඇରশͰ͋Γ਺ֶతͳڑ཭ͷఆٛΛຬͨ͞ͳ͍
    ରশʹͨ͠΋ͷΛ+FOTFO4IBOOPOμΠόʔδΣϯε
    P(x) Q(x) (−logQ(x)) − (−logP(x)) = log P(x) Q(x) DKL (P||Q) = ∑ i P(i)log P(i) Q(i) 32
33. +FOTFO4IBOOPOμΠόʔδΣϯε ͱ Λରশʹѻ͏
    
    
    ͜͜Ͱ ͷ࣌ରশ
    P(x) Q(x) 0 ≤ λ ≤ 1 DJS = λDKL (Q||λQ + (1 − λ)P) + (1 − λ)DKL (P||λP + (1 − λ)Q) λ = 1 2 DJS (P||Q) = 1 2 (DKL (P|| P 2 + Q 2 ) + DKL (Q|| Q 2 + P 2 )) = DJS (Q||P) 33