【論文紹介】Bayesian Conditional GAN (BC-GAN)

Bayesian Conditional Generative Adverserial Networks

https://arxiv.org/abs/1706.05477

Transcript

1. BAYESIAN CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS ʲ࿦จ঺հʳ
2. ֓ཁ Introduction ▸ ॻࢽ৘ใ http://arxiv.org/abs/1706.05477 Submitted: 17 Jun 2017 ▸ ֓ཁ ▸ ௨ৗͷGANͱ͸ҟͳΓɺೖྗͰ͸ͳؔ͘਺ଆʹϥϯμϜੑΛ࣋ͨ͢͜ͱͰෆ࣮֬ ੑΛදݱ ▸ GANʹϕΠζͷϑϨʔϜϫʔΫΛద༻
3. Ϟσϧ Bayesian Conditional GAN ɾGenerator, Discriminator ͱ΋ʹϞσϧύϥϝʔλʔࣗମ͕෼෍Λ΋ͭ (Bayesian) ɾGeneretor΁ͷೖྗ͸ϥϕϧ৘ใ (Conditional) ɾग़ྗͷظ଴஋Λܭࢉ͢ΔͨΊʹϞϯςΧϧϩ๏Λར༻ ɹɾϚϧίϑ࿈࠯ϞϯςΧϧϩ๏ (MCMC) ɹɾϥϯδϡόϯಈྗֶ (Gradient Langevin dynamics)
4. Ϟσϧ Learning ▸ MAP-MC ▸ Stochastic Gradient Langevin Dynamics * l is the loss function: log loss *This added noise will ensure the parameters are not only traversing towards the mode of the distributions but also sampling them according to their density. ϊΠζ Discriminator Generator Real Fake Fake RealͱFakeͷෆҰக౓ ෆҰக౓ (Maximum Mean Discrepancy)
5. αϯϓϦϯά Sampling Functions υϩοϓΞ΢τΛ༻͍Δ͜ͱͰϕΠζ తղऍ͕Մೳͳ͜ͱ͕ใࠂ͞Ε͍ͯΔ (https://arxiv.org/abs/1506.02142) ὎ υϩοϓΞ΢τ + ϊΠζʹΑͬͯɹ ϞσϧύϥϝʔλʔΛαϯϓϦϯά αϯϓϦϯά αϯϓϦϯά ظ଴஋ͱ෼ࢄ
6. ࣮ݧ Experiments ▸ MNIST Generator: શ݁߹૚ x 3, Discriminator: શ݁߹૚ ͲͪΒ΋શ݁߹૚ͷޙʹυϩοϓΞ΢τ(ratio=0.1~0.05)ͱΨ΢εϊΠζ (variance=0.9)Λ෇༩͢Δ૚Λ௥Ճ αϯϓϧ਺͸2ճ গ਺ϥϕϧ(semi-supervised)ͰͷֶशΛ࣮ࢪ Discriminatorͷ࠷ऴ૚ͷΈweight normalization Test error MAP-MC Langevin dynamics
7. ࣮ݧ Experiments ▸ CIFAR-10 Generator: FC૚ + Deconvolution x 3, Discriminator: CNN૚ x9 + FC૚ x 2 ֤૚ͷޙʹυϩοϓΞ΢τ(ratio=0.2)ͱΨ΢εϊΠζ(variance=0.9)Λ෇༩͢Δ૚ Λ௥Ճ Discriminator͸֤૚ʹ͍ͭͯweight normalizationΛ࣮ࢪ MAP-MC Langevin dynamics Mode collapse Langevin dynamicsΛ༻͍ͨ৔߹Ͱ͸mode collapse͕ ࢄݟ͞Ε্खֶ͘श͞Εͣ ὎ early stop΍υϩοϓΞ΢τٴͼϊΠζΛେ͖͘͢Δ ͳͲͨ͠ํ͕Α͔͔ͬͨ΋͠Εͳ͍
8. ·ͱΊ Conclusion ▸ Generatorͷෆ࣮֬ੑΛར༻ͯ͠αϯϓϧΛੜ੒͢Δɺ Bayesian Conditional GANΛఏҊ ▸ Discriminatorͷෆ࣮֬ੑΛධՁ͢Δ͜ͱͰɺͦͷύϑΥʔϚϯ εʢconfidence)ΛධՁ͢Δ͜ͱ͕Մೳ ▸ Semi-supervised learning problemʹ͓͍ͯɺSOTAΛୡ੒