Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction of GAN

caojiezhang
December 04, 2018
29

Introduction of GAN

caojiezhang

December 04, 2018
Tweet

Transcript

  1. 2012 2013 2014 2015 2016 2017 2018 generative adversarial reinforcement

    Goodfellow et al., 2014 GAN becomes a very important technology Number of papers whose titles include the keyword
  2. All Kinds of GAN … GAN WGAN BGAN DCGAN ACGAN

    fGAN LCC-GAN (ICML) CGAN ……
  3. GAN Zoo ACGAN BGAN CGAN DGAN EGAN fGAN GAGAN HiGAN

    iGAN JointGAN LCC-GAN KGAN MGAN NetGAN OT-GAN PGAN RCGAN SEGAN ? TGAN UGAN VGAN WGAN XGAN ZipNet-GAN ? https://github.com/hindupuravinash/the-gan-zoo
  4. ➢Generative Modeling ➢Generative Adversarial Networks (GANs) ➢Conditional GANs (CGANs) ➢Training

    Procedure and Evaluation Methods ➢Common Issues and Some Tips ➢Some Applications Contents
  5. Generative Modeling • Have training examples ~ () • Want

    a model that can draw samples: ~ () • Where () ≈ () ~ () ~ ()
  6. Why Generative Models? • Generative models — Date generation —

    Speech synthesis: — Machine translation: Chinese English • Chinese: 我喜欢深度学习。 • English: I love deep learning. • Environment simulator — Reinforcement learning — Planning • Leverage unlabeled data
  7. Data Generation Generator Image Generation 0.1 −0.1 ⋮ 0.7 −0.3

    0.1 ⋮ 0.9 0.3 −0.1 ⋮ −0.7 In a specific range
  8. Data Generation Generator Image Generation Sentence Generation 0.1 −0.1 ⋮

    0.7 −0.3 0.1 ⋮ 0.9 0.3 −0.1 ⋮ −0.7 In a specific range
  9. Data Generation Generator Image Generation Sentence Generation Generator 0.1 −0.1

    ⋮ 0.7 −0.3 0.1 ⋮ 0.9 How are you? 0.3 −0.1 ⋮ −0.7 0.3 −0.1 ⋮ −0.7 In a specific range
  10. Data Generation Generator Image Generation Sentence Generation Generator 0.1 −0.1

    ⋮ 0.7 −0.3 0.1 ⋮ 0.9 0.1 −0.1 ⋮ 0.2 Good morning. How are you? 0.3 −0.1 ⋮ −0.7 0.3 −0.1 ⋮ −0.7 In a specific range
  11. Data Generation Generator Image Generation Sentence Generation Generator 0.1 −0.1

    ⋮ 0.7 −0.3 0.1 ⋮ 0.9 0.1 −0.1 ⋮ 0.2 −0.3 0.1 ⋮ 0.5 Good morning. How are you? 0.3 −0.1 ⋮ −0.7 0.3 −0.1 ⋮ −0.7 Good afternoon. In a specific range
  12. ➢Generative modeling ➢Generative adversarial networks (GANs) ➢Conditional GANs (CGANs) ➢Training

    procedure and evaluation methods ➢Common issues and some tips ➢Some Applications Contents
  13. A Game Theory Perspective ➢Two agents: — Generative model G:

    generate new samples that are as similar as data and fool the discriminator — Discriminative model D: act as a judge to distinguish the generated samples from the real one adversarial Generator Discriminator Vs
  14. Generator Network • G is a neural network with parameters

    . • is a latent variable drawn from a prior distribution. • Trainable for any size of A generative model tries to learn the distribution of real data from the prior distribution ().
  15. Generator Generator It is a neural network (NN), or a

    function. Generator 0.1 −3 ⋮ 2.4 0.9 image vector Generator 3 −3 ⋮ 2.4 0.9 Generator 0.1 2.1 ⋮ 5.4 0.9 Generator 0.1 −3 ⋮ 2.4 3.5 high dimensional vector Each dimension of input vector represents some characteristic. Long hair blue hair Open mouth low dimensional
  16. • A discriminative model learns a function that distinguish the

    generated samples from the real data • (): estimate the probability that is a real data sample — = 1: regards as a real sample — = 0: regards as a fake sample Discriminative Model Binary classification
  17. Discriminator Probability value Image Larger value: real smaller value: fake.

    Discriminator Discriminator Discriminator 1.0 1.0 0.1 Discriminator 0.1 Discriminator It is a neural network (NN), or a function.
  18. ➢Generative modeling ➢Generative adversarial networks (GANs) ➢Conditional GANs (CGANs) ➢Training

    procedure and evaluation methods ➢Common issues and some tips ➢Some Applications Contents
  19. Conditional GANs condition • Simple modification to the original GAN

    framework that conditions the model on additional information for better multi-modal learning. • Many practical applications of GANs when we have explicit supervision available
  20. Conditional GANs D scalar True text-image pairs: G Gaussian distribution

    x = G(c,z) c: train Image is realistic or not + c and are matched or not (train , ) (train , ) (cat , ) 1 0 0 Condition
  21. Conditional GANs paired data blue eyes red hair short hair

    red hair, green eyes blue hair, red eyes
  22. ➢Generative modeling ➢Generative adversarial networks (GANs) ➢Conditional GANs (CGANs) ➢Training

    procedure and evaluation methods ➢Common issues and some tips ➢Some Applications Contents
  23. Training GANs: Two-player game Generator networks: try to fool the

    discriminator by generating real-looking images Discriminator networks: try to distinguish between real and fake images
  24. Minimax Game • Generator minimizes the log-probability of D being

    correct • Discriminator maximizes cross-entropy of real and generated data • Equilibrium is a saddle point of the discriminator loss Cross-entropy Loss
  25. Minimax Game • Discriminator aims to maximize objective such that

    D(x) is close to 1 (real) and D(G(z)) is close to 0 (fake) • Generator aims to minimize objective such that D(G(z)) is close to 1 (discriminator is fooled into thinking generated G(z) is real)
  26. Training Procedure • Use SGD algorithm on two mini-batches simultaneously:

    • A mini-batch of training samples • A mini-batch of generated samples • Optional: Run steps of one player for every step of the other player - example: WGAN updates Discriminator for 5 steps and Generator for 1 step.
  27. sample generated samples G D Update vector vector vector vector

    0 0 0 0 1 1 1 1 randomly sampled Database Step 1: Fix generator G, and update discriminator D Fix Training Algorithm: Discriminator Objective of the discriminator:
  28. Step 2: Fix discriminator D, and update generator G Discriminator

    Generator vector 0.13 hidden layer update fix large network Generator learns to “fool” the discriminator Training Algorithm: Generator Objective of the discriminator:
  29. • Sample m examples 1, 2, … , from database

    • Sample m noise samples 1, 2, … , from a distribution • Obtaining generated data ෤ 1, ෤ 2, … , ෤ , ෤ = • Update discriminator parameters to maximize • ෨ = 1 σ =1 + 1 σ =1 1 − ෤ • ← + ෨ • Sample m noise samples 1, 2, … , from a distribution • Update generator parameters to minimize • ෨ = 1 σ =1 1 − • ← − ෨ Learning D Learning G Training Algorithm of GANs Repeat k times
  30. Training: An Intuitive Explanation Discriminative distribution Data distribution Generative distribution

    After updating D Poorly fit model After updating G Mixed strategy equilibrium
  31. Evaluation: Inception Score = ෍ ෍ | | − ෍

    Negative entropy of P(y|x) Entropy of P(y) Inception Score = 1 ෍ | | class 1 class 2 class 3 Formally: Peculiarity Diversity
  32. ❖ Multi-scale structural similarity (MS-SSIM): evaluate the diversity of the

    generated samples ❖ Fré chet Inception Distance (FID) a new metric to evaluate the performance of GANs Other Evaluation Metrics
  33. ➢Generative modeling ➢Generative adversarial networks (GANs) ➢Conditional GANs (CGANs) ➢Training

    procedure and evaluation methods ➢Common issues and some tips ➢Some Applications Contents
  34. Common Issues in GANs • Non-convergence: although GAN training is

    theoretically guaranteed to converge, the model often break down during the training • Oscillation: GAN can be trained for a very long time without clearly generating better samples • Training imbalance: discriminator becomes too strong too quickly and the generator ends up not learning anything • Mode collapse: almost all the samples look the same (very low diversity) • Visual problems: counting, perspective, global structure, etc.
  35. Cross-entropy (CE) loss: • Default discriminator cost: • One-sided label

    smoothed cost (Salimans et al. 2016) : label : probability Benefits • Prevents discriminator from giving very large gradient signal to generator • Stabilize the training and prevents extrapolating to encourage extreme samples One-Sided Label Smoothing: Some Solutions
  36. • A modified loss function • DCGAN model design •

    Track failures early (D loss goes to 0: failure mode) • If you have labels, use them • Add noise to inputs, decay over time Tricks for GANs . . . . . .
  37. ➢Generative modeling ➢Generative adversarial networks (GANs) ➢Conditional GANs (CGANs) ➢Training

    procedure and evaluation methods ➢Common issues and some tips ➢Some Applications Contents
  38. Conclusions • GANs are generative models that are implemented using

    two stochastic neural network modules: Generator and Discriminator. • Generator tries to generate samples from random noise as input. • Discriminator tries to distinguish the samples from Generator and samples from the real data distribution. • Both players are trained adversarially to fool each other, and they become better at their respective tasks.