generate new samples that are as similar as data and fool the discriminator — Discriminative model D: act as a judge to distinguish the generated samples from the real one adversarial Generator Discriminator Vs
generated samples from the real data • (): estimate the probability that is a real data sample — = 1: regards as a real sample — = 0: regards as a fake sample Discriminative Model Binary classification
• A mini-batch of training samples • A mini-batch of generated samples • Optional: Run steps of one player for every step of the other player - example: WGAN updates Discriminator for 5 steps and Generator for 1 step.
theoretically guaranteed to converge, the model often break down during the training • Oscillation: GAN can be trained for a very long time without clearly generating better samples • Training imbalance: discriminator becomes too strong too quickly and the generator ends up not learning anything • Mode collapse: almost all the samples look the same (very low diversity) • Visual problems: counting, perspective, global structure, etc.
smoothed cost (Salimans et al. 2016) : label : probability Benefits • Prevents discriminator from giving very large gradient signal to generator • Stabilize the training and prevents extrapolating to encourage extreme samples One-Sided Label Smoothing: Some Solutions
two stochastic neural network modules: Generator and Discriminator. • Generator tries to generate samples from random noise as input. • Discriminator tries to distinguish the samples from Generator and samples from the real data distribution. • Both players are trained adversarially to fool each other, and they become better at their respective tasks.