← All papers

Generative Adversarial Nets

Goodfellow, Pouget-Abadie, Mirza et al. · 2014 · NeurIPS 2014

GenerationRead on arXiv

Introduced the adversarial training framework where a generator and discriminator compete in a minimax game. GANs created an entirely new paradigm for generative modeling and dominated image synthesis for half a decade.

Key Idea

Train two neural networks in competition: a Generator G that produces fake samples, and a Discriminator D that tries to distinguish real data from fakes. The generator learns to produce increasingly realistic outputs to fool the discriminator, while the discriminator gets better at detecting fakes.

The Minimax Game

  • Objective: min_G max_D [E[log D(x)] + E[log(1 - D(G(z)))]]
  • The discriminator maximizes its ability to classify real vs fake
  • The generator minimizes the discriminator's ability to detect fakes
  • At equilibrium, the generator produces samples from the true data distribution, and the discriminator outputs 0.5 for all inputs

Training Dynamics

  • G takes random noise z ~ N(0,1) and maps it to data space
  • D takes a sample (real or generated) and outputs a probability of being real
  • Alternate training: update D for k steps, then update G for 1 step
  • Training is notoriously unstable, mode collapse, vanishing gradients, and oscillation are common

Why It Matters

  • Created an entirely new generative paradigm, learning without explicit density estimation
  • Dominated image synthesis from 2014-2021 (StyleGAN, BigGAN, CycleGAN, Pix2Pix)
  • Applications: image generation, super-resolution, style transfer, data augmentation, domain adaptation
  • Eventually superseded by diffusion models for image generation, but the adversarial principle lives on in training techniques (adversarial training, discriminator-based losses)

Key Takeaways for Interviews

  • GANs learn through adversarial competition, not maximum likelihood estimation
  • Mode collapse (generator produces limited variety) is the signature failure mode
  • Wasserstein GAN (WGAN) improved training stability by using Wasserstein distance instead of JS divergence
  • GANs are largely superseded by diffusion models for image generation, but the adversarial principle remains influential