← Back to Deep Learning Mind Map
🧠 Deep LearningID: wgan

WGAN & Wasserstein

WGAN 与 Wasserstein
🎯Core Definition
WGAN replaces the JS divergence with the Wasserstein distance (earth-mover, via the Kantorovich-Rubinstein duality): W(Pr,Pg)=sup⁡∥f∥L≤1Ex∼Pr[f(x)]−Ez∼pz[f(G(z))]W(P_r, P_g) = \sup_{\Vert f \Vert_L \le 1} \mathbb{E}_{x \sim P_r}[f(x)] - \mathbb{E}_{z \sim p_z}[f(G(z))], requiring the critic ff to be 1-Lipschitz. The original WGAN enforces this via weight clipping, which distorts the distribution; WGAN-GP instead penalizes the gradient norm on linear interpolations x^\hat{x} between real and fake samples: λEx^∼Px^[(∥∇x^D(x^)∥2−1)2]\lambda \mathbb{E}_{\hat{x} \sim P_{\hat{x}}}\left[(\Vert \nabla_{\hat{x}} D(\hat{x}) \Vert_2 - 1)^2\right] (default λ=10\lambda = 10).
💡Use Cases
replacing standard GANs for stable image generation (e.g. DCGAN backbone + WGAN-GP loss); interviews compare Wasserstein vs JSD/KL and ask how the 1-Lipschitz constraint is implemented.
⚡Key Problems Solved
JSD saturates at log⁡2\log 2 and its gradient vanishes almost everywhere when the supports of real and generated distributions do not overlap (common early in training), starving the generator; the Wasserstein distance is continuous and differentiable everywhere and gives meaningful gradients even for disjoint supports, stabilizing training and yielding an unsaturated loss usable as a convergence signal. Cost: the 1-Lipschitz constraint must be maintained — weight clipping causes exploding/vanishing gradients, so gradient penalty is the robust fix.
🎯Core Interview Exam Points
Fundamentals · Production Scenarios · Deep Dive
Core Def
Explain the core definition, underlying mechanism/math, and key principles of WGAN & Wasserstein.
Use Cases
In production engineering, what are the primary use cases for WGAN & Wasserstein and what core pain points does it solve?
🔥 High-Frequency Deep-Dive Questions
1
State the definition of Wasserstein distance and its Kantorovich-Rubinstein dual form?
2
Why do JSD/KL gradients vanish for non-overlapping supports while Wasserstein does not?
3
How is the 1-Lipschitz constraint enforced? Trade-offs of weight clipping vs gradient penalty?
4
Why is the WGAN-GP penalty minimized when ∥∇x^D(x^)∥2=1\Vert \nabla_{\hat{x}} D(\hat{x}) \Vert_2 = 1?
5
Why does WGAN's critic omit sigmoid, and what does its loss value mean?
Updated 2026-08-12
🎯
Verify Mastery: Practice Exam Questions for "WGAN & Wasserstein"
Take pitfall multiple-choice questions with derivation options; mistakes are automatically logged.
🚀 Start Card Practice ➔
← Previous CardGAN Adversarial TrainingNext Card →VAE & Reparameterization

🔗 More Deep Learning Knowledge Cards

引用出处: TalentMe 450+ 工业级 AI 架构考点知识库 (Tech Vault) · 遵循 CC-BY-4.0 知识共享协议
URL: https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/wgan
中文速览与面试要点

WGAN 与 Wasserstein (WGAN & Wasserstein)

定义: WGAN 用 Wasserstein 距离 (推土机距离, 经 Kantorovich-Rubinstein 对偶) 度量两个分布: W(P_r, P_g) = _ f _L 1 E_x P_r[f(x)] - E_z p_z[f(G(z))], 要求判别器 (critic) f 满足 1-Lipschitz 约束。原始 WGAN 用权重裁剪实现 Lipschitz, 但裁剪易破坏分布; WGAN-GP 改为在真实与生成样本的线性插值点 x 上施加梯度惩罚 E_ x P_ x [( _ x D( x) _2 - 1)^2 ] (默认 = 10)。 使用场景: 图像生成中替代标准 GAN 以获得稳定训练 (DCGAN 骨架 + WGAN-GP 损失); 面试必问 Wasserstein 与 JSD/KL 的对比、1-Lipschitz 约束的实现方式。 解决的问题: 标准 GAN 的 JSD 在真实与生成分布支撑集不重叠 (训练早期常见) 时趋近常数 2, 梯度几乎处处为 0, 生成器收不到信号; Wasserstein 距离处处连续可导, 即使分布不重叠也提供有意义的梯度, 训练更稳定, 且 loss 不再饱和、可作收敛指标。代价: 必须维护 1-Lipschitz 约束 — 权重裁剪会致梯度爆炸/消失, 梯度惩罚是更稳健的实现。

核心要点

  • 写出 Wasserstein 距离的定义与 Kantorovich-Rubinstein 对偶形式?
  • 为什么 JSD/KL 在分布不重叠时梯度消失, 而 Wasserstein 距离不会?
  • 1-Lipschitz 约束如何实现? 权重裁剪与梯度惩罚各有什么问题?
  • WGAN-GP 梯度惩罚项为什么在 _ x D( x) _2 = 1 时最优?
  • 为什么 WGAN 的 critic 输出不加 sigmoid? WGAN 的 loss 数值有什么含义?