🧠 Deep LearningID: label-smoothing

Label Smoothing

Q: Write the label-smoothing formula $y' = y(1-\epsilon) + \frac{\epsilon}{K}$, explain the roles of $\epsilon$ and $K$, and state the usual value of $\epsilon$.

【Label Smoothing 核心考点 1】Write the label-smoothing formula $y' = y(1-\epsilon) + \frac{\epsilon}{K}$, explain the roles of $\epsilon$ and $K$, and state the usual value of $\epsilon$.。技术解析：Definition: label smoothing mixes the one-hot label with a uniform distribution: y' = y(1-\epsilon) + \frac{\epsilon}{K}, where K is the number of classes and \epsilon is typically 0.1; the true class gets 1-\epsilon+\frac{\epsilon}{K} and every other class gets \frac{\epsilon}{K... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/label-smoothing

Q: Why does label smoothing prevent overconfidence? How does it affect logit magnitudes and model calibration?

【Label Smoothing 核心考点 2】Why does label smoothing prevent overconfidence? How does it affect logit magnitudes and model calibration?。技术解析：Definition: label smoothing mixes the one-hot label with a uniform distribution: y' = y(1-\epsilon) + \frac{\epsilon}{K}, where K is the number of classes and \epsilon is typically 0.1; the true class gets 1-\epsilon+\frac{\epsilon}{K} and every other class gets \frac{\epsilon}{K... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/label-smoothing

Q: Why does label smoothing hurt knowledge distillation? What information in the teacher's output does it destroy?

【Label Smoothing 核心考点 3】Why does label smoothing hurt knowledge distillation? What information in the teacher's output does it destroy?。技术解析：Definition: label smoothing mixes the one-hot label with a uniform distribution: y' = y(1-\epsilon) + \frac{\epsilon}{K}, where K is the number of classes and \epsilon is typically 0.1; the true class gets 1-\epsilon+\frac{\epsilon}{K} and every other class gets \frac{\epsilon}{K... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/label-smoothing

Q: What regularization term is label smoothing equivalent to adding to cross-entropy? Write the corresponding loss form.

【Label Smoothing 核心考点 4】What regularization term is label smoothing equivalent to adding to cross-entropy? Write the corresponding loss form.。技术解析：Definition: label smoothing mixes the one-hot label with a uniform distribution: y' = y(1-\epsilon) + \frac{\epsilon}{K}, where K is the number of classes and \epsilon is typically 0.1; the true class gets 1-\epsilon+\frac{\epsilon}{K} and every other class gets \frac{\epsilon}{K... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/label-smoothing

Q: In which scenarios should label smoothing be avoided? How does it interact with temperature scaling?

【Label Smoothing 核心考点 5】In which scenarios should label smoothing be avoided? How does it interact with temperature scaling?。技术解析：Definition: label smoothing mixes the one-hot label with a uniform distribution: y' = y(1-\epsilon) + \frac{\epsilon}{K}, where K is the number of classes and \epsilon is typically 0.1; the true class gets 1-\epsilon+\frac{\epsilon}{K} and every other class gets \frac{\epsilon}{K... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/label-smoothing

🎯Core Definition

label smoothing mixes the one-hot label with a uniform distribution:

y' = y(1-\epsilon) + \frac{\epsilon}{K}

, where

K

is the number of classes and

\epsilon

is typically 0.1; the true class gets

1-\epsilon+\frac{\epsilon}{K}

and every other class gets

\frac{\epsilon}{K}

. It is equivalent to adding a KL penalty toward the uniform distribution to the cross-entropy loss.

💡Use Cases

standard for ImageNet-style classification (Inception / ResNet v2 families), Transformer machine translation, and LLM pretraining/finetuning to suppress overconfidence and improve calibration. Interviews ask for the formula, why it prevents overfitting, and its side effect on distillation.

⚡Key Problems Solved

it prevents extreme logits (confidence saturating to 1) on one-hot labels, improving generalization and calibration. Classic side effect (Müller et al. 2019, 'When Does Label Smoothing Help?'): smoothing flattens the relative logit information ('dark knowledge') of the teacher, so student distillation gains drop noticeably — when distillation is planned, the teacher is usually trained without label smoothing.

🎯5 High-Frequency Exam Points

Write the label-smoothing formula

y' = y(1-\epsilon) + \frac{\epsilon}{K}

, explain the roles of

\epsilon

and

K

, and state the usual value of

\epsilon

Why does label smoothing prevent overconfidence? How does it affect logit magnitudes and model calibration?

Why does label smoothing hurt knowledge distillation? What information in the teacher's output does it destroy?

What regularization term is label smoothing equivalent to adding to cross-entropy? Write the corresponding loss form.

In which scenarios should label smoothing be avoided? How does it interact with temperature scaling?

📖 In-depth Guide:📄 normalization-and-regularization →

Updated 2026-08-12

🎯

Test Your Knowledge: Practice Questions for "Label Smoothing"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardDropout Next Card →Weight Decay & AdamW

🔗 More Deep Learning Knowledge Cards

Activation Functions Adam & AdamW Autograd Compute Graph Batch Normalization