E3Medium★ Core Essential · Industrial BedrockPart E · Loss Functions Handbook
Focal Loss for Dense Object Detection
Down-weight easy background examples with (1 - pt)^gamma to address severe class imbalance.
⏱️ Time:
O(N)💾 Space:
O(N)🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def focal_loss_binary(
logits: np.ndarray, # 未激活的 logit 预测
targets: np.ndarray, # 0 或 1
alpha: float = 0.25,
gamma: float = 2.0,
reduction: str = "mean"
) -> float:
# 1. 稳定计算 sigmoid 概率
probs =
# 2. 计算对应目标类别的概率 pt 与类别权重 alpha_t
pt = np.where(targets == 1, probs, 1.0 - probs)
alpha_t = np.where(targets == 1, alpha, 1.0 - alpha)
# 3. 聚焦因子 (1 - pt)^gamma
focal_weight = alpha_t * ((1.0 - pt) ** gamma)
# 4. 交叉熵核心 (添加 1e-12 避免 log(0))
ce_loss = -np.log(np.clip(pt, 1e-12, 1.0))
loss = focal_weight * ce_loss
if reduction == "mean":
return
return loss
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: What does Focal Loss reduce to when gamma = 0?
💡
Core Mental Anchor / Mnemonic
Multiply cross-entropy by (1 - pt)^gamma so confident easy negatives contribute near-zero loss.
📐 Mathematical Derivation & Core Formula
### Easy-Example Down-Weighting & Hard Mining
In dense object detection (e.g., RetinaNet), background negatives can outnumber positives by 1000:1, overwhelming positive gradients even when individual easy-negative losses are tiny.
Focal Loss adds a modulating factor (typically ):
- Easy example ($p_t = 0.99$): , suppressing easy loss by ;
- Hard example ($p_t = 0.2$): , preserving training signal;
Combined with class-balancing weight , it focuses training on hard misclassified regions.
In dense object detection (e.g., RetinaNet), background negatives can outnumber positives by 1000:1, overwhelming positive gradients even when individual easy-negative losses are tiny.
Focal Loss adds a modulating factor (typically ):
- Easy example ($p_t = 0.99$): , suppressing easy loss by ;
- Hard example ($p_t = 0.2$): , preserving training signal;
Combined with class-balancing weight , it focuses training on hard misclassified regions.
🔄 Tensor Dimensions & Shape Flow
logits, targets -> probs -> pt -> (1 - pt)^gamma factor * BCE -> weighted scalar loss
🛡️ Numerical Stability & Pitfalls Checklist
- Clamp log(pt) inputs away from zero (e.g., eps=1e-12) to avoid -inf.
- Initialize classifier final bias to
-log((1 - pi) / pi)(e.g., pi=0.01) to prevent early positive-gradient shock.
💻 Reference Code Implementation
import numpy as np
def focal_loss_binary(
logits: np.ndarray, # 未激活的 logit 预测
targets: np.ndarray, # 0 或 1
alpha: float = 0.25,
gamma: float = 2.0,
reduction: str = "mean"
) -> float:
# 1. 稳定计算 sigmoid 概率
probs = 1.0 / (1.0 + np.exp(-np.clip(logits, -88.0, 88.0)))
# 2. 计算对应目标类别的概率 pt 与类别权重 alpha_t
pt = np.where(targets == 1, probs, 1.0 - probs)
alpha_t = np.where(targets == 1, alpha, 1.0 - alpha)
# 3. 聚焦因子 (1 - pt)^gamma
focal_weight = alpha_t * ((1.0 - pt) ** gamma)
# 4. 交叉熵核心 (添加 1e-12 避免 log(0))
ce_loss = -np.log(np.clip(pt, 1e-12, 1.0))
loss = focal_weight * ce_loss
if reduction == "mean":
return float(np.mean(loss))
return loss
🧪 Runnable Assertions & Validation
Copy and run directly in Python / Jupyter:
import numpy as np
logits = np.array([5.0, 0.0, -5.0])
targets = np.array([1, 1, 1])
fl = focal_loss_binary(logits, targets, alpha=0.25, gamma=2.0)
# 容易样本 (logit=5, target=1) 的损失应被压缩至极小
fl_easy = focal_loss_binary(np.array([5.0]), np.array([1]))
fl_hard = focal_loss_binary(np.array([-5.0]), np.array([1]))
assert fl_easy < fl_hard * 0.01, "简单样本未能被有效压制"
print("✓ Focal Loss 困难样本聚焦自测通过")🎯 Core Follow-up Inquiries & Reference Answers
Q1:What does Focal Loss reduce to when gamma = 0?
When , the modulating factor , so Focal Loss reduces directly to -weighted binary cross-entropy loss.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.