E3Medium★ Core Essential · Industrial BedrockPart E · Loss Functions Handbook

Focal Loss for Dense Object Detection

⚡Flashcard Drill➔

Down-weight easy background examples with (1 - pt)^gamma to address severe class imbalance.

⏱️ Time: O(N)
💾 Space: O(N)
🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def focal_loss_binary(
logits: np.ndarray, # 未激活的 logit 预测
targets: np.ndarray, # 0 或 1
alpha: float = 0.25,
gamma: float = 2.0,
reduction: str = "mean"
) -> float:
# 1. 稳定计算 sigmoid 概率
probs =
# 2. 计算对应目标类别的概率 pt 与类别权重 alpha_t
pt = np.where(targets == 1, probs, 1.0 - probs)
alpha_t = np.where(targets == 1, alpha, 1.0 - alpha)
# 3. 聚焦因子 (1 - pt)^gamma
focal_weight = alpha_t * ((1.0 - pt) ** gamma)
# 4. 交叉熵核心 (添加 1e-12 避免 log(0))
ce_loss = -np.log(np.clip(pt, 1e-12, 1.0))
loss = focal_weight * ce_loss
if reduction == "mean":
return
return loss
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: What does Focal Loss reduce to when gamma = 0?
Next: InfoNCE Contrastive Loss →
💡

Core Mental Anchor / Mnemonic

Multiply cross-entropy by (1 - pt)^gamma so confident easy negatives contribute near-zero loss.

📐 Mathematical Derivation & Core Formula

FL(pt)=−αt(1−pt)γlog⁡(pt)\mathrm{FL}(p_t) = -\alpha_t (1 - p_t)^\gamma \log(p_t)
### Easy-Example Down-Weighting & Hard Mining
In dense object detection (e.g., RetinaNet), background negatives can outnumber positives by 1000:1, overwhelming positive gradients even when individual easy-negative losses are tiny.
Focal Loss adds a modulating factor (1−pt)γ(1 - p_t)^\gamma (typically γ=2.0\gamma = 2.0):
- Easy example ($p_t = 0.99$): (1−0.99)2=10−4(1 - 0.99)^2 = 10^{-4}, suppressing easy loss by 10,000×10{,}000\times;
- Hard example ($p_t = 0.2$): (1−0.2)2=0.64(1 - 0.2)^2 = 0.64, preserving training signal;
Combined with class-balancing weight αt\alpha_t, it focuses training on hard misclassified regions.

🔄 Tensor Dimensions & Shape Flow

logits, targets -> probs -> pt -> (1 - pt)^gamma factor * BCE -> weighted scalar loss

🛡️ Numerical Stability & Pitfalls Checklist

  • Clamp log(pt) inputs away from zero (e.g., eps=1e-12) to avoid -inf.
  • Initialize classifier final bias to -log((1 - pi) / pi) (e.g., pi=0.01) to prevent early positive-gradient shock.

💻 Reference Code Implementation

import numpy as np

def focal_loss_binary(
    logits: np.ndarray,      # 未激活的 logit 预测
    targets: np.ndarray,     # 0 或 1
    alpha: float = 0.25,
    gamma: float = 2.0,
    reduction: str = "mean"
) -> float:
    # 1. 稳定计算 sigmoid 概率
    probs = 1.0 / (1.0 + np.exp(-np.clip(logits, -88.0, 88.0)))
    
    # 2. 计算对应目标类别的概率 pt 与类别权重 alpha_t
    pt = np.where(targets == 1, probs, 1.0 - probs)
    alpha_t = np.where(targets == 1, alpha, 1.0 - alpha)
    
    # 3. 聚焦因子 (1 - pt)^gamma
    focal_weight = alpha_t * ((1.0 - pt) ** gamma)
    
    # 4. 交叉熵核心 (添加 1e-12 避免 log(0))
    ce_loss = -np.log(np.clip(pt, 1e-12, 1.0))
    loss = focal_weight * ce_loss
    
    if reduction == "mean":
        return float(np.mean(loss))
    return loss

🧪 Runnable Assertions & Validation

Copy and run directly in Python / Jupyter:

import numpy as np
logits = np.array([5.0, 0.0, -5.0])
targets = np.array([1, 1, 1])
fl = focal_loss_binary(logits, targets, alpha=0.25, gamma=2.0)
# 容易样本 (logit=5, target=1) 的损失应被压缩至极小
fl_easy = focal_loss_binary(np.array([5.0]), np.array([1]))
fl_hard = focal_loss_binary(np.array([-5.0]), np.array([1]))
assert fl_easy < fl_hard * 0.01, "简单样本未能被有效压制"
print("✓ Focal Loss 困难样本聚焦自测通过")

🎯 Core Follow-up Inquiries & Reference Answers

Q1:What does Focal Loss reduce to when gamma = 0?
When γ=0\gamma = 0, the modulating factor (1−pt)0=1(1 - p_t)^0 = 1, so Focal Loss reduces directly to α\alpha-weighted binary cross-entropy loss.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
← Prev: Binary Cross-Entropy with LogitsBack to Part E & All 105Next: InfoNCE Contrastive Loss →