M2-082M2: Classical Machine LearningModel CalibrationMedium
⚡Flashcard Drill➔Mastery:

Comparing Platt Scaling and Isotonic Regression for Probability Calibration

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-083 →
📐 Mathematical Definition
Platt: p^′=σ(ap^+b)\text{Platt}:\ \hat p'=\sigma(a\hat p+b)
⚡ One-Sentence Core Answer
Platt scaling fits a parametric sigmoid to model logits; Isotonic regression fits a non-parametric piecewise-constant isotonic step function.

📌 Key Takeaways

  • •
    Platt scaling: parametric logistic regression 1/(1+exp⁡(Af(x)+B))1 / (1 + \exp(A f(x) + B)), robust on small datasets
  • •
    Isotonic regression: non-parametric monotonic step function via Pool Adjacent Violators Algorithm (PAVA)
  • •
    Risk trade-off: Platt scaling assumes sigmoidal calibration curve; Isotonic overfits small sample sizes

📐 Mathematical Derivations

Methodology: ① Platt Scaling: Fits a logistic transformation on uncalibrated model scores f(x)f(x): P^(y=1∣f)=11+exp⁡(Af(x)+B)\hat{P}(y=1|f) = \frac{1}{1 + \exp(A f(x) + B)}, where parameters A,BA, B are found via maximum likelihood on a held-out calibration set. To prevent overfitting, targets are regularized: y+=N++1N++2,y−=1N−+2y_+ = \frac{N_+ + 1}{N_+ + 2}, y_- = \frac{1}{N_- + 2}. ② Isotonic Regression: Minimizes squared error subject to monotonic order constraints: min⁡m∑i=1N(yi−m(fi))2s.t.m(fi)≤m(fj) whenever fi≤fj\min_m \sum_{i=1}^N (y_i - m(f_i))^2 \quad \text{s.t.} \quad m(f_i) \le m(f_j) \text{ whenever } f_i \le f_j. Solved exactly in O(N)O(N) using the Pool Adjacent Violators Algorithm (PAVA).

🏭 Production Trade-offs

Selection criteria: ① Sample Size: When calibration data is small (N<1000N < 1000), use Platt scaling to avoid step-function overfitting. When calibration data is abundant (N>5000N > 5000), Isotonic regression can correct arbitrary monotonic distortions. ② Shape of Distortion: SVM margins typically produce sigmoidal distortion, fitting Platt scaling naturally. Non-sigmoidal distortions (e.g., naive Bayes over-pushing probabilities to 0 and 1) are better corrected by Isotonic regression.
⚠️ Common Interview Pitfalls
  • ✕
    Fitting calibration models on the training dataset rather than an independent validation set
  • ✕
    Using Isotonic regression on very small validation sets, resulting in staircase artifacts and poor generalization
🎯 Interviewer Follow-up Inquiries
  • ?
    How does the Pool Adjacent Violators Algorithm (PAVA) solve isotonic regression in linear time?
  • ?
    Why does Platt scaling apply smoothed target labels rather than hard 0/1 labels during calibration?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-081: Defining Model Calibration: Why High Accuracy Does Not Imply Good Calibration📋 Back to BankNext →M2-083: Expected Calibration Error (ECE): Mathematical Formulation and Practical Limitations