M5-106M5: NLP & Large Language ModelsHallucination & AI SafetyMedium
⚡Flashcard Drill➔Mastery:
Uncertainty Quantification and Probability Calibration for Hallucination Detection
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Calibrates model confidence to reflect empirical accuracy (ECE), leveraging semantic entropy, self-consistency divergence, and logit distributions to identify high-risk hallucinated responses.
📌 Key Takeaways
- •Calibration definition: a model is calibrated when its predicted probability matches empirical ground truth accuracy ()
- •Systemic overconfidence: standard LLMs exhibit severe overconfidence on incorrect assertions due to MLE training and uncalibrated softmax logits
- •Detection metrics: Expected Calibration Error (ECE), Semantic Entropy (clustering paraphrased responses), and internal hidden-state probing
📐 Mathematical Derivations
Mathematical Mechanism: 1. Expected Calibration Error (ECE): Partition predictions into confidence bins : 2. Semantic Entropy (Kuhn et al. 2023): Sample candidate completions . Cluster completions into discrete semantic equivalence classes using an NLI bidirectional entailment model (): High semantic entropy indicates true epistemic uncertainty (hallucination risk), filtering out benign syntactic paraphrasing.
🏭 Production Trade-offs
Deep Dive & Engineering Trade-offs: ① Token Entropy vs Semantic Entropy: Raw token-level perplexity or logit entropy fails to measure uncertainty because natural language has many valid paraphrases (high token diversity, identical meaning). Semantic entropy clusters paraphrases first; if all 10 sampled trajectories express the same core fact in different words, semantic entropy is 0 (high confidence), correctly avoiding false-positive hallucination alarms. ② Temperature Scaling for Logit Calibration: Softmax probabilities are uncalibrated out-of-the-box. Applying a single post-hoc scalar parameter : optimized via negative log-likelihood on a validation set softens overconfident probability distributions without altering top-1 rankings. ③ Self-Assessment Prompting ('P(True)'): Prompting the model: 'Is the above statement true? Reply ONLY with a probability from 0.0 to 1.0' can provide rough calibration, but models often suffer from meta-cognitive overconfidence. Combining verbalized confidence with semantic entropy yields the highest AUROC for hallucination detection. ④ Operationalizing Uncertainty in Production: Implement a three-way routing gate: - Low Entropy: stream directly to user. - Medium Entropy: trigger real-time web/RAG search to cross-verify claims. - High Entropy: fallback to conservative refusal or route to human agent review. ⑤ Interview Strategy: Define ECE, derive the Semantic Entropy formula and explain why bidirectional NLI clustering is critical, contrast token entropy with semantic entropy, and present the three-tier uncertainty routing architecture.
⚠️ Common Interview Pitfalls
- ✕Using raw token-level generation perplexity as a proxy for uncertainty without grouping semantic paraphrases
- ✕Trusting verbalized model confidence ('I am 99% certain') without prior post-hoc temperature calibration
- ✕Applying binary thresholding that rejects useful answers with moderate linguistic diversity
🎯 Interviewer Follow-up Inquiries
- ?Why is Semantic Entropy substantially more predictive of factual accuracy than sequence-level token perplexity?
- ?How does post-hoc temperature scaling mathematically minimize Expected Calibration Error (ECE) without altering top-1 prediction rank?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚