M7-009MediumLLM 评估 · LLM evaluation

如何度量 LLM 的“幻觉率”?

How do you measure an LLM's hallucination rate?
💡 考场高分一句话答案:

分类型度量:事实错误(可核查)、无据推断(上下文不支持)、引用错误(引用不存在或不支持论断),分别用不同方法。

Measure by type: factual errors, unsupported inference, and citation errors, each with its own method and reported with harm weighting rather than one average.

📌 核心考察要点 (Key Points)

  • 事实错误:与知识库/检索结果比对,或用独立模型核查(需人工抽检校准)
  • 无据推断:faithfulness——每个论断是否被上下文支持(NLI 模型或 judge)
  • 引用错误:检查 URL 是否存在、内容是否支持该论断
  • 报告方式:按危害加权(医疗/金融类错误权重更高),而不是单一平均率
  • Factual errors: compare against a knowledge base or retrieved context, with human spot checks to calibrate
  • Unsupported inference: faithfulness — whether each claim is entailed by the context
  • Citation errors: does the URL exist and does it support the claim
  • Reporting: weight by harm (medical, financial), and report 'should-answer-but-did-not' separately from 'answered wrongly'

⚖️ 业务权衡与取舍 (Trade-offs & Calibration)

幻觉率的分母很关键:把“我不知道”算成错误会激励模型乱答;不算又会低估。稳妥做法是分别报告“该答未答率”与“答错率”,并明确“拒答”是允许的行为。这与分类器的 precision/recall 权衡是同一个结构。

⚠️ 常见高频面试陷阱与避坑指南

  • 用一个笼统的“幻觉率”掩盖类型差异
  • 分母包含本就没有答案的问题

🔍 大厂面试官连环追问 (Follow-ups)

  • 如何自动检测引用是否支持论断?
  • 拒答率与幻觉率如何权衡?

嫌字多看太累?进入苏格拉底式对话推演实验室

零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。

💬 开启对话式破冰 ➔
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔