M2-013HardAI 功能评估 · AI feature evaluation
Search 上线了 AI 摘要后,自然结果的点击率下降了 15%。这是好事还是坏事?
After AI summaries launched on Search, organic result CTR fell 15%. Good or bad?
💡 考场高分一句话答案:
不能单看点击率。要看用户任务是否更快完成、是否减少重搜、以及内容生态(创作者流量)是否受损——三者可能方向相反。
CTR alone cannot tell you. Look at whether the task completes faster, whether re-queries fall, and whether the creator ecosystem is being damaged — these can move in opposite directions.
📌 核心考察要点 (Key Points)
- 用户侧:refinement rate 是否下降、session 是否更短且不再回来、满意度是否上升
- 生态侧:外链站点流量下降 → 创作者供给减少 → 长期内容质量下降(滞后 6–18 个月)
- 商业侧:广告曝光是否被挤出、RPM 是否下降
- 判据:如果用户任务完成更快且满意度上升,但创作者流量显著下降,这是价值转移而非价值创造,需要补偿机制
- User side: does refinement rate fall, are sessions shorter but followed by return, does satisfaction rise?
- Ecosystem side: external site traffic falls → creator supply falls → content quality declines with a six-to-eighteen-month lag
- Commercial side: are ad impressions being displaced, is RPM falling?
- Test: if the task completes faster and satisfaction rises but creator traffic falls sharply, that is value transfer rather than value creation, and needs a compensation mechanism
📐 第一性原理与数理推导机理
这是一个典型的多目标冲突问题,适合用约束优化表述:$$\max\ \text{TaskSuccess}\quad \text{s.t.}\ \text{CreatorTraffic}\ge \tau,\ \text{RPM}\ge \rho$$其中 $\tau,\rho$ 是由长期生态健康决定的约束下界。没有约束的单一目标优化一定会牺牲生态。
⚖️ 业务权衡与取舍 (Trade-offs & Calibration)
这类问题的正确答案往往是“分查询类型看”:事实型查询的 AI 摘要可能是净增益(用户不用点开多个链接),而需要深度阅读的查询(评测、教程)AI 摘要可能损害体验与生态。因此上线策略应是查询类型白名单而非全局开关。
⚠️ 常见高频面试陷阱与避坑指南
- 把点击率下降直接等同于体验变差(也可能意味着用户不用点了)
- 忽略创作者生态的滞后反馈
🔍 大厂面试官连环追问 (Follow-ups)
- 如何量化“创作者流量下降”的长期代价?
- 如果 AI 摘要提升了留存但降低了广告收入,你怎么决策?
嫌字多看太累?进入苏格拉底式对话推演实验室
零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.