M3-011Medium长期效应 · Long-term effects
什么是 novelty effect?如何检测和校正?
What is the novelty effect and how do you detect and correct for it?
💡 考场高分一句话答案:
新事物带来的短期兴奋导致效应虚高;通过画效应-时间曲线、对比新老用户、或延长观察窗来检测。
Newness produces an inflated short-term effect. Detect it by plotting effect against day, comparing new versus existing users, or extending the observation window.
📌 核心考察要点 (Key Points)
- 检测:效应随天数的曲线;若前 3 天高、之后回落至接近 0,即为 novelty
- 新老用户对比:老用户 novelty 更弱,若效应只在新用户上出现,说明是新奇感
- 校正:用后期(如第 3–4 周)的稳态效应作决策依据,而非全期平均
- 极端情况:用 holdout 观察 3–6 个月
- Detection: the effect is high for the first three days and decays towards zero afterwards
- New versus existing: existing users show less novelty; if the effect appears only among new users it is probably novelty
- Correction: base the decision on the steady-state effect from the later weeks rather than the full-period average
- Extreme case: use a holdout and observe for three to six months
⚖️ 业务权衡与取舍 (Trade-offs & Calibration)
novelty 的反面是 primacy effect:用户不习惯新设计,短期效应为负,随熟练度提升转正。两者都表现为“效应随时间变化”,但方向相反。判断方法是看曲线形状:novelty 是“高开低走”,primacy 是“低开高走”。
⚠️ 常见高频面试陷阱与避坑指南
- 用全期平均效应做决策,掩盖了时间趋势
- 看到效应下降就断定是 novelty,不排除外部事件
🔍 大厂面试官连环追问 (Follow-ups)
- 如何区分 novelty 与“早期采用者效应”?
- 如果效应持续上升,你会延长实验吗?
嫌字多看太累?进入苏格拉底式对话推演实验室
零打字负担,以大厂考官真实困境切入,通过点选直觉选项暴露思维盲区,手把手推导 Why。
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.