← Back to System Design Mind Map
🏗️ System DesignID: imbalanced-cost-sensitive-learning

Imbalanced & Cost-Sensitive Learning

极端不平衡样本与代价敏感学习
🎯Core Definition
Imbalanced Classification & Cost-Sensitive Machine Learning addresses the extreme sample skew (fraud positive prevalence <0.01%∼0.1%<0.01\% \sim 0.1\%, ratios exceeding 1:10,0001:10,000) and highly asymmetric misclassification costs in financial risk modeling; it operates across 3 synergistic tiers: 1) Data Resampling (SMOTE oversampling and EasyEnsemble/BalanceCascade bagged negative downsampling); 2) Loss Re-weighting via Focal Loss FL(pt)=−αt(1−pt)γln⁡(pt)\text{FL}(p_t) = -\alpha_t (1 - p_t)^\gamma \ln(p_t) (dynamically suppressing gradients from easy legitimate negatives to focus capacity on hard fraud cases); 3) Cost-Sensitive Decision Matrices that explicitly quantify the asymmetric business penalties of False Positives (user friction/churn) vs False Negatives (unrecoverable financial chargebacks) to calibrate optimal threshold cutoffs.
💡Use Cases
Payment chargeback prevention, AML transaction screening, loan default risk prediction, and malicious bot mitigation.
⚡Key Problems Solved
Standard binary cross-entropy collapses into predicting all transactions as legitimate (yielding 99.99% dummy accuracy while leaking all fraud); cost-sensitive learning minimizes monetary loss while maintaining strict false-positive caps.
🎯Core Interview Exam Points
Fundamentals · Production Scenarios · Deep Dive
Core Def
Explain the core definition, underlying mechanism/math, and key principles of Imbalanced & Cost-Sensitive Learning.
Use Cases
In production engineering, what are the primary use cases for Imbalanced & Cost-Sensitive Learning and what core pain points does it solve?
🔥 High-Frequency Deep-Dive Questions
1
Derive Focal Loss FL(pt)=−αt(1−pt)γln⁡(pt)\text{FL}(p_t) = -\alpha_t (1 - p_t)^\gamma \ln(p_t) and explain how the focusing parameter γ\gamma downweights easy examples?
2
Compare EasyEnsemble vs BalanceCascade in training multiple bagging learners over partitioned negative sample majorities?
3
Why is Accuracy prohibited in fraud evaluation, and how do KS statistic, PR-AUC, and Top-1% Capture Rate serve as authoritative metrics?
4
Derive the optimal Bayesian decision threshold p∗=CFPCFP+CFNp^* = \frac{C_{\text{FP}}}{C_{\text{FP}} + C_{\text{FN}}} from cost matrix parameters CFPC_{\text{FP}} and CFNC_{\text{FN}}?
5
How do unsupervised anomaly detection models (Isolation Forest, AutoEncoders) capture novel zero-day fraud without labeled positives?
Updated 2026-08-14
🎯
Verify Mastery: Practice Exam Questions for "Imbalanced & Cost-Sensitive Learning"
Take pitfall multiple-choice questions with derivation options; mistakes are automatically logged.
🚀 Start Card Practice ➔
← Previous CardFlink Real-Time Feature StreamNext Card →Graph Risk & Collusion Detection

🔗 More System Design Knowledge Cards

引用出处: TalentMe 450+ 工业级 AI 架构考点知识库 (Tech Vault) · 遵循 CC-BY-4.0 知识共享协议
URL: https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/imbalanced-cost-sensitive-learning
中文速览与面试要点

极端不平衡样本与代价敏感学习 (Imbalanced & Cost-Sensitive Learning)

定义: 极端不平衡样本与代价敏感学习 (Imbalanced Classification & Cost-Sensitive Machine Learning) 是解决金融风控中正负样本比例极度悬殊(欺诈正样本通常低于 0.01\% 0.1\%,比例达 1:10000)且误分类代价极不对称的核心算法体系;架构从三个层面协同攻关:1) 数据层重采样:自适应合成少数类样本 (SMOTE / Borderline-SMOTE) 与 EasyEnsemble / BalanceCascade 分级负采样;2) 算法层损失重构:Focal Loss FL(p_t) = - _t (1 - p_t)^ (p_t)(动态抑制海量易分负样本的梯度贡献,迫使模型聚焦于难分欺诈样本)与类别加权损失;3) 决策层代价敏感矩阵 (Cost-Sensitive Matrix):显式定义误杀正常用户 (False Positive: 损失单笔交易手续费与用户信任) 与漏放欺诈盗刷 (False Negative: 承担全额坏账本金损失) 的不对称惩罚代价,动态求解最优决策阈值。 使用场景: 信用卡盗刷检测、反洗钱大额异常监控、信贷违约坏账预警、恶意黑产撞库拦截。 解决的问题: 标准交叉熵损失在 1:10000 样本下会退化为“全猜正常样本即可取得 99.99% 虚假准确率”的灾难性局部最优,且无法体现漏检与误杀在真实资金层面的不对称损失;代价敏感体系确保了极低误杀率下的高欺诈召回率。

核心要点

  • 推导 Focal Loss 的数学形式 FL(p_t) = - _t (1 - p_t)^ (p_t),并解释调节参数 如何动态降低易分样本权重?
  • 详细对比 EasyEnsemble 与 BalanceCascade 在利用海量负样本构建 Bagging 子模型中的负样本抽样策略?
  • 风控模型评估中为什么严禁使用 Accuracy?如何使用 KS 值 (Kolmogorov-Smirnov)、PR-AUC 与 Top-1% 捕获率作为核心验收标准?
  • 构建风控代价敏感决策矩阵:根据预期坏账损失 C_ FN 与误杀用户投诉成本 C_ FP 推导最优二分类决策阈值 p^* = C_ FPC_ FP + C_ FN?
  • 无监督与半监督异常检测模型(如 Isolation Forest 孤立森林、One-Class SVM、AutoEncoder 重构误差)在应对全新零日欺诈模式中的应用?