← Back to ML Engineer Mind Map
💻 ML EngineerID: mle-feature-selection-shap

Feature Selection, Permutation & SHAP

特征选择与置换重要性 / SHAP
🎯Core Definition
Feature Selection Taxonomy, Permutation Importance & SHAP Values provide the systematic framework to compress high-dimensional feature spaces, purge collinear noise, and attribute individual feature contributions via game theory; the 3 selection paradigms comprise: 1) Filter Methods (Mutual Information, Chi-Square, Pearson Correlation); 2) Wrapper Methods (Recursive Feature Elimination, Forward/Backward search); 3) Embedded Methods (L1 Lasso sparsity, Tree Gini Gain); Permutation Importance shuffles a single feature column on validation sets to quantify true performance drops; SHAP (SHapley Additive exPlanations) computes exact Shapley values ϕi=∑S⊆F∖{i}∣S∣!(∣F∣−∣S∣−1)!∣F∣![f(S∪{i})−f(S)]\phi_i = \sum_{S \subseteq F \setminus \{i\}} \frac{|S|!(|F|-|S|-1)!}{|F|!} [f(S \cup \{i\}) - f(S)], delivering game-theoretically consistent feature attribution.
💡Use Cases
High-dimensional feature pruning, credit risk regulatory compliance explanations, and diagnosing multicollinearity.
⚡Key Problems Solved
Default tree feature importance (Split Counts) is heavily biased towards high-cardinality continuous variables; Permutation and TreeSHAP yield unbiased, mathematically proven global and local feature attributions.
🎯Core Interview Exam Points
Fundamentals · Production Scenarios · Deep Dive
Core Def
Explain the core definition, underlying mechanism/math, and key principles of Feature Selection, Permutation & SHAP.
Use Cases
In production engineering, what are the primary use cases for Feature Selection, Permutation & SHAP and what core pain points does it solve?
🔥 High-Frequency Deep-Dive Questions
1
Derive Shapley value axiomatic properties (Efficiency, Symmetry, Dummy, Additivity) and explain its marginal contribution formulation?
2
Explain why GBDT Split Count importance is biased towards continuous features with many unique values, and how TreeSHAP corrects this?
3
Explain how multi-collinearity distorts Permutation Importance and how Hierarchical Clustering Ward linkage groups redundant features?
4
How does TreeSHAP optimize exponential Shapley calculation down to polynomial time O(TLD2)O(T L D^2) using decision tree path structures?
5
Explain the algorithmic workflow of RFECV (Recursive Feature Elimination with Cross-Validation) in selecting optimal feature subsets?
🔗Foundational Prerequisite Cards (Click to Review)
Updated 2026-08-14
🎯
Verify Mastery: Practice Exam Questions for "Feature Selection, Permutation & SHAP"
Take pitfall multiple-choice questions with derivation options; mistakes are automatically logged.
🚀 Start Card Practice ➔
← Previous CardOOF Target Encoding & Anti-LeakageNext Card →Production PSI Drift Monitoring

🔗 More ML Engineer Knowledge Cards

引用出处: TalentMe 450+ 工业级 AI 架构考点知识库 (Tech Vault) · 遵循 CC-BY-4.0 知识共享协议
URL: https://talentme.airsota.com/resources/tech/roadmap/mindmap/MLE/card/mle-feature-selection-shap
中文速览与面试要点

特征选择与置换重要性 / SHAP (Feature Selection, Permutation & SHAP)

定义: 特征选择体系与博弈论 SHAP 可解释性分析 (Feature Selection Taxonomy, Permutation Importance & SHAP Values) 是机器学习工程师从海量候选特征池(数百上千个特征)中提炼高价值子集、剔除噪声共线性特征并进行全局与局部归因分析的方法论;特征选择三大范式:1) 过滤法 (Filter: 方差阈值、皮尔逊相关系数、互信息 Mutual Information、卡方检验);2) 包裹法 (Wrapper: 递归特征消除 RFE、前向/后向贪心选择);3) 嵌入法 (Embedded: Lasso L1 稀疏惩罚、GBDT 树分裂增益);更进一步,置换特征重要性 (Permutation Feature Importance) 通过随机打乱单列特征评估验证集指标下降幅度消除基数偏置;SHAP (SHapley Additive exPlanations) 基于合作博弈论 Shapley 值公式 _i = _S F \i\ |S|!(|F|-|S|-1)!|F|! [f(S \i\) - f(S)],提供严格满足加性、效率性与对称性的黄金归因解释。 使用场景: 工业模型降维精简、模型可解释性业务汇报、消除多重共线性特征与敏感偏见审计。 解决的问题: 传统树模型自带的“基于分裂次数 (Split Count)”或“增益 (Gain)”的重要度极度偏向高基数连续数值特征导致严重误判;Permutation 与 TreeSHAP 提供了客观、无偏且具有博弈论数学保证的特征重要性排序。

核心要点

  • 推导合作博弈论 Shapley 值的经典公式 _i = _S F \i\ |S|!(|F|-|S|-1)!|F|! [f(S \i\) - f(S)] 的公理化性质(效率性、对称性、虚拟性、加性)?
  • 为什么在 GBDT 树模型中,直接使用默认的 `feature_importances_` (Split Count) 会对高基数浮点数产生严重的虚假偏好?
  • 置换特征重要性 (Permutation Importance) 在遇到高度共线性特征(Collinear Features: 如两列特征相关度达 0.99)时的失真陷阱与聚类排查?
  • TreeSHAP 算法如何利用决策树的局部路由结构将 Shapley 值的指数级时间复杂度 O(2^|F|) 压缩至多项式时间 O(T L D^2)?
  • 递归特征消除 (RFE: Recursive Feature Elimination) 配合交叉验证 (RFECV) 在寻找全局最优特征数量 K^* 时的算法流转?