M2-028M2: Classical Machine Learning集成方法 (Bagging/RF)Easy
⚡Flashcard Drill➔Mastery:
Explain the Two Sources of Randomness in Random Forests and How They Decorrelate Trees
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Random Forests introduce randomness via (1) Bootstrap sample bagging (random rows with replacement) and (2) Feature subspace projection (random columns at each split); together they slash inter-tree correlation to minimize ensemble variance.
📌 Key Takeaways
- •1. Bootstrap Sample Randomness (Rows): Each tree is trained on a distinct bootstrap sample containing unique observations, leaving as Out-Of-Bag (OOB) data.
- •2. Feature Subspace Randomness (Columns): At every split, only a random subset of features is evaluated.
- •Role of Feature Randomness: Prevents dominant strong features from being chosen at the root of every tree, decorrelating tree structures.
📐 Mathematical Derivations
Proof of the bootstrap coverage: For sample size , the probability that a specific observation is NOT selected in a single draw with replacement is . Across independent draws, the probability that is completely omitted from the bootstrap sample is: . Taking the limit as : . Therefore, the expected proportion of unique training observations included in each tree is . The remaining constitutes the Out-of-Bag (OOB) evaluation set.
🏭 Production Trade-offs
Why feature subsampling is critical: Suppose dataset has one super-predictive feature and 20 moderately predictive features. Without feature subsampling (standard Bagging), every single tree picks as the root split, making all trees structurally identical and causing correlation . By forcing trees to evaluate only features at each split, in of splits is excluded, forcing trees to explore orthogonal feature representations and driving close to 0.
⚠️ Common Interview Pitfalls
- ✕Setting
max_features=Nonein Random Forest (reverts to standard Bagging, losing the tree decorrelation benefit). - ✕Using feature subsampling once per tree rather than dynamically re-sampling at every individual split node.
🎯 Interviewer Follow-up Inquiries
- ?Why is the standard default for classification while is standard for regression in Random Forests?
- ?How does Extremely Randomized Trees (Extra-Trees) add a third layer of randomness via random split thresholds?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚