M2-061M2: Classical Machine LearningDimensionality ReductionHard
⚡Flashcard Drill➔Mastery:
How to Choose Target Dimensions in Dimensionality Reduction? Variance Ratio Method and Limitations
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Select dimensions that explain 90–95% of cumulative variance; however, because this heuristic is downstream-agnostic, downstream validation performance should guide the final choice.
📌 Key Takeaways
- •Variance ratio criteria: 90% or 95% threshold, or scree plot elbow point
- •Downstream validation metric (AUC, RMSE) is the ultimate criterion
- •Unsupervised variance does not necessarily correlate with label information
📐 Mathematical Derivations
Cumulative Explained Variance Ratio: Compute the proportion of variance explained by the top principal components relative to the total variance, choosing the minimum such that (common thresholds ). In practice, inspect the scree plot (eigenvalues plotted in decreasing order) to identify the elbow point. Advantages: Fast, deterministic, and requires no label information. Limitations: Unsupervised variance does not guarantee predictive relevance—directions with low variance may contain critical label signal, while large-variance components may simply reflect background noise.
🏭 Production Trade-offs
Industrial selection strategies: ① Downstream Validation Metric: Treat as a hyperparameter and optimize cross-validation score (AUC, F1, NDCG) on downstream models. ② Computational and Latency Constraints: In high-throughput inference (e.g., real-time vector search), cap based on latency and memory budgets (e.g., ). ③ Intrinsic Dimensionality Estimation: Use Maximum Likelihood Estimation (MLE) or correlation dimension methods to mathematically estimate manifold dimension. ④ Reconstruction Error: In autoencoders, select where marginal reconstruction loss plateaus.
⚠️ Common Interview Pitfalls
- ✕Blindly choosing 95% variance ratio without evaluating downstream task performance
- ✕Assuming low-variance components are useless when they might hold the primary classification signal
🎯 Interviewer Follow-up Inquiries
- ?Why might a low-variance principal component be more informative for classification than the top component?
- ?How does scree plot analysis differ from cross-validated dimension tuning in production?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚