M2-061M2: Classical Machine LearningDimensionality ReductionHard
⚡Flashcard Drill➔Mastery:

How to Choose Target Dimensions in Dimensionality Reduction? Variance Ratio Method and Limitations

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-062 →
📐 Mathematical Definition
ratiok=∑i≤kλi∑iλi\text{ratio}_k=\frac{\sum_{i\le k}\lambda_i}{\sum_i\lambda_i}
⚡ One-Sentence Core Answer
Select dimensions that explain 90–95% of cumulative variance; however, because this heuristic is downstream-agnostic, downstream validation performance should guide the final choice.

📌 Key Takeaways

  • •
    Variance ratio criteria: 90% or 95% threshold, or scree plot elbow point
  • •
    Downstream validation metric (AUC, RMSE) is the ultimate criterion
  • •
    Unsupervised variance does not necessarily correlate with label information

📐 Mathematical Derivations

Cumulative Explained Variance Ratio: Compute the proportion of variance explained by the top kk principal components relative to the total variance, choosing the minimum kk such that ∑i=1kλi/∑j=1dλj≥τ\sum_{i=1}^k \lambda_i / \sum_{j=1}^d \lambda_j \ge \tau (common thresholds τ∈{0.90,0.95,0.99}\tau \in \{0.90, 0.95, 0.99\}). In practice, inspect the scree plot (eigenvalues plotted in decreasing order) to identify the elbow point. Advantages: Fast, deterministic, and requires no label information. Limitations: Unsupervised variance does not guarantee predictive relevance—directions with low variance may contain critical label signal, while large-variance components may simply reflect background noise.

🏭 Production Trade-offs

Industrial selection strategies: ① Downstream Validation Metric: Treat kk as a hyperparameter and optimize cross-validation score (AUC, F1, NDCG) on downstream models. ② Computational and Latency Constraints: In high-throughput inference (e.g., real-time vector search), cap kk based on latency and memory budgets (e.g., k≤64k \le 64). ③ Intrinsic Dimensionality Estimation: Use Maximum Likelihood Estimation (MLE) or correlation dimension methods to mathematically estimate manifold dimension. ④ Reconstruction Error: In autoencoders, select kk where marginal reconstruction loss plateaus.
⚠️ Common Interview Pitfalls
  • ✕
    Blindly choosing 95% variance ratio without evaluating downstream task performance
  • ✕
    Assuming low-variance components are useless when they might hold the primary classification signal
🎯 Interviewer Follow-up Inquiries
  • ?
    Why might a low-variance principal component be more informative for classification than the top component?
  • ?
    How does scree plot analysis differ from cross-validated dimension tuning in production?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-060: Compare t-SNE and UMAP: Manifold Assumptions, Global vs. Local Structure, and Computational Scaling📋 Back to BankNext →M2-062: Common Categorical Feature Encoding Methods and Applicable Scenarios