M1-016M1: Mathematics & Statistics FundamentalsLinear AlgebraEasy
⚡Flashcard Drill➔Mastery:

Explain the Geometric Meaning of Eigenvalues, Eigenvectors, and Singular Value Decomposition (SVD)

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M1-017 →
📐 Mathematical Definition
A=UΣV⊤,Avi=λivi (对称阵)A=U\Sigma V^\top,\qquad Av_i=\lambda_i v_i\ (\text{对称阵})
⚡ One-Sentence Core Answer
Eigenvectors identify directions invariant under linear transformations with eigenvalues scaling magnitude; SVD generalizes this to rectangular matrices via rotation, scaling, and rotation (UΣVTU\Sigma V^T).

📌 Key Takeaways

  • •
    Eigen-decomposition Av=λvA v = \lambda v applies only to square matrices along invariant axes.
  • •
    SVD A=UΣVTA = U \Sigma V^T decomposes any m×nm\times n matrix into input orthonormal basis VV, singular values Σ\Sigma, and output basis UU.
  • •
    Eckart-Young-Mirsky Theorem: Truncated SVD provides the provably optimal low-rank matrix approximation under Frobenius and spectral norms.

📐 Mathematical Derivations

For real symmetric matrix AA, spectral theorem ensures A=QΛQTA = Q \Lambda Q^T with orthonormal eigenvectors QQ. For arbitrary m×nm\times n matrix AA, ATAA^T A is symmetric positive semi-definite with ATAvi=σi2viA^T A v_i = \sigma_i^2 v_i, defining right singular vectors VV. Defining ui=Aviσiu_i = \frac{A v_i}{\sigma_i} yields orthonormal left singular vectors UU. Geometrically, the transformation x↦Axx \mapsto Ax transforms an nn-dimensional unit hypersphere into an mm-dimensional hyperellipsoid: VTV^T rotates the sphere, Σ\Sigma scales axes by singular values σi\sigma_i, and UU rotates the resulting ellipsoid in the target space.

🏭 Production Trade-offs

SVD underpins Principal Component Analysis (PCA), Latent Semantic Analysis (LSA), and parameter-efficient LoRA adaptations. In LoRA, weight updates ΔW∈Rd×k\Delta W \in \mathbb{R}^{d\times k} are parameterized as BAB A with rank r≪min⁡(d,k)r \ll \min(d, k), directly motivated by the low-rank singular value concentration discovered in trained weight matrices.
⚠️ Common Interview Pitfalls
  • ✕
    Confusing singular values σi\sigma_i (always real and non-negative) with eigenvalues λi\lambda_i (can be negative or complex).
  • ✕
    Forgetting that eigen-decomposition requires square, diagonalizable matrices, whereas SVD exists for all matrices.
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does truncated SVD minimize ∥A−Ak∥F2\|A - A_k\|_F^2?
  • ?
    How is PCA mathematically derived from the SVD of a centered data matrix XX?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM1-015: Explain the Physical Intuition of Perplexity (PPL) and Prove Why It Equals the Exponential of Cross-Entropy📋 Back to BankNext →M1-017: Explain Matrix Rank, Nullspace, Column Space, and Their Role in Linear System Solvability