M1-017M1: Mathematics & Statistics FundamentalsLinear AlgebraEasy
⚡Flashcard Drill➔Mastery:

Explain Matrix Rank, Nullspace, Column Space, and Their Role in Linear System Solvability

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M1-018 →
📐 Mathematical Definition
rank(A)+dim⁡N(A)=n,Ax=b 可解  ⟺  b∈C(A)\mathrm{rank}(A)+\dim\mathcal N(A)=n,\qquad Ax=b\ \text{可解}\iff b\in\mathcal C(A)
⚡ One-Sentence Core Answer
Column space is the span of achievable outputs, nullspace comprises inputs mapped to zero, and rank equals column space dimension; Ax=bAx=b is solvable if and only if b∈Col(A)b \in \text{Col}(A).

📌 Key Takeaways

  • •
    Rank-Nullity Theorem: Rank(A)+Nullity(A)=n\text{Rank}(A) + \text{Nullity}(A) = n (total column dimensions).
  • •
    Solvability: Ax=bAx = b has a solution iff Rank(A)=Rank([A∣b])\text{Rank}(A) = \text{Rank}([A\mid b]).
  • •
    Uniqueness: Solution is unique iff Nullity(A)=0\text{Nullity}(A) = 0 (columns are linearly independent).

📐 Mathematical Derivations

Let A∈Rm×nA \in \mathbb{R}^{m\times n}. Column space Col(A)={Ax:x∈Rn}⊆Rm\text{Col}(A) = \{Ax : x \in \mathbb{R}^n\} \subseteq \mathbb{R}^m, with dimension r=Rank(A)r = \text{Rank}(A). Nullspace Null(A)={x∈Rn:Ax=0}⊆Rn\text{Null}(A) = \{x \in \mathbb{R}^n : Ax = 0\} \subseteq \mathbb{R}^n. The fundamental theorem of linear algebra dictates Row(A)⊥Null(A)\text{Row}(A) \perp \text{Null}(A) and Col(A)⊥Null(AT)\text{Col}(A) \perp \text{Null}(A^T). Any vector xx decomposes uniquely into x=xr+xnx = x_r + x_n where xr∈Row(A)x_r \in \text{Row}(A) and xn∈Null(A)x_n \in \text{Null}(A). If b∈Col(A)b \in \text{Col}(A), the general solution is x=xp+xnx = x_p + x_n with particular solution xpx_p. If Null(A)≠{0}\text{Null}(A) \ne \{0\}, infinite solutions exist.

🏭 Production Trade-offs

In overdetermined systems (m>nm > n, more equations than unknowns, e.g. linear regression), bb rarely lies precisely in Col(A)\text{Col}(A). Ordinary Least Squares projects bb orthogonally onto Col(A)\text{Col}(A), yielding the normal equations ATAx=ATbA^T A x = A^T b. If AA is full column rank, ATAA^T A is invertible, yielding unique x^=(ATA)−1ATb\hat{x} = (A^T A)^{-1} A^T b. If rank-deficient, regularization (L2L2 Ridge) adds λI\lambda I to restore full rank.
⚠️ Common Interview Pitfalls
  • ✕
    Assuming full rank implies an exact solution exists for any bb in overdetermined rectangular matrices.
  • ✕
    Attempting to invert ATAA^T A directly when multicollinear features cause rank deficiency.
🎯 Interviewer Follow-up Inquiries
  • ?
    How does the Moore-Penrose pseudoinverse select the minimum-norm solution when nullspace is non-trivial?
  • ?
    Why does L2 regularization guarantee positive definiteness for ATA+λIA^T A + \lambda I?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM1-016: Explain the Geometric Meaning of Eigenvalues, Eigenvectors, and Singular Value Decomposition (SVD)📋 Back to BankNext →M1-018: Define Ill-Conditioned Matrices and Condition Numbers, and Explain Their Impact on Numerical Solvers and Training