M2-002M2: Classical Machine LearningLinear RegressionEasy
⚡Flashcard Drill➔Mastery:

Enumerate the Four Classical Assumptions of Linear Regression and the Consequences of Their Violation

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-003 →
📐 Mathematical Definition
y=Xw+ε,E[ε]=0, Var(ε)=σ2Iy=Xw+\varepsilon,\quad \mathbb E[\varepsilon]=0,\ \mathrm{Var}(\varepsilon)=\sigma^2I
⚡ One-Sentence Core Answer
The classical OLS assumptions are Linearity, Exogeneity (zero conditional mean of errors), Homoscedasticity (constant error variance with zero autocorrelation), and Residual Normality; violations cause coefficient bias, deflated standard errors, and invalid p-values.

📌 Key Takeaways

  • •
    1. Linearity: E[y∣X]=XβE[y\mid X] = X\beta; violation causes severe model underfitting and systematic estimation bias.
  • •
    2. Strict Exogeneity: E[ϵ∣X]=0E[\epsilon \mid X] = 0; violation (omitted variables, measurement error, simultaneity) causes endogeneity and inconsistent β^\hat{\beta}.
  • •
    3. Spherical Errors: Var(ϵ∣X)=σ2I\text{Var}(\epsilon \mid X) = \sigma^2 I (homoscedasticity + no autocorrelation); violation invalidates standard errors and t-tests.
  • •
    4. Normality of Residuals: ϵ∼N(0,σ2I)\epsilon \sim \mathcal{N}(0, \sigma^2 I); required for exact finite-sample t-tests and F-tests.

📐 Mathematical Derivations

Impact of exogeneity violation: β^=(XTX)−1XT(Xβ+ϵ)=β+(XTX)−1XTϵ\hat{\beta} = (X^T X)^{-1} X^T (X\beta + \epsilon) = \beta + (X^T X)^{-1} X^T \epsilon. Taking expectations conditional on XX: E[β^∣X]=β+(XTX)−1XTE[ϵ∣X]E[\hat{\beta}\mid X] = \beta + (X^T X)^{-1} X^T E[\epsilon\mid X]. If E[ϵ∣X]≠0E[\epsilon\mid X] \ne 0 (endogeneity), β^\hat{\beta} is fundamentally biased and inconsistent. Impact of heteroscedasticity: Let Var(ϵ∣X)=Ω≠σ2I\text{Var}(\epsilon\mid X) = \Omega \ne \sigma^2 I. The true covariance matrix is the sandwich covariance Var(β^)=(XTX)−1XTΩX(XTX)−1\text{Var}(\hat{\beta}) = (X^T X)^{-1} X^T \Omega X (X^T X)^{-1}. Using standard OLS covariance σ2(XTX)−1\sigma^2 (X^T X)^{-1} produces severely deflated standard errors, resulting in artificially tiny p-values and massive false positive discoveries.

🏭 Production Trade-offs

Industrial diagnostics and remediations: (1) Heteroscedasticity: Diagnosed via Breusch-Pagan test or residual-vs-fitted plots; resolved by using Huber-White heteroscedasticity-robust standard errors (HC1/HC3). (2) Endogeneity: Resolved via Instrumental Variables (2SLS) or fixed-effects panel models. (3) Non-linearity: Resolved via polynomial features, splines, or switching to tree-based ensembles (GBDT).
⚠️ Common Interview Pitfalls
  • ✕
    Assuming normality of raw features XX is an assumption of linear regression (OLS makes ZERO assumptions about the distribution of XX, only about residuals ϵ\epsilon).
  • ✕
    Ignoring cluster correlation in hierarchical data (e.g. Repeated sessions per user), causing standard errors to be underestimated by factors of 3 to 10.
🎯 Interviewer Follow-up Inquiries
  • ?
    How do White's heteroscedasticity-consistent standard errors correct inference without modifying point estimates?
  • ?
    Why does residual non-normality become practically irrelevant in large samples due to the Central Limit Theorem?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-001: Formulate the Closed-Form Normal Equation for Linear Regression and Its Necessary Preconditions📋 Back to BankNext →M2-003: Explain Multicollinearity, Its Impact on Coefficient Stability and Significance Tests, and Diagnostic Thresholds