M2-002M2: Classical Machine LearningLinear RegressionEasy
⚡Flashcard Drill➔Mastery:
Enumerate the Four Classical Assumptions of Linear Regression and the Consequences of Their Violation
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
The classical OLS assumptions are Linearity, Exogeneity (zero conditional mean of errors), Homoscedasticity (constant error variance with zero autocorrelation), and Residual Normality; violations cause coefficient bias, deflated standard errors, and invalid p-values.
📌 Key Takeaways
- •1. Linearity: ; violation causes severe model underfitting and systematic estimation bias.
- •2. Strict Exogeneity: ; violation (omitted variables, measurement error, simultaneity) causes endogeneity and inconsistent .
- •3. Spherical Errors: (homoscedasticity + no autocorrelation); violation invalidates standard errors and t-tests.
- •4. Normality of Residuals: ; required for exact finite-sample t-tests and F-tests.
📐 Mathematical Derivations
Impact of exogeneity violation: . Taking expectations conditional on : . If (endogeneity), is fundamentally biased and inconsistent. Impact of heteroscedasticity: Let . The true covariance matrix is the sandwich covariance . Using standard OLS covariance produces severely deflated standard errors, resulting in artificially tiny p-values and massive false positive discoveries.
🏭 Production Trade-offs
Industrial diagnostics and remediations: (1) Heteroscedasticity: Diagnosed via Breusch-Pagan test or residual-vs-fitted plots; resolved by using Huber-White heteroscedasticity-robust standard errors (HC1/HC3). (2) Endogeneity: Resolved via Instrumental Variables (2SLS) or fixed-effects panel models. (3) Non-linearity: Resolved via polynomial features, splines, or switching to tree-based ensembles (GBDT).
⚠️ Common Interview Pitfalls
- ✕Assuming normality of raw features is an assumption of linear regression (OLS makes ZERO assumptions about the distribution of , only about residuals ).
- ✕Ignoring cluster correlation in hierarchical data (e.g. Repeated sessions per user), causing standard errors to be underestimated by factors of 3 to 10.
🎯 Interviewer Follow-up Inquiries
- ?How do White's heteroscedasticity-consistent standard errors correct inference without modifying point estimates?
- ?Why does residual non-normality become practically irrelevant in large samples due to the Central Limit Theorem?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚