M2-092M2: Classical Machine LearningLinear RegressionMedium
⚡Flashcard Drill➔Mastery:
Weighted Least Squares (WLS) and Handling Heteroscedasticity
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
WLS weights each residual inversely by the error variance (), transforming heteroscedastic data into homoscedastic space to restore BLUE optimality.
📌 Key Takeaways
- •Heteroscedasticity problem: OLS estimates remain unbiased, but standard errors are biased and efficiency is lost
- •WLS solution: solves with weight matrix
- •Feasible GLS (FGLS): estimates variance function from OLS residuals when variance is unknown
📐 Mathematical Derivations
Mathematical formulation: In ordinary regression , homoscedasticity assumes . When heteroscedasticity is present, .
Under Gauss-Markov, OLS is no longer the Best Linear Unbiased Estimator (BLUE). Multiply the system by : . The transformed disturbance has covariance .
Applying OLS to the transformed system yields the WLS Estimator: , where .
Under Gauss-Markov, OLS is no longer the Best Linear Unbiased Estimator (BLUE). Multiply the system by : . The transformed disturbance has covariance .
Applying OLS to the transformed system yields the WLS Estimator: , where .
🏭 Production Trade-offs
Practical application: If variance is unknown, estimate it via auxiliary regression of squared residuals on covariates (Feasible Generalized Least Squares, FGLS). Alternatively, keep OLS point estimates and use Huber-White robust standard errors (sandwich estimator) to obtain valid confidence intervals without modifying coefficients.
⚠️ Common Interview Pitfalls
- ✕Believing heteroscedasticity biases OLS point estimates ; it only destroys statistical efficiency and invalidates standard errors
- ✕Mis-specifying the variance weighting model in FGLS, which can yield worse estimators than unweighted OLS
🎯 Interviewer Follow-up Inquiries
- ?How does the Huber-White sandwich estimator correct standard errors without requiring WLS?
- ?What statistical tests can detect heteroscedasticity (e.g., Breusch-Pagan, White test)?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚