M2-093M2: Classical Machine LearningLinear RegressionMedium
⚡Flashcard Drill➔Mastery:

Interaction and Polynomial Terms in Regression: Formulations and Regularization

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-094 →
📐 Mathematical Definition
y=w0+w1x1+w2x2+w3x1x2+εy=w_0+w_1x_1+w_2x_2+w_3x_1x_2+\varepsilon
⚡ One-Sentence Core Answer
Incorporate non-linear effects and conditional dynamics via product terms; mitigate overfitting using the hierarchical principle, centering, and L1/L2 regularization.

📌 Key Takeaways

  • •
    Polynomial terms model non-linear curvature; interaction terms model conditional feature effects
  • •
    Hierarchical principle: never include interaction x1x2x_1 x_2 without retaining main effects x1x_1 and x2x_2
  • •
    Multicollinearity control: center features (x−xˉx - \bar{x}) before computing products to reduce collinearity with main effects

📐 Mathematical Derivations

Model Formulation: y=β0+β1x1+β2x2+β12x1x2+β11x12+ϵy = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_{12} x_1 x_2 + \beta_{11} x_1^2 + \epsilon.
- Hierarchical Principle (Well-formulated Models): An interaction term β12x1x2\beta_{12} x_1 x_2 implies that the marginal effect of x1x_1 is ∂y∂x1=β1+β12x2\frac{\partial y}{\partial x_1} = \beta_1 + \beta_{12} x_2. If main effect β1\beta_1 is omitted, the model forces the marginal effect to vanish at x2=0x_2 = 0, which is an arbitrary artifact of measurement scaling.
- Centering: Raw products x1x2x_1 x_2 are naturally collinear with x1x_1 and x2x_2. Centering transforms variables to x~i=xi−xˉi\tilde{x}_i = x_i - \bar{x}_i, making Cov(x~1,x~1x~2)≈0\text{Cov}(\tilde{x}_1, \tilde{x}_1 \tilde{x}_2) \approx 0 under symmetric distributions.

🏭 Production Trade-offs

Regularization strategy: When expanding to O(d2)O(d^2) terms, apply Lasso or ElasticNet with hierarchical constraints (e.g., HierNet or Group Lasso) to ensure sparse selection while honoring structural hierarchies.
⚠️ Common Interview Pitfalls
  • ✕
    Omitting linear main effects when including interaction terms, making model predictions scale-dependent
  • ✕
    Generating uncentered polynomial terms, inducing extreme multicollinearity and numeric instability
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does shifting the origin (e.g., Celsius vs Kelvin) break a regression model that contains interaction terms without main effects?
  • ?
    How does Group Lasso enforce the hierarchical principle between main effects and interaction terms?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-092: Weighted Least Squares (WLS) and Handling Heteroscedasticity📋 Back to BankNext →M2-094: Multinomial Logistic Regression (Softmax Regression) and Parameter Identifiability