M2-008M2: Classical Machine LearningLogistic Regression & GLMMedium
⚡Flashcard Drill➔Mastery:
Define Generalized Linear Models (GLMs) and Detail Their Three Foundational Components
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
A Generalized Linear Model (GLM) extends linear regression to non-Gaussian distributions within the exponential family; composed of (1) Random Component (exponential family distribution), (2) Systematic Component (linear predictor ), and (3) Link Function ().
📌 Key Takeaways
- •1. Random Component: The target variable follows an exponential family distribution (Gaussian, Bernoulli, Poisson, Gamma) with mean .
- •2. Systematic Component: Linear predictor .
- •3. Link Function: A monotonic, differentiable function that connects the conditional mean to the linear predictor: .
📐 Mathematical Derivations
The canonical link function equates the linear predictor directly to the natural parameter of the exponential family . Since , the canonical link is . Derivations of classical models: (1) Linear Regression: Normal distribution with . The canonical link is identity: . (2) Logistic Regression: Bernoulli distribution with . The canonical link is logit: . (3) Poisson Regression: Poisson distribution with . The canonical link is log: .
🏭 Production Trade-offs
GLMs eliminate ad-hoc non-linear transformations: (1) If modeling positive count data (e.g. Taxi trips or customer purchases), fitting OLS to fails when . Poisson or Negative Binomial regression directly models counts with exact discrete likelihoods. (2) Canonical links guarantee that the negative log-likelihood is strictly convex and that the Hessian equals the Fisher Information Matrix, ensuring fast, stable IRLS convergence.
⚠️ Common Interview Pitfalls
- ✕Using Poisson regression when the variance vastly exceeds the mean (overdispersion; must use Negative Binomial or Quasi-Poisson GLM).
- ✕Confusing the link function with transforming the raw response variable (GLM models , not ; by Jensen's inequality, ).
🎯 Interviewer Follow-up Inquiries
- ?Why does Jensen's inequality explain why in Poisson regression is fundamentally different from fitting in OLS?
- ?What is Tweedie regression and why is it the gold standard in insurance claims and ad-spend LTV modeling with mass at zero?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚