Live Coding: Logistic Regression SGD

Live Coding Logistic Regression with L2 Regularization & Mini-Batch SGD in Pure Numpy assesses an ML engineer's foundational mathematical derivations, Sigmoid activations, cross-entropy gradient calculus, and matrix broadcasting; model formulation: $P(y=1|x) = \sigma(X w + b) = \frac{1}{1 + e^{-(X w + b)}}$; with an L2 penalty, the objective is $\mathcal{L}(w) = -\frac{1}{N} \sum [y \ln(\hat{y}) + (1-y)\ln(1-\hat{y})] + \frac{\lambda}{2N} \|w\|^2$; exact analytical gradient is $\nabla_w \mathcal{L} = \frac{1}{N} X^T (\hat{y} - y) + \frac{\lambda}{N} w$; Mini-Batch SGD updates weights via $w \leftarrow w - \eta \nabla_w \mathcal{L}$ until convergence.

Baseline linear classification, CTR prediction building blocks, and 15-minute senior MLE live-coding interview drills.

Validates whether an engineer can execute the complete end-to-end forward-backward optimization loop in zero-dependency pure NumPy without relying on black-box auto-grad frameworks.

Write the complete Pure Numpy `LogisticRegression` class with `fit(X, y)`, `predict_proba(X)`, and vectorized L2 regularization?

Derive the log-likelihood cross-entropy loss and its parameter gradient $\nabla_w = \frac{1}{N} X^T (\hat{y} - y)$ from Maximum Likelihood principles?

Prove that $\sigma'(z) = \sigma(z)(1 - \sigma(z))$ and explain its application in backpropagation chain rule simplifications?

Compare Newton-Raphson (IRLS) second-order quadratic convergence against first-order SGD in compute complexity ($O(d^3)$ Hessian inverse)?

Why is Feature Scaling strictly required for Logistic Regression, and how do elongated elliptical contours slow down gradient descent?