📊 Classic MLID: logistic-regression

Logistic Regression

Q: Whiteboard: invert $\ln\frac{p}{1-p} = w^Tx + b$ into $p = \frac{1}{1+e^{-w^Tx}}$, then derive $\frac{\partial L}{\partial z} = p - y$ via $p' = p(1-p)$ and the chain rule.

【Logistic Regression 核心考点 1】Whiteboard: invert $\ln\frac{p}{1-p} = w^Tx + b$ into $p = \frac{1}{1+e^{-w^Tx}}$, then derive $\frac{\partial L}{\partial z} = p - y$ via $p' = p(1-p)$ and the chain rule.。技术解析：Definition: logistic regression models the log-odds as a linear function \ln\frac{p}{1-p} = w^Tx + b, which inverts to the Sigmoid p = \frac{1}{1 + e^{-(w^Tx + b)}} \in [0,1]. Parameters come from MLE: the per-sample log-likelihood y\ln p + (1-y)\ln(1-p) negated is cross-entropy ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/ML/card/logistic-regression

Q: Why does LR use cross-entropy instead of squared loss? Explain the gradient saturation of squared loss as $p \to y$.

【Logistic Regression 核心考点 2】Why does LR use cross-entropy instead of squared loss? Explain the gradient saturation of squared loss as $p \to y$.。技术解析：Definition: logistic regression models the log-odds as a linear function \ln\frac{p}{1-p} = w^Tx + b, which inverts to the Sigmoid p = \frac{1}{1 + e^{-(w^Tx + b)}} \in [0,1]. Parameters come from MLE: the per-sample log-likelihood y\ln p + (1-y)\ln(1-p) negated is cross-entropy ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/ML/card/logistic-regression

Q: Show MLE ≡ cross-entropy: derive the negative log-likelihood $L = -\sum_i [y_i \ln p_i + (1-y_i)\ln(1-p_i)]$.

【Logistic Regression 核心考点 3】Show MLE ≡ cross-entropy: derive the negative log-likelihood $L = -\sum_i [y_i \ln p_i + (1-y_i)\ln(1-p_i)]$.。技术解析：Definition: logistic regression models the log-odds as a linear function \ln\frac{p}{1-p} = w^Tx + b, which inverts to the Sigmoid p = \frac{1}{1 + e^{-(w^Tx + b)}} \in [0,1]. Parameters come from MLE: the per-sample log-likelihood y\ln p + (1-y)\ln(1-p) negated is cross-entropy ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/ML/card/logistic-regression

Q: Compare LR vs SVM: hinge vs log loss, probabilistic output, support-vector dependence, and outlier robustness.

【Logistic Regression 核心考点 4】Compare LR vs SVM: hinge vs log loss, probabilistic output, support-vector dependence, and outlier robustness.。技术解析：Definition: logistic regression models the log-odds as a linear function \ln\frac{p}{1-p} = w^Tx + b, which inverts to the Sigmoid p = \frac{1}{1 + e^{-(w^Tx + b)}} \in [0,1]. Parameters come from MLE: the per-sample log-likelihood y\ln p + (1-y)\ln(1-p) negated is cross-entropy ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/ML/card/logistic-regression

Q: Why can't LR do multiclass directly? Contrast OvR vs Softmax extensions, and how class imbalance shifts the decision threshold.

【Logistic Regression 核心考点 5】Why can't LR do multiclass directly? Contrast OvR vs Softmax extensions, and how class imbalance shifts the decision threshold.。技术解析：Definition: logistic regression models the log-odds as a linear function \ln\frac{p}{1-p} = w^Tx + b, which inverts to the Sigmoid p = \frac{1}{1 + e^{-(w^Tx + b)}} \in [0,1]. Parameters come from MLE: the per-sample log-likelihood y\ln p + (1-y)\ln(1-p) negated is cross-entropy ... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/ML/card/logistic-regression

逻辑回归 Log-Odds

🎯Core Definition

logistic regression models the log-odds as a linear function

\ln\frac{p}{1-p} = w^Tx + b

, which inverts to the Sigmoid

p = \frac{1}{1 + e^{-(w^Tx + b)}} \in [0,1]

. Parameters come from MLE: the per-sample log-likelihood

y\ln p + (1-y)\ln(1-p)

negated is cross-entropy

L = -\sum_i [y_i \ln p_i + (1-y_i)\ln(1-p_i)]

. Differentiating w.r.t.

z = w^Tx + b

using

p' = p(1-p)

gives

\frac{\partial L}{\partial z} = p - y

, hence

\frac{\partial L}{\partial w} = (p - y)x

; there is no closed form, so optimize by gradient descent / Newton (convex, unique global optimum).

💡Use Cases

the binary-classification baseline (CTR, credit risk, diagnosis); whiteboard staples include deriving Sigmoid from log-odds, deriving

\frac{\partial L}{\partial z} = p - y

, and contrasting LR with SVM / naive Bayes.

⚡Key Problems Solved

fitting raw linear regression to probabilities yields illegal values outside

[0,1]

and is outlier-sensitive; LR guarantees valid probabilities, and the CE gradient

p - y

decays only linearly as

z \to \pm\infty

(never saturates), whereas squared loss

\frac{1}{2}(p-y)^2

gives vanishing gradients near

p \to y

— slow to train; adding

\frac{\lambda}{2}\Vert w\Vert^2

yields regularized LR for high-dimensional collinearity.

🎯5 High-Frequency Exam Points

Whiteboard: invert

\ln\frac{p}{1-p} = w^Tx + b

into

p = \frac{1}{1+e^{-w^Tx}}

, then derive

\frac{\partial L}{\partial z} = p - y

via

p' = p(1-p)

and the chain rule.

Why does LR use cross-entropy instead of squared loss? Explain the gradient saturation of squared loss as

p \to y

Show MLE ≡ cross-entropy: derive the negative log-likelihood

L = -\sum_i [y_i \ln p_i + (1-y_i)\ln(1-p_i)]

Compare LR vs SVM: hinge vs log loss, probabilistic output, support-vector dependence, and outlier robustness.

Why can't LR do multiclass directly? Contrast OvR vs Softmax extensions, and how class imbalance shifts the decision threshold.

📖 In-depth Guide:📄 linear-and-logistic-regression →

Updated 2026-08-12

🎯

Test Your Knowledge: Practice Questions for "Logistic Regression"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardVIF Multicollinearity Next Card →Multiclass Strategies: OvR/OvO/Softmax

🔗 More Classic ML Knowledge Cards

AdaBoost Derivation Bagging & Random Forest Baum-Welch (HMM EM)GBDT Negative Gradient