M2-037M2: Classical Machine LearningSupport Vector Machines & KernelsEasy
⚡Flashcard Drill➔Mastery:
Formulate Soft-Margin SVM, Slack Variables, and the Regularization Role of Parameter C
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Soft-margin SVM introduces slack variables to tolerate non-separable margin violations, with hyperparameter governing the tradeoff between margin width and classification error penalties.
📌 Key Takeaways
- •Primal Objective: subject to and .
- •Slack Variable : (correctly classified beyond margin); (correctly classified inside margin); (misclassified).
- •Parameter : Large heavily penalizes errors (narrow margin, high variance / overfitting); small tolerates errors (wide margin, high bias / underfitting).
📐 Mathematical Derivations
Equivalence to Hinge Loss: Rearranging the constraint gives . Since , the minimal feasible slack variable is . Substituting into the primal objective yields the unconstrained formulation: . This reveals that Soft-Margin SVM is mathematically equivalent to L2-regularized Empirical Risk Minimization using the Hinge Loss function with regularization parameter .
🏭 Production Trade-offs
Dual problem formulation: subject to and . Notice that appears strictly as an upper bound (box constraint) on the dual multipliers , preventing any single outlier from exerting unbounded influence on the decision hyperplane.
⚠️ Common Interview Pitfalls
- ✕Setting on noisy or non-linearly separable data (causes solver failure or extreme overfitting).
- ✕Confusing the role of in SVM ( is inversely proportional to regularization : large means weak regularization, small means strong regularization).
🎯 Interviewer Follow-up Inquiries
- ?Why is the Hinge loss piecewise linear and non-differentiable at , and how does SMO handle its subgradients?
- ?What happens to the support vectors when vs ?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚