M2-040M2: Classical Machine LearningSupport Vector Machines & KernelsHard
⚡Flashcard Drill➔Mastery:
Assess the Sensitivity of SVM to Class Imbalance and Label Noise and Industrial Mitigations
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
SVM is highly sensitive to class imbalance (which shifts the hyperplane toward the minority class) and label noise near the margin; mitigated via class-weighted penalties () and slack variable bounds.
📌 Key Takeaways
- •Class Imbalance Sensitivity: The margin objective balances against misclassification penalties; when majority class dominates, the hyperplane collapses toward the minority class, predicting all majority.
- •Label Noise Sensitivity: Misclassified points near the boundary have slack and non-zero , exerting maximum possible leverage on the hyperplane orientation.
- •Class-Weighted SVM: Assigns asymmetric penalty weights and (
class_weight='balanced').
📐 Mathematical Derivations
Class-Weighted SVM objective: subject to standard slack constraints. In the dual formulation: the box constraints on Lagrange multipliers become asymmetric: for positive instances, and for negative instances. Setting forces the optimizer to treat a minority false negative as far more costly than a majority false positive, pushing the decision boundary away from the minority cluster.
🏭 Production Trade-offs
Remediation toolkit: (1) Class-weighted SVM: Built into scikit-learn via
class_weight='balanced'. (2) Threshold Moving: Predict continuous decision function scores and tune the classification threshold using precision-recall curves rather than raw . (3) Under-sampling the majority class: Reduces training set size while balancing classes, providing faster quadratic programming convergence.⚠️ Common Interview Pitfalls
- ✕Using standard unweighted SVM on 99:1 imbalanced fraud data (model will achieve 99% accuracy by predicting zero fraud).
- ✕Setting very large in the presence of label noise (forces SVM to twist the decision boundary to fit mislabeled points).
🎯 Interviewer Follow-up Inquiries
- ?How does the -SVM formulation parameterize the fraction of margin errors using parameter ?
- ?Why does Support Vector Data Description (SVDD / One-Class SVM) excel at extreme anomaly detection?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚