M2-040M2: Classical Machine LearningSupport Vector Machines & KernelsHard
⚡Flashcard Drill➔Mastery:

Assess the Sensitivity of SVM to Class Imbalance and Label Noise and Industrial Mitigations

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M2-041 →
📐 Mathematical Definition
class weight: Ci=C⋅wyi\text{class weight}:\ C_i=C\cdot w_{y_i}
⚡ One-Sentence Core Answer
SVM is highly sensitive to class imbalance (which shifts the hyperplane toward the minority class) and label noise near the margin; mitigated via class-weighted penalties (C+≠C−C_+ \ne C_-) and slack variable bounds.

📌 Key Takeaways

  • •
    Class Imbalance Sensitivity: The margin objective balances ∥w∥22\|w\|_2^2 against misclassification penalties; when majority class dominates, the hyperplane collapses toward the minority class, predicting all majority.
  • •
    Label Noise Sensitivity: Misclassified points near the boundary have slack ξi>1\xi_i > 1 and non-zero αi=C\alpha_i = C, exerting maximum possible leverage on the hyperplane orientation.
  • •
    Class-Weighted SVM: Assigns asymmetric penalty weights C+=N2N+CC_+ = \frac{N}{2 N_+} C and C−=N2N−CC_- = \frac{N}{2 N_-} C (class_weight='balanced').

📐 Mathematical Derivations

Class-Weighted SVM objective: min⁡w,b,ξ12∥w∥22+C+∑i:yi=+1ξi+C−∑j:yj=−1ξj\min_{w, b, \xi} \frac{1}{2}\|w\|_2^2 + C_+ \sum_{i: y_i=+1} \xi_i + C_- \sum_{j: y_j=-1} \xi_j subject to standard slack constraints. In the dual formulation: the box constraints on Lagrange multipliers become asymmetric: 0≤αi≤C+0 \le \alpha_i \le C_+ for positive instances, and 0≤αj≤C−0 \le \alpha_j \le C_- for negative instances. Setting C+=C⋅N−N+C_+ = C \cdot \frac{N_-}{N_+} forces the optimizer to treat a minority false negative as far more costly than a majority false positive, pushing the decision boundary away from the minority cluster.

🏭 Production Trade-offs

Remediation toolkit: (1) Class-weighted SVM: Built into scikit-learn via class_weight='balanced'. (2) Threshold Moving: Predict continuous decision function scores f(x)=wTϕ(x)+bf(x) = w^T \phi(x) + b and tune the classification threshold τ\tau using precision-recall curves rather than raw f(x)≥0f(x) \ge 0. (3) Under-sampling the majority class: Reduces training set size while balancing classes, providing 10×10\times faster quadratic programming convergence.
⚠️ Common Interview Pitfalls
  • ✕
    Using standard unweighted SVM on 99:1 imbalanced fraud data (model will achieve 99% accuracy by predicting zero fraud).
  • ✕
    Setting CC very large in the presence of label noise (forces SVM to twist the decision boundary to fit mislabeled points).
🎯 Interviewer Follow-up Inquiries
  • ?
    How does the ν\nu-SVM formulation parameterize the fraction of margin errors using parameter ν∈(0,1)\nu \in (0, 1)?
  • ?
    Why does Support Vector Data Description (SVDD / One-Class SVM) excel at extreme anomaly detection?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM2-039: Compare Linear, Polynomial, and RBF Kernels and Their Decision Boundaries📋 Back to BankNext →M2-041: Formulate Naive Bayes Classification Decision Rule and the Exact Nature of Its 'Naive' Assumption