Research map / Label-efficient and robust learning / Recognition and long-tail
Multi-Class Support Vector Machine with Maximizing Minimum Margin
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
Multi-class SVM extensions (OvR, OvO, unified formulations) fail to maximize the margin between every class pair simultaneously, and many deviate from the margin principle, hurting generalization.
Approach
Formulate multi-class SVM as maximizing the minimum pairwise margin, approximate the max-min with a p-norm style surrogate (Theorem 1 gives sum of ||wk-wl||_2^p), yielding a convex smooth loss over all class pairs that generalizes Crammer-Singer and Weston-Watkins (recovered at p=2) and can replace softmax in neural networks.
Key results
- M3SVM achieves the best test accuracy on all eight traditional datasets, e.g., 0.865 on Cornell vs 0.845 for the best OvO baseline and 0.744 on Glass vs 0.685 for OvO.
- Test accuracy peaks around p≈4 in the tested range [1,8]; p=2 (Crammer-Singer-like regularizer) is not optimal across datasets.
- Applied as a softmax replacement on SVHN, the proposed loss reportedly mitigates overfitting in training/test accuracy curves.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use M3SVM instead of OvO | Multi-class classification · Cornell | accuracy: 0.845 ± 0.028 -> 0.865 ± 0.013 | Helped |
| Use M3SVM instead of OvO | Multi-class classification · ISOLET | accuracy: 0.942 ± 0.004 -> 0.945 ± 0.002 | Helped |
| Use M3SVM instead of Top-k | Multi-class classification · HHAR | accuracy: 0.970 ± 0.007 -> 0.981 ± 0.004 | Helped |
| Use M3SVM instead of SMLR | Multi-class classification · USPS | accuracy: 0.937 ± 0.004 -> 0.956 ± 0.011 | Helped |
| Use M3SVM instead of OvO | Multi-class classification · ORL | accuracy: 0.975 ± 0.000 -> 0.975 ± 0.000 | No clear effect |
| Use M3SVM instead of OvO | Multi-class classification · Dermatology | accuracy: 0.971 ± 0.003 -> 0.988 ± 0.001 | Helped |
| Use M3SVM instead of OvR | Multi-class classification · Vehicle | accuracy: 0.794 ± 0.016 -> 0.800 ± 0.011 | Helped |
| Use M3SVM instead of OvO | Multi-class classification · Glass | accuracy: 0.685 ± 0.008 -> 0.744 ± 0.007 | Helped |
| Increase the margin regularizer exponent p (ablation) | M3SVM · Six datasets shown in Figure 4 | test accuracy | Mixed |
| Vary the trade-off parameter λ around p = 4 (ablation) | M3SVM · Eight datasets | test accuracy | No clear effect |
Surprising findings
- The p=2 special case (equivalent to existing l2-regularized formulations) is not optimal across a wide range of datasets.
Limitations
- Sensitive to hyperparameters p and lambda; excessively large p leads to poor convergence.
- Evaluated mainly on linear models trained in original feature space; deep-learning results are limited to one dataset.
- Only test accuracy reported; no class-imbalance evaluation.
Experimental setup
Datasets: Cornell ISOLET HHAR USPS ORL Dermatology Vehicle Glass SVHN
Benchmarks: Cornell ISOLET HHAR USPS ORL Dermatology Vehicle Glass
Metrics: test accuracy
Baselines: OvR OvO Crammer-Singer M-SVM Top-k SVM Multi-LR SMLR
Builds on: soft-margin SVM Weston-Watkins multi-class SVM Crammer-Singer multi-class SVM Bredensteiner-Bennett M-SVM
Closest papers by meaning
- Multi-Class Optimal Margin Distribution Machine (ICML 2017 · similarity 0.89)
- Weston-Watkins Hinge Loss and Ordered Partitions (NeurIPS 2020 · similarity 0.87)
- Near-Tight Margin-Based Generalization Bounds for Support Vector Machines (ICML 2020 · similarity 0.86)
- Consistent Structured Prediction with Max-Min Margin Markov Networks (ICML 2020 · similarity 0.86)
- Multi-Class Support Vector Machine with Differential Privacy (NeurIPS 2025 · similarity 0.85)
- Deep Principal Support Vector Machines for Nonlinear Sufficient Dimension Reduction (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.