Research map / Label-efficient and robust learning / Semi-supervised learning
OTMatch: Improving Semi-Supervised Learning with Optimal Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OTMatch, which adds an optimal-transport loss with a class-relationship cost bootstrapped from classifier head vectors on top of FreeMatch to improve semi-supervised learning on vision and language benchmarks.
Problem
Pseudo-labeling semi-supervised methods trained with cross-entropy align each pseudo-labeled sample to a single class, ignoring inter-class relationships, and suffer from overconfident and wrong pseudo-labels, especially with very few labels.
Approach
Views FreeMatch-style pseudo-labeling as matching semantic distributions via (inverse) optimal transport, and adds an OT loss whose cost matrix is updated by EMA from normalized classification-head vector similarities, so the loss encodes inter-class relationships; the OT cost reduces to an O(K) computation under a lemma.
Key results
- On CIFAR-10 with 4 labels per class, OTMatch reaches 4.89% error vs. 8.07% for FreeMatch (lowest error in the table).
- On ImageNet with 100 labels per class, OTMatch achieves 39.29% top-1 error vs. 40.57% for FreeMatch.
- On Amazon Review (250 labels), OTMatch achieves 43.81% error vs. 45.75% for FlexMatch; per-iteration time 0.25 s vs. 0.23 s for FreeMatch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · CIFAR-10, 10 labels | error rate: 8.07{ $\pm$ 4.24} -> 4.89{ $\pm$ 0.76} | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · CIFAR-10, 40 labels | error rate: 4.90{ $\pm$0.04} -> 4.72{ $\pm$ 0.08} | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · CIFAR-10, 250 labels | error rate: 4.88{ $\pm$0.18} -> 4.60{ $\pm$ 0.15} | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · CIFAR-100, 400 labels | error rate: 37.98{ $\pm$0.42} -> 37.29{ $\pm$ 0.76} | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · CIFAR-100, 2500 labels | error rate: 26.03{ $\pm$0.35} -> 26.04{ $\pm$ 0.21} | Hurt |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · STL-10, 40 labels | error rate: 15.56{ $\pm$0.55} -> 12.10{ $\pm$ 0.72} | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · STL-10, 1000 labels | error rate: 5.63{ $\pm$0.15} -> 5.60{ $\pm$ 0.14} | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · ImageNet, 100 labels per class | Top-1 error rate: 40.57 -> 39.29 | Helped |
| Add the OT loss with learned class-semantic cost to FreeMatch | FreeMatch · ImageNet, 100 labels per class | Top-5 error rate: 18.77 -> 17.77 | Helped |
| Add the OT loss to FlexMatch | FlexMatch · Amazon Review, 250 labels | error rate: 45.75 -> 43.81 | Helped |
| Add the OT loss to FlexMatch | FlexMatch · Amazon Review, 1000 labels | error rate: 43.14 -> 42.35 | Helped |
| Add the OT loss to DINO | DINO · CIFAR-10 | accuracy: 88.06 -> 88.20 | Helped |
Surprising findings
- The OT cost matrix learned from the model's classifier head recovers semantically sensible hierarchies (e.g., vehicles and animals cluster separately on CIFAR-10).
- A simple binary cost gives notably worse error (5.20%) than the learned cost (4.72%) on CIFAR-10 with 4 labels per class.
Limitations
- The O(K) computation relies on a lemma holding under mild conditions that may not always apply.
- Gains on several settings are small (e.g., CIFAR-100 with 400 labels: 37.29 vs 37.98 error, and CIFAR-100 2500 labels is second best), and standard deviations overlap in places.
- Hyperparameter lambda is tuned per dataset (0.5, 0.15, 0.01).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 STL-10 ImageNet Amazon Review Yelp Review
Benchmarks: CIFAR-10 CIFAR-100 STL-10 ImageNet Amazon Review Yelp Review
Metrics: error rate (100% - accuracy) top-1 error top-5 error
Baselines: FreeMatch FlexMatch FixMatch Dash MPL UDA ReMixMatch MixMatch
Builds on: FreeMatch FixMatch inverse optimal transport Sinkhorn algorithm
Closest papers by meaning
- OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning (NeurIPS 2024 · similarity 0.87)
- Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data (AAAI 2024 · similarity 0.86)
- InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning (ICML 2024 · similarity 0.86)
- Recover and Match: Open-Vocabulary Multi-Label Recognition through Knowledge-Constrained Optimal Transport (CVPR 2025 · similarity 0.85)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.85)
- Revisiting Semi-Supervised Learning in the Era of Foundation Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.