Research map / Label-efficient and robust learning / Recognition and long-tail
Learning to Help in Multi-Class Settings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the binary Learning to Help (L2H) framework to multi-class classification, proposing a differentiable, convex stage-switching surrogate loss with a consistency guarantee and algorithms for pay-per-request, intermittent availability, and bounded reject rate settings.
Problem
Training a server model and a rejector (deferral rule) for a fixed legacy on-device multi-class classifier, under costs per request, intermittent server availability, or bounded reject rates, where the generalized 0-1 loss is non-differentiable.
Approach
A stage-switching surrogate loss that sums a cross-entropy loss for the server classifier and a rejector loss that depends on the fixed client's prediction and the server's correctness; it is differentiable, convex (or monotone), and consistent with the Bayes rule of the multi-class generalized 0-1 loss, enabling synchronous or asynchronous training.
Key results
- On CIFAR-10 with c_1=1.25 and c_e=0.25, the rejector sends 55.9% of samples to the server, where the server (e) accuracy is 67.7% versus 54.5% for the client on those samples.
- On SVHN, the rejected subset (8.29% of data) has server accuracy 72.8% versus client accuracy 61.2%.
- Experiments show that adding a server classifier with the rejector increases overall accuracy across the PPR, IA, and BRR settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the server classifier instead of the client classifier on samples routed local | L2H with LeNet-5 client and AlexNet server · CIFAR-10 test, r(x)=local | accuracy: 73.9 -> 81.9; 8.0 | Helped |
| Use the server classifier instead of the client classifier on samples routed remote | L2H with LeNet-5 client and AlexNet server · CIFAR-10 test, r(x)=remote | accuracy: 54.5 -> 67.7; 13.2 | Helped |
| Use the server classifier instead of the client classifier on samples routed local | L2H with LeNet-5 client and AlexNet server · SVHN test, r(x)=local | accuracy: 90.6 -> 93.3; 2.7 | Helped |
| Use the server classifier instead of the client classifier on samples routed remote | L2H with LeNet-5 client and AlexNet server · SVHN test, r(x)=remote | accuracy: 61.2 -> 72.8; 11.6 | Helped |
| Add a server classifier and rejector to the fixed local model | Multi-class L2H · CIFAR-10 and SVHN | overall accuracy | Helped |
| Increase synchronization interval S in asynchronous training | Multi-class L2H asynchronous training · Training convergence | convergence rate | Hurt |
| Increase synchronization interval S in asynchronous training | Multi-class L2H asynchronous training · Training convergence | converged loss | No clear effect |
| Use asynchronous IA training instead of synchronized PPR training | Multi-class L2H · CIFAR-10 and SVHN | accuracy | No clear effect |
Limitations
- Client classifier is fixed, so the rejector can only be as good as the legacy model.
- Theoretical results hold over the space of all measurable functions and do not account for finite-sample effects.
Experimental setup
Datasets: CIFAR-10 SVHN CIFAR-100
Benchmarks: CIFAR-10 SVHN CIFAR-100
Metrics: accuracy reject rate
Baselines: Confidence-LWA Separate rejector-LWA Softmax-L2D OvA-L2D Two-stage L2D Random rejection (BRR comparison)
Builds on: Learning to Help (binary) Learning to Defer Learning with Abstention
Closest papers by meaning
- Learning with Adaptive Resource Allocation (ICML 2024 · similarity 0.82)
- Learn2Mix: Training Neural Networks Using Adaptive Data Integration (NeurIPS 2025 · similarity 0.82)
- A Two-Stage Learning-to-Defer Approach for Multi-Task Learning (ICML 2025 · similarity 0.82)
- Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training (ICML 2025 · similarity 0.81)
- Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning (AAAI 2024 · similarity 0.81)
- Gatekeeper: Improving Model Cascades Through Confidence Tuning (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.