Research map / Label-efficient and robust learning / Recognition and long-tail
Aligned Objective for Soft-Pseudo-Label Generation in Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEAL, which alternately trains a predictive model and a soft-pseudo-label generator whose objective is a meta-network-parameterized label enhancement loss, and applies it to supervised classification and partial label learning.
Problem
Soft pseudo-label generators are usually trained with classification loss, which is misaligned with the goal of capturing data knowledge for soft labels; manually designing a good objective is hard because ground-truth soft labels are unavailable.
Approach
Learn the objective of the soft-pseudo-label generator with a meta-network, optimized via meta-learning using task-loss feedback from the predictive model's performance, so the generator is aligned with the learning task; the task loss can be swapped for other tasks such as partial label learning.
Key results
- SEAL reaches 91.79% on CIFAR-10, 70.72% on CIFAR-100 and 56.76% on TinyImageNet, the best among compared soft-label methods; on TinyImageNet it beats the second-best method by 1.09%.
- Replacing the meta-network objective with cross-entropy (SEAL-NM) drops accuracy to 91.41%, 68.63% and 55.31% on the three datasets.
- With the partial label learning task loss, SEAL outperforms all seven PLL baselines on every CIFAR-10 and CIFAR-100 corruption setting, e.g., 92.93% vs 92.07% (IDGP) at flipping probability 0.3 on CIFAR-10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SEAL instead of TF-KDself | ResNet-20 · CIFAR-10 | accuracy: 91.66 ± 0.04% -> 91.79 ± 0.11% | Helped |
| Use SEAL instead of TF-KDself | ResNet-44 · CIFAR-100 | accuracy: 70.44 ± 0.26% -> 70.72 ± 0.17% | Helped |
| Use SEAL instead of CS-KD | ResNet-18 · TinyImageNet | accuracy: 55.67 ± 0.58% -> 56.76 ± 0.22% | Helped |
| Use SEAL instead of the strongest baseline in each partial-label setting | ResNet-32 with the PLL task loss · CIFAR-10, flipping probabilities 0.3, 0.5, 0.7, and 0.9 | accuracy: 92.07 0.32 -> 92.93 0.29; 91.04 0.13 -> 91.98 0.28; 89.31 0.39 -> 89.88 0.34; 84.02 1.16 -> 85.36 0.41 | Helped |
| Use SEAL instead of the strongest baseline in each partial-label setting | ResNet-32 with the PLL task loss · CIFAR-100, flipping probabilities 0.03, 0.05, and 0.1 | accuracy: 68.19 0.02 -> 71.27 0.40; 67.68 0.29 -> 68.75 0.67; 65.27 0.18 -> 67.21 1.31 | Helped |
| Add the meta-network-parameterized objective instead of using cross-entropy for the generator objective (ablation) | SEAL · CIFAR-10 | accuracy: 91.41 ± 0.33% -> 91.79 ± 0.11% | Helped |
| Add the meta-network-parameterized objective instead of using cross-entropy for the generator objective (ablation) | SEAL · CIFAR-100 | accuracy: 68.63 ± 0.42% -> 70.72 ± 0.17% | Helped |
| Add the meta-network-parameterized objective instead of using cross-entropy for the generator objective (ablation) | SEAL · TinyImageNet | accuracy: 55.31 ± 0.64% -> 56.76 ± 0.22% | Helped |
Limitations
- Gains on CIFAR-10 over some baselines are small and not always statistically significant (e.g., TF-KD self 91.66 vs SEAL 91.79).
- Theoretical analysis relies on assumptions about extracted features (Appendix A) and a specific prototype setting.
- Meta-learning adds per-iteration bi-level optimization overhead, with cost not quantified in the main text.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet
Metrics: classification accuracy test accuracy under symmetric label noise partial label learning accuracy
Baselines: CE LSR TF-KD self TF-KD reg CS-KD PS-KD DLB USKD SEAL-NM PRODEN CC VALEN
Builds on: label enhancement learning to reweight / meta-network loss parameterization (Bechtle et al., 2020) self-knowledge distillation PRODEN-style progressive identification
Closest papers by meaning
- Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling (AAAI 2025 · similarity 0.85)
- Dirichlet-Based Prediction Calibration for Learning with Noisy Labels (AAAI 2024 · similarity 0.85)
- Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling (NeurIPS 2026 · similarity 0.85)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.85)
- Reduction-based Pseudo-label Generation for Instance-dependent Partial Label Learning (NeurIPS 2025 · similarity 0.84)
- $\epsilon$-Softmax: Approximating One-Hot Vectors for Mitigating Label Noise (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.