Research map / Label-efficient and robust learning / Semi-supervised learning

Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling

NeurIPS 2026 · Label-efficient and robust learning › Semi-supervised learning · approach: Confidence-based pseudo-label selection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SemiPrune, which uses FixMatch pseudo-labels trained on a 10% labeled subset, then applies supervised pruning scores (AUM with cutoff, DUAL with Beta sampling) on the pseudo-labeled pool to select coresets.

Problem

Dataset pruning typically requires fully labeled data, and label-free methods relying on pretrained features degrade when the target data differs from the pretraining distribution (domain shift, corruption, class imbalance).

Approach

Replace pretrained-feature pseudo-labels (deep clustering) with SSL (FixMatch) pseudo-labels trained on the target dataset from a small random labeled subset, then reuse existing supervised pruning scores (AUM, DUAL) on those pseudo-labels, with a modified DUAL exponent gamma.

Key results

• On Food-101 at 90% pruning, Semi-DUAL+Beta reaches 51.8% vs 37.7% for Random and 49.2% for ELFS (DINO), using 10% labels.
• On 30% corrupted CIFAR-100, Semi-DUAL+Beta achieves 73.3% at 30% pruning and 47.5% at 90%, outperforming ELFS (DINO) at high pruning ratios.
• SSL pseudo-labels reach 65.4% accuracy on CIFAR-100-C vs 40.3% for deep clustering (TEMI), per the pseudo-label quality table.

Surprising findings

• Score Extrapolation with 10% labels underperforms label-free ELFS (DINO) on several datasets, and at high pruning on corrupted CIFAR-100 it collapses to 12.9% at 90% pruning.
• Prototypicality and ZCore can fall below random selection when pretrained features are poorly aligned with the target data.

Limitations

• Relies on FixMatch as the SSL algorithm; reliability of pseudo-labels for other SSL methods is not explored.
• Initial labeled subset is chosen randomly; no active selection of labeled examples.
• Extra hyperparameters (cutoff ratio, T, c_D, gamma) are tuned per dataset using pseudo-labeled validation, and some gains are small or mixed on standard benchmarks.

Experimental setup

Datasets: Food-101 SUN397 CIFAR-100-C Tiny-ImageNet-C CIFAR-100-LT Caltech-101 CIFAR-10 CIFAR-100 ImageNet-1K

Benchmarks: Food-101 SUN397 CIFAR-100-C Tiny-ImageNet-C CIFAR-100-LT Caltech-101 CIFAR-10 CIFAR-100 ImageNet-1K

Metrics: test accuracy of coreset-trained model pseudo-label accuracy ARI NMI balanced accuracy

Baselines: Random Prototypicality ZCore ELFS (DINO) ELFS (Self-Encoder) Score Extrapolation DUAL+Beta

Builds on: ELFS DUAL AUM FixMatch Score Extrapolation

Closest papers by meaning

• Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty (ICML 2025 · similarity 0.90)
• Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.88)
• Exploring Learning Complexity for Efficient Downstream Dataset Pruning (ICLR 2025 · similarity 0.88)
• Distilling the Knowledge in Data Pruning (ICML 2025 · similarity 0.87)
• Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR 2024 · similarity 0.87)
• CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)