Research map / Label-efficient and robust learning / Semi-supervised learning

Re-Evaluating the Impact of Unseen-Class Unlabeled Data on Semi-Supervised Learning Model

ICLR 2025 · Label-efficient and robust learning › Semi-supervised learning · approach: Safe and realistic semi-supervised learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Re-evaluates how unseen-class unlabeled data affects semi-supervised learning by holding seen-class unlabeled data fixed and varying unseen-class data, introducing the RE-SSL framework with five robustness metrics across several factors.

Problem

Prior safe SSL evaluations fix the unlabeled set size and vary the unseen-class proportion, confounding unseen-class and seen-class sample counts, so the claim that unseen classes harm SSL may be unreliable.

Approach

Fix the seen-class unlabeled ratio r_s and vary only the unseen-class ratio r_u, then summarize accuracy curves with regression-based global metrics (R_slope, GM, P_AD>=0) and local adjacent-discrepancy metrics (WAD, BAD), studied along five factors (sample number, category number, category index, nearness, label distribution).

Key results

• On CIFAR-10 with 100 labels, PseudoLabel and ICT show near-zero R_slope (-0.020 and 0.002) while FixMatch (-0.223) and FreeMatch (-0.287) decline sharply as unseen classes are added.
• UDA on CIFAR-100 has a positive R_slope (0.015) with P_AD>=0 of 0.833, suggesting unseen classes can slightly help it.
• When r_s is decreased with r_u fixed on CIFAR-10, most methods drop significantly, supporting the claim that seen-class quantity confounds prior evaluations.

Measured effects reported in the paper

Surprising findings

• Unseen-class unlabeled data does not necessarily hurt SSL and can in some cases improve seen-class accuracy (e.g., UDA on CIFAR-100).
• Previously reported safe SSL methods such as UASD and ICT are robust across datasets, whereas FixMatch and FreeMatch are highly sensitive to unseen classes.

Limitations

• No theoretical analysis is provided.
• Only considers unseen-class shifts; does not study shifts in feature or label distributions of seen classes in unlabeled data.
• Metrics are computed from a small number of r points (0, 0.2, 0.4, 0.5, 0.6, 0.8, 1.0) with three seeds, so conclusions are sensitive to noise.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 ImageNet-100 CUB MNIST

Benchmarks: CIFAR-10 CIFAR-100 ImageNet-100 CUB

Metrics: accuracy R_slope GM WAD BAD P_AD>=0 p-value

Baselines: Supervised PseudoLabel PiModel ICT VAT MixMatch UDA FixMatch FlexMatch FreeMatch SoftMatch UASD

Builds on: FixMatch MixUp DS3L Safe-Student OpenMatch LAMDA-SSL

Closest papers by meaning

• Robust Semi-Supervised Learning when Not All Classes have Labels (NeurIPS 2022 · similarity 0.91)
• On Non-Random Missing Labels in Semi-Supervised Learning (ICLR 2022 · similarity 0.89)
• CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
• Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization (CVPR 2025 · similarity 0.87)
• InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning (ICML 2024 · similarity 0.87)
• Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Data (NeurIPS 2024 · similarity 0.86)