Research map / Label-efficient and robust learning / Semi-supervised learning
Contrastive Credibility Propagation for Reliable Semi-supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Contrastive Credibility Propagation (CCP), a transductive pseudo-label refinement SSL method using credibility vectors and a generalized contrastive loss, evaluated across five data-quality scenarios on CIFAR-10/100 and text datasets to reliably beat the supervised baseline.
Problem
Semi-supervised learning often fails to outperform a supervised baseline under real-world data issues such as few labels, open-set unlabeled data, label noise, and class distribution imbalance or misalignment.
Approach
Iteratively refines pseudo-labels transductively via credibility vectors (similarity to a class minus highest similarity to other classes) and a soft generalized contrastive loss over non-discrete positive pairs, averages pseudo-labels across epochs to correct noise (SEAL-style), and uses KL-divergence-constrained subsampling to reset unreliable pseudo-labels.
Key results
- On CIFAR-10 CCP never drops below 90% in any data-variable scenario and is the only method that outperforms the supervised baseline in every experiment.
- On text datasets, CCP outperforms or matches the baseline in every few-label experiment, e.g. 87.04% vs 78.37% baseline on AG News at 0.1% labels.
- CCP's base-case CIFAR-10 accuracy is 96.85% vs 97.00% for the supervised-baseline-level comparisons; CCP trails FixMatch and others in label efficiency in the base case.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use credibility adjustments instead of softmax labels, omitting scaling, clipping, and subsampling in the softmax variant (ablation) | CCP iterations · CIFAR-10 base case and few-label experiments | pseudo-label accuracy | Helped |
| Add D_KL-based subsampling between CCP iterations (ablation) | CCP · High-severity data-variable experiments | pseudo-label accuracy | Helped |
| Add D_KL-based subsampling between CCP iterations (ablation) | CCP · CIFAR-10 base case | pseudo-label accuracy | No clear effect |
| Add D_KL-based subsampling between CCP iterations (ablation) | CCP · AG News, 0.05% labels, iteration 7 | pseudo-label accuracy: 82.72% -> 82.98% | Helped |
| Add D_KL-based subsampling between CCP iterations (ablation) | CCP · Rotten Tomatoes, 0.1% labels | pseudo-label accuracy | Mixed |
| Add D_KL-based subsampling between CCP iterations (ablation) | CCP · Text classification datasets, few-label experiments | pseudo-label accuracy | Mixed |
| Minimize both contrastive loss and classifier loss during classifier training instead of classifier loss alone (ablation) | CCP classifier training · Experiments reported in the paper | performance | Helped |
| Reset the encoder and contrastive head after the final CCP iteration instead of reusing their last-iteration state (ablation) | CCP classifier training · Experiments reported in the paper | performance | Helped |
| Subsample final pseudo-labels during classifier training (ablation) | CCP classifier training · Test set | accuracy | No clear effect |
| Use CCP instead of the supervised baseline | Semi-supervised learning · CIFAR-10, noisy-label 60% | accuracy: 58.60% -> 94.28% | Helped |
| Use CCP instead of the supervised baseline | Semi-supervised learning · CIFAR-100, noisy-label 60% | accuracy: 35.70% -> 71.08% | Helped |
| Use CCP instead of the baseline | Text classification with few labels · AG News, DBpedia, and Yahoo! Answers at 0.05% labels | accuracy | Helped |
Surprising findings
- Softmax-based pseudo-labels cause quick and catastrophic degradation to maximum-entropy pseudo-labels, whereas credibility vectors better separate correct and incorrect pseudo-labels.
- Credibility-based SSL methods that build on FixMatch-style thresholds handle open-set well, and all algorithms except CCP build on FixMatch.
Limitations
- CCP does not achieve the best accuracy in every scenario, particularly in label efficiency (base case and few-label on CIFAR).
- Computationally expensive with many CCP iterations; subsampling schedule may need tuning if unstable.
- Open-set experiments use only a single CCP iteration due to increased confidence in OOD samples with more iterations.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 AG News DBpedia Rotten Tomatoes Yahoo! Answers
Benchmarks: CIFAR-10 CIFAR-100 AG News DBpedia Rotten Tomatoes Yahoo! Answers
Metrics: test accuracy pseudo-label accuracy
Baselines: CoMatch OpenMatch ACR FixMatch SimCLR SupCon CCLP
Builds on: SEAL SimCLR SupCon FixMatch CCLP
Closest papers by meaning
- Class-Aware Contrastive Semi-Supervised Learning (CVPR 2022 · similarity 0.89)
- DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples (NeurIPS 2021 · similarity 0.89)
- Don’t fear the unlabelled: safe semi-supervised learning via debiasing (ICLR 2023 · similarity 0.89)
- Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition (NeurIPS 2024 · similarity 0.88)
- Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025 · similarity 0.88)
- RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.