Research map / Label-efficient and robust learning / Noisy labels
Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SREM, a noisy-correspondence learning framework for image-text retrieval that filters clean pairs via energy uncertainty, weights gradients by swapped classification entropy, and adds cross-modal biased complementary learning, improving robustness without extra clean data or ensembles.
Problem
Cross-modal (image-text) retrieval trained on web-harvested pairs suffers from noisy correspondences (mismatched pairs); existing memorization-based methods rely on similarity-guided hard-negative training that causes self-reinforcing errors.
Approach
Views sample matching as in-batch classification, uses energy uncertainty over classification logits for clean-sample selection, swapped classification entropy to weight gradients, and complementary-label learning over overlooked negatives, with a biased (similarity-weighted) selection of complementary labels.
Key results
- On Flickr30K with 20% synthetic noise, SREM reaches R_sum 507.8 vs 504.7 for BiCro and 502.1 for MSCN.
- On CC152K (real-world noise), SREM reaches R_sum 372.2, above BiCro (370.2) and MSCN (366.7).
- SREM reduces per-epoch training time by more than 40% relative to MSCN and BiCro, per the efficiency table (1506 s training vs 6344 s and 3093 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SREM instead of BiCro | SGRAF · Flickr30K 60% noise | R_sum: 456.6 -> 480.3 | Helped |
| Use SREM instead of the strongest baseline, MSCN | SGRAF · MS-COCO 20% noise | R_sum: 524.6 -> 524.1 | Hurt |
| Use SREM instead of BiCro | SGRAF · CC152K | R_sum: 370.2 -> 372.2 | Helped |
| Add sample filtration (ablation) | Vanilla model on CC152K · CC152K | image-to-text R@1: 32.5 -> 37.3 | Helped |
| Add swapped gradient weighting to sample filtration (ablation) | CC152K model with sample filtration · CC152K | image-to-text R@1: 37.3 -> 40.2 | Helped |
| Add swapped gradient weighting to sample filtration (ablation) | CC152K model with sample filtration · CC152K | image-to-text R@5: 63.7 -> 63.2 | Hurt |
| Add complementary learning (ablation) | CC152K model with sample filtration and gradient weighting · CC152K | image-to-text R@5: 63.2 -> 67.3 | Helped |
| Add label rectification to complementary learning (ablation) | CC152K model with sample filtration, gradient weighting, and complementary learning · CC152K | image-to-text R@1: 40.5 -> 40.9 | Helped |
| Add label rectification to complementary learning (ablation) | CC152K model with sample filtration, gradient weighting, and complementary learning · CC152K | text-to-image R@1: 42.5 -> 41.5 | Hurt |
| Use SREM instead of BiCro | SGRAF · CC152K | training time per epoch: 3093 -> 1506 | Helped |
Surprising findings
- Similarity-guided hard-negative training with MSCN and BiCro shows rising noisy-gradient ratios that correlate with performance decline during training, supporting the self-reinforcing error hypothesis.
- Under 60% noise on MS-COCO, prior methods such as NCR and SAF collapse to near-zero R_sum, while SREM retains performance.
Limitations
- Relies on synthetic shuffling noise for most experiments; real-world evaluation is limited to CC152K.
- Several hyperparameters (margins, tau, lambda1, lambda2, beta, b) must be tuned.
Experimental setup
Datasets: MS-COCO Flickr30K CC152K
Benchmarks: Flickr30K MS-COCO 1K CC152K
Metrics: R@1 R@5 R@10 R_sum training time per epoch GPU memory
Baselines: SCAN VSRN IMRAM SAF SGR NCR DECL BiCro MSCN RCL
Builds on: SGRAF RCL DECL NCR BiCro
Closest papers by meaning
- Negative Pre-aware for Noisy Cross-Modal Matching (AAAI 2024 · similarity 0.89)
- Noisy Correspondence Rectification via Asymmetric Similarity Learning (AAAI 2025 · similarity 0.89)
- Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval (CVPR 2024 · similarity 0.88)
- Robust Noisy Correspondence Learning with Equivariant Similarity Consistency (CVPR 2024 · similarity 0.88)
- ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning (CVPR 2025 · similarity 0.86)
- Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.