Research map / Label-efficient and robust learning / Noisy labels
TSVC: Tripartite Learning with Semantic Variation Consistency for Robust Image-Text Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TSVC, a tripartite (Coordinator, Master, Assistant) co-training framework with mutual-information-based soft label estimation (SIVC) and a distribution-adaptive soft margin loss (DASM) to make image-text retrieval robust to noisy correspondence.
Problem
Cross-modal image-text retrieval degrades when training pairs are misaligned (noisy correspondence), and co-teaching with identical architectures yields homogeneous models with limited extra information.
Approach
Replaces two-network co-teaching with a three-model tripartite scheme (Coordinator partitions data, Assistant filters noisy-set samples for Master) and estimates soft correspondence labels from variations in histogram-based mutual information across pairs, images and texts relative to an anchor pair; a DASM loss sets a margin from the soft label and distance to the clean-loss center.
Key results
- On Flickr30K with 40% simulated noise, TSVC reaches Rsum 500.3 versus 489.6 for the best noise-robust baseline ESC (the paper cites a 10.7 gain).
- On CC152K real-world noise, TSVC reaches Rsum 380.1, outperforming ESC (375.9) by 4.2.
- On MSCOCO 1K with 60% noise, TSVC reaches Rsum 519.3 versus 515.0 for ESC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SIVC soft-label estimation instead of hard labels (ablation) | TSVC · Flickr30K, 40% noise | Image-Text R@1: 74.8 -> 77.7 | Helped |
| Use DASM instead of regular soft-margin triplet loss (ablation) | TSVC · Flickr30K, 40% noise | Image-Text R@1: 75.6 -> 77.7 | Helped |
| Use Tri-Learning instead of Co-teaching (ablation) | TSVC · Flickr30K, 40% noise | Image-Text R@1: 75.0 -> 77.7 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · Flickr30K, 20% noise | Rsum: +6.5 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · Flickr30K, 40% noise | Rsum: +10.7 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · Flickr30K, 60% noise | Rsum: +4.8 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · MSCOCO-1K, 20% noise | Rsum: +3.6 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · MSCOCO-1K, 40% noise | Rsum: +4.0 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · MSCOCO-1K, 60% noise | Rsum: +4.3 | Helped |
| Use TSVC rather than ESC | Image-text retrieval · CC152K | Rsum: 4.2 | Helped |
Surprising findings
- Tri-learning initially performs worse than Co-teaching under high noise ratios during training, attributed to misclassifying noisy samples as clean in the re-division stage.
Limitations
- Tri-learning may misclassify noisy samples as clean during re-division, hurting early training under high noise.
- Hyperparameter sensitivity: performance depends on the filtering threshold delta and soft-margin parameter m.
Experimental setup
Datasets: Flickr30K MSCOCO CC152K
Benchmarks: Flickr30K MSCOCO 1K CC152K
Metrics: R@1 R@5 R@10 Rsum
Baselines: SCAN IMRAM SAF SGR NCR MSCN BiCro ESC
Builds on: co-teaching NCR BiCro ESC
Closest papers by meaning
- PCSR: Pseudo-label Consistency-Guided Sample Refinement for Noisy Correspondence Learning (AAAI 2026 · similarity 0.88)
- Towards Robust Uncertainty Calibration for Composed Image Retrieval (NeurIPS 2025 · similarity 0.86)
- STiTch: Semantic Transition and Transportation in Collaboration for Training-Free Zero-Shot Composed Image Retrieval (CVPR 2026 · similarity 0.86)
- RoDA: Robust Domain Alignment for Cross-Domain Retrieval Against Label Noise (AAAI 2025 · similarity 0.86)
- Rethinking Noisy Video-Text Retrieval via Relation-aware Alignment (CVPR 2025 · similarity 0.86)
- Cross-Modal and Uni-Modal Soft-Label Alignment for Image-Text Retrieval (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.