Research map / Vision-language and multimodal / Cross-modal learning

ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning

CVPR 2025 · Vision-language and multimodal › Cross-modal learning · approach: Unimodal-to-multimodal representation transfer · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReCon, a noisy correspondence learning framework that adds intra-modal relation consistency (matching object-relation matrices across modalities via a proxy) to cross-modal InfoNCE and uses it to divide noisy data into clean, local-correspondence, and noisy partitions for robust image-text retrieval.

Problem

Noisy (mismatched) image-text pairs in cross-modal retrieval training cause hard negatives with high similarity to be mistaken as positives, degrading retrieval performance.

Approach

Combine cross-modal relation consistency with intra-modal relation consistency (aligning relation matrices of one modality with a proxy reconstructed from the other) to discriminate true correspondences, then partition and reweight data accordingly, with pseudo-label refinement for the noisy partition.

Key results

• On CC152K (real-world noise), ReCon reaches rSum 380.5 versus 375.9 for the second-best ESC.
• On Flickr30K with 40% simulated noise, ReCon achieves rSum 505.2, and on MS-COCO 1K with 40% noise it achieves 524.5 (per Table 2).
• On MS-COCO 5K with 50% noise, ReCon (rSum 428.3) surpasses zero-shot CLIP-32 (rSum 361.6 at 0% noise) and CLIP fine-tuned under 50% noise (131.1), per the reported table.

Measured effects reported in the paper

Surprising findings

• Under 50% noise on MS-COCO 5K, ReCon's rSum (428.3) exceeds CLIP-32 fine-tuned with 0% noise (361.6) in the paper's comparison table, though the comparison setups differ.

Limitations

• Relies on small-loss GMM division and hand-tuned thresholds (omega1, omega2 both 0.5) and scale alpha, sensitivity shown only on Flickr30K
• Intra-modal relation needs a proxy reconstructed via cross-modal argmax matching because object annotations are unavailable
• Real-world CC152K gains are modest relative to strongest baselines in some metrics

Experimental setup

Datasets: Flickr30K MS-COCO Conceptual Captions (CC152K)

Benchmarks: Flickr30K MS-COCO 1K MS-COCO 5K CC152K

Metrics: R@1 R@5 R@10 rSum

Baselines: SCAN NCR DECL MSCN BiCro RCL CRCL SREM PC2 L2RM ESC GSC

Builds on: SGRAF NCR IAIS small-loss criterion InfoNCE

Closest papers by meaning

• Noisy Correspondence Rectification via Asymmetric Similarity Learning (AAAI 2025 · similarity 0.90)
• Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning (CVPR 2024 · similarity 0.87)
• Robust Noisy Correspondence Learning with Equivariant Similarity Consistency (CVPR 2024 · similarity 0.86)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation (AAAI 2024 · similarity 0.86)
• Reconstruction Alignment Improves Unified Multimodal Models (ICLR 2026 · similarity 0.85)
• On the Value of Cross-Modal Misalignment in Multimodal Representation Learning (NeurIPS 2025 · similarity 0.84)