Research map / Vision-language and multimodal / Cross-modal learning
Noisy Correspondence Rectification via Asymmetric Similarity Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Asymmetric Similarity Learning (ASL), which decouples positive and negative pair optimization in triplet-based image-text matching and uses VBGMM-estimated soft boundaries to rectify noisy correspondence.
Problem
Image-text matching trained on web-collected data with mismatched pairs (noisy correspondence); symmetric triplet losses with soft margins under-optimize negatives and positives under noise.
Approach
An asymmetric dynamic matching loss (adapted from asymmetric loss ideas) that penalizes potentially noisy positive pairs via a self-paced soft upper boundary without affecting negative-pair gradients, combined with VBGMM-based per-sample loss modeling for soft label estimation.
Key results
- On Flickr30K with 20% synthetic noise, ASL reaches rSum 503.8 vs NCR 496.7 and MSCN 500.0.
- On Flickr30K with 60% noise, ASL achieves rSum 485.1 vs L2RM 467.6 and CRCL 471.5 (Table 1).
- On real-noise CC152K, ASL reaches rSum 367.9, above MSCN 366.7 and L2RM 364.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ASL instead of CRCL | SGR image-text matching · Flickr30K, 60% noise | rSum: 471.5 -> 485.1 | Helped |
| use ASL instead of MSCN | SGR image-text matching · MS-COCO 1K, 60% noise | rSum: 505.2 -> 514.2 | Helped |
| use ASL instead of MSCN | SGR image-text matching · CC152K | rSum: 366.7 -> 367.9 | Helped |
| remove warmup training (ablation) | ASL · Flickr30K, 60% noise, Image→Text | R@1: 73.9 -> 13.9 | Hurt |
| remove warmup training (ablation) | ASL · Flickr30K, 60% noise, Text→Image | R@1: 54.8 -> 11.0 | Hurt |
| replace VBGMM with GMM (ablation) | ASL with warmup and asymmetric loss · Flickr30K, 60% noise, Image→Text | R@1: 73.9 -> 71.3 | Hurt |
| replace VBGMM with GMM (ablation) | ASL with warmup and asymmetric loss · Flickr30K, 60% noise, Text→Image | R@1: 54.8 -> 52.5 | Hurt |
| replace asymmetric loss with triplet loss (ablation) | ASL with warmup and VBGMM · Flickr30K, 60% noise, Image→Text | R@1: 73.9 -> 70.6 | Hurt |
| replace asymmetric loss with triplet loss (ablation) | ASL with warmup and VBGMM · Flickr30K, 60% noise, Text→Image | R@1: 54.8 -> 51.2 | Hurt |
| replace VBGMM with GMM and asymmetric loss with triplet loss (ablation) | ASL with warmup · Flickr30K, 60% noise, Image→Text | R@1: 73.9 -> 16.4 | Hurt |
| replace VBGMM with GMM and asymmetric loss with triplet loss (ablation) | ASL with warmup · Flickr30K, 60% noise, Text→Image | R@1: 54.8 -> 13.4 | Hurt |
| vary relaxation factor z (best reported setting z = 3) (ablation) | ASL · Flickr30K, 60% noise | rSum | Mixed |
Surprising findings
- Several baselines (e.g., SGR, SAF) collapse under 40-60% synthetic noise, while the noisy-correspondence methods remain stable.
Limitations
- Gains on CC152K real noise are modest (e.g., 367.9 vs 366.7 for MSCN), and at 20% noise ASL trails some baselines on certain R@K metrics.
- Hyperparameters (lambda, z) show sensitivity; larger lambda becomes unstable.
Experimental setup
Datasets: Flickr30K MS-COCO CC152K
Benchmarks: Flickr30K MS-COCO 1K CC152K
Metrics: R@1 R@5 R@10 rSum
Baselines: SCAN SGR SAF NCR DECL RCL MSCN BiCro CRCL L2RM
Builds on: NCR SGR asymmetric loss triplet loss VSE++
Closest papers by meaning
- Robust Noisy Correspondence Learning with Equivariant Similarity Consistency (CVPR 2024 · similarity 0.91)
- ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning (CVPR 2025 · similarity 0.90)
- Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation (AAAI 2024 · similarity 0.89)
- Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval (CVPR 2024 · similarity 0.88)
- Negative Pre-aware for Noisy Cross-Modal Matching (AAAI 2024 · similarity 0.88)
- On the Value of Cross-Modal Misalignment in Multimodal Representation Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.