Research map / Label-efficient and robust learning / Noisy labels
Learning with Noisy Triplet Correspondence for Composed Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TME, a framework that makes composed image retrieval robust to noisy ref-mod-tar triplet correspondence via GMM sample selection, pseudo-text adapters from visual differences, and a learnable task-oriented prompt replacing the reference image.
Problem
Composed image retrieval training assumes reference, modification, and target triplets are accurately associated, but annotation and web-scraping produce partially or completely mismatched triplets (noisy triplet correspondence).
Approach
Reframes triplet noise as an adapter mismatch problem: visual differences between reference and target are projected into pseudo-text tokens aligned to the modification, while clean samples (chosen by GMM loss-based selection) train a reference-independent prompt+modification query to reduce overfitting from partially matched pairs.
Key results
- On CIRR at 20% noise, TME reaches Avg 79.74 versus 76.98 for the best baseline SPRC.
- On CIRR at 80% noise, TME reaches Avg 74.58 versus 60.74 for SPRC and 73.18 for RCL.
- On FashionIQ at 80% noise, TME gets average R@10/R@50 of 57.63 versus 54.16 for RCL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TME instead of SPRC | CIR retrieval · CIRR test, 0% noise | Avg(R@5, Rsubset@1): 81.39 -> 82.01 | Helped |
| Use TME instead of RCL | CIR retrieval · CIRR test, 20% noise | Avg(R@5, Rsubset@1): 79.31 -> 79.74 | Helped |
| Use TME instead of RCL | CIR retrieval · CIRR test, 50% noise | Avg(R@5, Rsubset@1): 76.52 -> 77.71 | Helped |
| Use TME instead of RCL | CIR retrieval · CIRR test, 80% noise | Avg(R@5, Rsubset@1): 73.18 -> 74.58 | Helped |
| Use TME instead of SPRC | FashionIQ retrieval · FashionIQ validation, 0% noise | AVG.: 64.85 -> 65.09 | Helped |
| Use TME instead of RCL | FashionIQ retrieval · FashionIQ validation, 20% noise | AVG.: 62.25 -> 63.97 | Helped |
| Use TME instead of RCL | FashionIQ retrieval · FashionIQ validation, 50% noise | AVG.: 59.42 -> 61.86 | Helped |
| Use TME instead of RCL | FashionIQ retrieval · FashionIQ validation, 80% noise | AVG.: 54.16 -> 57.63 | Helped |
| Add GMM filtering (ablation) | RFQ on CIRR · CIRR validation, 20% noise | Avg(R@5, Rsubset@1): 78.75 -> 79.69 | Helped |
| Add complementary contrastive loss (CCL) (ablation) | RFQ with GMM filtering on CIRR · CIRR validation, 20% noise | Avg(R@5, Rsubset@1): 79.69 -> 80.50 | Helped |
| Add PTE (ablation) | RFQ with GMM filtering and CCL on CIRR · CIRR validation, 20% noise | R@5: 82.64 -> 83.01 | Helped |
| Add PTE (ablation) | RFQ with GMM filtering, CCL, and TOP on CIRR · CIRR validation, 20% noise | R@5: 83.02 -> 83.27 | Helped |
Surprising findings
- Even at 0% noise, the proposed modules improve over SPRC on CIRR and FashionIQ, which the authors attribute to inherent partial matching in the datasets.
Limitations
- Sample selection may keep noisy triplets in the clean set, since partially matched triplets are mild noise.
- Hyperparameters alpha, beta, gamma need tuning and performance is sensitive to them.
- Evaluated only on two domain-specific datasets (FashionIQ and CIRR).
Experimental setup
Datasets: CIRR FashionIQ
Benchmarks: CIRR FashionIQ
Metrics: R@K Rsubset@K Avg(R@5, Rsubset@1)
Baselines: TG-CIR CASE COVR-BLIP Re-ranking SSN CALA SPRC RCL RDE
Builds on: BLIP-2 SPRC RCL RDE GMM sample selection
Closest papers by meaning
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.88)
- Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval (CVPR 2024 · similarity 0.87)
- CoLLM: A Large Language Model for Composed Image Retrieval (CVPR 2025 · similarity 0.87)
- Do Composed Image Retrieval Benchmarks Require Multimodal Composition? (NeurIPS 2026 · similarity 0.87)
- ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval (CVPR 2025 · similarity 0.87)
- Vision-by-Language for Training-Free Compositional Image Retrieval (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.