Research map / Vision-language and multimodal / Human and embodied modeling
Aligning Composed Query with Image via Discriminative Perception from Negative Correspondences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DIPNEC, a composed image retrieval model that adds difference quantization alignment, composed word-level alignment, and optimal transport with semantic preservation over hard negative triplets, reaching 65.32 mean R@10/R@50 on FashionIQ and state-of-the-art-level results on CIRR.
Problem
Composed image retrieval models rely on partial correlations in positive triplets and underuse negative correspondences, so they confuse targets with mismatched candidates and underweight discriminative modifier words.
Approach
Combine a difference-based similarity (textual modifiers vs. visual differences between reference and candidates) and word-level weighted query composition with an optimal transport re-assignment of similarity scores, masked to hard negatives and trained with a Jensen-Shannon loss.
Key results
- On FashionIQ, DIPNEC reaches 65.32 average mean recall, versus 62.21 for TG-CIR and 58.05 for CaLa.
- On CIRR, DIPNEC achieves R@1 of 47.24 and R@5 of 80.20, with the highest R@1 among compared methods; SPN has slightly higher R@50 (98.17 vs 97.87).
- Ablations show removing OT assignment drops FashionIQ R@10 mean from 54.00 to 52.09, and masking only hard negatives performs best among mask variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DIPNEC instead of TG-CIR | composed image retrieval · FashionIQ | Mean: 62.21 -> 65.32 | Helped |
| Use DIPNEC instead of TG-CIR | composed image retrieval · CIRR | R@1: 45.25 -> 47.24 | Helped |
| Use DIPNEC instead of SPN | composed image retrieval · CIRR | R@50: 98.17 -> 97.87 | Hurt |
| Add optimal transport (ablation) | DIPNEC · FashionIQ | Mean: 52.09 -> 54.00 | Helped |
| Add the semantic preservation mask (ablation) | DIPNEC optimal transport · FashionIQ | Mean: 52.57 -> 54.00 | Helped |
| Filter hard negative triplets with the opposite mask (ablation) | DIPNEC optimal transport · FashionIQ | Mean: 52.39 -> 54.00 | Hurt |
| Replace Jensen-Shannon divergence with KL divergence (ablation) | DIPNEC optimal transport · FashionIQ | Mean: 53.72 -> 54.00 | Hurt |
| Replace Jensen-Shannon divergence with symmetric KL divergence (ablation) | DIPNEC optimal transport · FashionIQ | Mean: 53.79 -> 54.00 | Hurt |
| Add Difference Quantization Alignment (ablation) | DIPNEC · FashionIQ | R@10: 52.96 -> 54.00 | Helped |
| Add Composed Word-level Alignment (ablation) | DIPNEC · FashionIQ | R@10: 52.08 -> 54.00 | Helped |
| Use enhanced modification features when composing the query (ablation) | DIPNEC · FashionIQ | R@10: 53.65 -> 54.00 | Helped |
| Add the visual-difference alignment score SD to the final score (ablation) | DIPNEC · FashionIQ | R@10: 53.35 -> 54.00 | Helped |
Surprising findings
- Applying the OT mask to easy negatives instead of hard ones gives worse results (52.39 vs 54.00), suggesting OT on easy negatives adds noise.
Limitations
- Mask ratio above about 60% degrades performance due to optimization complexity.
- Gains over SPN on CIRR R@50 are not achieved; SPN is slightly higher.
Experimental setup
Datasets: FashionIQ CIRR
Benchmarks: FashionIQ CIRR
Metrics: R@10 R@50 R@1 R@5 Recall_subset@1 mean recall
Baselines: TG-CIR CaLa SPN FAME-ViL CLIP4Cir DWC MGUR BLIP4CIR CompoDiff
Builds on: InfoNCE Sinkhorn optimal transport CLIP4Cir TG-CIR
Closest papers by meaning
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.86)
- Expanding the Scope of Negatives: Boosting Image-Text Matching with Negatives Distribution Guided Learning (AAAI 2025 · similarity 0.86)
- Noisy Correspondence Rectification via Asymmetric Similarity Learning (AAAI 2025 · similarity 0.86)
- QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval (ICML 2025 · similarity 0.86)
- Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion (ICLR 2025 · similarity 0.86)
- Learning with Noisy Triplet Correspondence for Composed Image Retrieval (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.