Research map / Image generation / Autoregressive and latent generation
CCIN: Compositional Conflict Identification and Neutralization for Composed Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CCIN, which uses GPT-4 to identify compositional conflicts between a reference image caption and modified instruction, then neutralizes them via a dual-instruction Q-Former (SPRC-based) to improve composed image retrieval.
Problem
Composed image retrieval suffers from compositional conflicts between the reference image attributes and the modified instruction (e.g., blue vs. grey shirt), which feature-level learnable masks handle coarsely and uncontrollably.
Approach
Identify conflicting attributes explicitly with an LLM over captioned reference and modification text, generate a kept instruction retaining non-conflicting attributes, and use dual instructions (kept and modified) to guide instruction-aware Q-Former visual feature extraction, followed by adaptive fusion.
Key results
- On FashionIQ, CCIN reaches 64.59 mean recall (R@10 54.42 / R@50 74.76 in ablation), improving over SPRC's 64.27 average in Table 1.
- On CIRR, CCIN achieves a mean of 81.66 versus 80.37 for SPRC.
- On Shoes, CCIN reaches a mean of 59.42 versus 58.05 for TG-CIR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CCIN instead of SPRC | Composed image retrieval · FashionIQ | Mean: 64.27 -> 64.59 | Helped |
| Use CCIN instead of SPRC | Composed image retrieval · CIRR | Mean: 80.37 -> 81.66 | Helped |
| Use CCIN instead of TG-CIR | Composed image retrieval · Shoes | Mean: 58.05 -> 59.42 | Helped |
| Add the CCIN architecture to the ITC-only baseline (ablation) | CCIN · FashionIQ | Mean: 63.11 -> 63.60 | Helped |
| Add weighted regularization triplet loss (ablation) | CCIN · FashionIQ | Mean: 63.60 -> 64.15 | Helped |
| Add orthogonal projection regularization loss (ablation) | CCIN · FashionIQ | Mean: 64.15 -> 64.31 | Helped |
| Add modified-instruction guidance alongside kept-instruction guidance (ablation) | CCN · FashionIQ | Mean: 18.93 -> 64.10 | Helped |
| Add modified-instruction guidance alongside kept-instruction guidance (ablation) | CCN · CIRR | Mean: 39.30 -> 81.33 | Helped |
| Add kept-instruction guidance alongside modified-instruction guidance (ablation) | CCN · FashionIQ | Mean: 47.99 -> 64.10 | Helped |
| Add kept-instruction guidance alongside modified-instruction guidance (ablation) | CCN · CIRR | Mean: 71.87 -> 81.33 | Helped |
| Add textual features to the kept- and modified-instruction visual features (ablation) | CCN · FashionIQ | Mean: 64.10 -> 64.59 | Helped |
| Add textual features to the kept- and modified-instruction visual features (ablation) | CCN · CIRR | Mean: 81.33 -> 81.66 | Helped |
Surprising findings
- Using only the kept instruction performs far worse than dual instructions (CIRR mean 39.30 vs 81.66 for the full model in the kept-only ablation), indicating the kept instruction alone captures only a small part of user intent.
- Text-only features without visual features cause a substantial performance drop (FashionIQ mean 47.99 vs 64.59).
Limitations
- Relies on closed-source GPT-4 for conflict identification and kept instruction generation, with per-query LLM cost.
- Performance gains over SPRC and TG-CIR are modest on some FashionIQ categories (e.g., Toptee is lower than SPRC).
- Ablation of LLM choice is only qualitative, shown on example cases rather than quantified.
Experimental setup
Datasets: FashionIQ CIRR Shoes
Benchmarks: FashionIQ CIRR Shoes
Metrics: R@1 R@5 R@10 R@50 Mean recall
Baselines: SPRC TG-CIR Re-ranking CaLa CoVR-BLIP CASE SSN BLIP4CIR CLIP4CIR SPIRIT
Builds on: SPRC InstructBLIP TG-CIR CLIP4CIR BLIP-2
Closest papers by meaning
- ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval (CVPR 2025 · similarity 0.88)
- Generative Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.87)
- Do Composed Image Retrieval Benchmarks Require Multimodal Composition? (NeurIPS 2026 · similarity 0.87)
- Vision-by-Language for Training-Free Compositional Image Retrieval (ICLR 2024 · similarity 0.86)
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.86)
- Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.