Research map / Image generation / Text-to-image diffusion
ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConText-CIR, a composed image retrieval model trained with a Text Concept-Consistency loss over noun-phrase cross-attention plus a GPT-4o-based synthetic CIR data pipeline, reaching SOTA on CIRR and CIRCO.
Problem
Composed image retrieval struggles with complex, multi-attribute modification texts, where CLIP-style models fail to align specific text noun phrases with the corresponding query image regions; existing CIR datasets have simple or low-quality annotations.
Approach
A Text Concept-Consistency loss that penalizes differences between a noun phrase's cross-attention to the query image in full-sentence context and its cross-attention when encoded alone, combined with a contrastive CIR loss; plus an LLM-driven pipeline (GPT-4o) that rewrites CIR captions and generates new CIR triplets.
Key results
- On CIRR test with ViT-H, ConText-CIR reaches R@1 55.24 vs 50.43 for the prior best (CoVR-2, ViT-G).
- Zero-shot CIRR R@1 improves by 4.78 (ViT-B), 5.38 (ViT-L), and 12.88 (ViT-H) over prior state of the art.
- Removing the Text CC loss drops ViT-H CIRR R@1 from 55.24 to 48.92.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CIRRR rewritten captions to CIRR training data (ablation) | ConText-CIR · CIRR test | Recall@1: 45.25 -> 48.54 | Helped |
| add LaSCo training data to CIRR+CIRRR (ablation) | ConText-CIR · CIRR test | Recall@1: 48.54 -> 53.12 | Helped |
| add Hotel-CIR training data to CIRR+CIRRR+LaSCo (ablation) | ConText-CIR · CIRR test | Recall@1: 53.12 -> 55.24 | Helped |
| freeze the text encoder (ablation) | ConText-CIR · CIRR test | Recall@1: 54.28 -> 49.52 | Hurt |
| freeze the image encoder (ablation) | ConText-CIR · CIRR test | Recall@1: 55.24 -> 54.18 | Hurt |
| remove the query image as a hard negative (ablation) | ConText-CIR · CIRR test | Recall@1: 55.24 -> 54.52 | Hurt |
| consider only leaf noun phrases (ablation) | ConText-CIR · CIRR test | Recall@1: 55.24 -> 54.67 | Hurt |
| add the Text CC loss (ablation) | ConText-CIR · CIRR test | Recall@1: 48.92 -> 55.24 | Helped |
Surprising findings
- Freezing the text encoder hurts far more than freezing the image encoder (R@1 49.52 vs 54.18), suggesting the pretrained CLIP text space is less suited to CIR than the image space.
Limitations
- Synthetic data generation relies on proprietary GPT-4o, and the MagicLens-style dataset replication is not possible due to unavailability of prior data.
- Zero-shot results rely on training sets (LaSCo, Hotel-CIR) and the aggregated dataset mixes several sources, making the gains partly data-driven.
Experimental setup
Datasets: CIRR LaSCo CIRRR Hotel-CIR FashionIQ ImageNet-R TraffickCam
Benchmarks: CIRR CIRCO FashionIQ ImageNet-R
Metrics: Recall@K Recall_subset@K mAP@K
Baselines: CASE CompoDiff CoVR-BLIP CoVR-2 MagicLens LinCIR LDRE CIReVL Slerp+TAT TFCIR
Builds on: CLIP MagicLens Set Transformer LaSCo CASE
Closest papers by meaning
- Generative Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.94)
- Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.92)
- Do Composed Image Retrieval Benchmarks Require Multimodal Composition? (NeurIPS 2026 · similarity 0.92)
- Language-only Training of Zero-shot Composed Image Retrieval (CVPR 2024 · similarity 0.92)
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.92)
- Vision-by-Language for Training-Free Compositional Image Retrieval (ICLR 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.