Research map / Vision-language and multimodal / Human and embodied modeling
Instance-Level Composed Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces i-CIR, an instance-level composed image retrieval benchmark with visual, textual, and composed hard negatives, and BASIC, a training-free CLIP-based fusion method that sets new SOTA on it and on class-level CIR datasets.
Problem
Composed image retrieval lacks high-quality evaluation data: existing datasets use semantic-level class definitions, automatically constructed triplets with false negatives, and weak hard negatives. Training data is also scarce.
Approach
A training-free method that computes image-to-image (in a PCA-projected semantic subspace from CLIP text features, with centering and query expansion) and text-to-image (with query contextualization) similarities, normalizes them, and fuses them with a Harris-corner-inspired term that rewards joint relevance and penalizes single-modality activation.
Key results
- BASIC achieves 31.64% macro-mAP on i-CIR versus 28.50% for CoVR-2 and 27.35% for MagicLens; without query expansion it reaches 34.35%.
- BASIC reaches 41.38 mAP on LTLL, 39.58 on MiniDN, 31.65 on NICO++ and 32.13 on ImageNet-R, the best among compared methods on those datasets.
- Text x Image baseline needs more than 40M random LAION distractors to match its i-CIR performance of 17.48%, showing the dataset is compact yet hard.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Text × Image with BASIC | CLIP ViT-L/14 · i-CIR | macro-mAP: 17.48 -> 31.64 | Helped |
| Add mean centering (ablation) | BASIC · i-CIR | macro-mAP: 17.48% -> 28.33% | Helped |
| Add min-based normalization (ablation) | BASIC · i-CIR | macro-mAP: 30.75 -> 31.64 | Helped |
| Add Harris fusion (ablation) | BASIC · i-CIR | macro-mAP: 31.61 -> 31.64 | No clear effect |
| Add text contextualization (ablation) | BASIC · i-CIR | macro-mAP: 25.85% -> 31.64% | Helped |
| Add semantic projection (ablation) | BASIC · i-CIR | macro-mAP: 31.61 -> 31.64 | No clear effect |
| Add query expansion (ablation) | BASIC · i-CIR | macro-mAP: 34.35 -> 31.64 | Hurt |
| Use a dataset-specific negative corpus rather than none (ablation) | BASIC semantic projection · LTLL | mAP: 41.80 -> 43.98 | Helped |
| Use an ImageNet-R-specific negative corpus rather than none (ablation) | BASIC semantic projection · ImageNet-R | mAP: 30.15 -> 33.22 | Helped |
| Use a dataset-specific negative corpus rather than none (ablation) | BASIC semantic projection · NICO++ | mAP: 30.67 -> 31.17 | Mixed |
| Use a dataset-specific negative corpus rather than none (ablation) | BASIC semantic projection · MiniDN | mAP: 38.64 -> 39.58 | Helped |
Surprising findings
- On i-CIR, the query-expansion step lowers performance (31.64 to 25.85 without it in one ablation row, and full BASIC without query expansion scores higher at 34.35), unlike on other datasets.
- On CIRR and FashionIQ the best unimodal result is text-only, so these legacy datasets reward composition only marginally.
Limitations
- Semi-automatic curation with manual vetting is slower and costlier than fully automatic datasets, limiting scale.
- Design assumes a single salient object per image query and English-only, public-domain imagery.
- Method assumes both modalities contribute complementary information; gains diminish for text-dominated tasks like CIRR-style datasets.
Experimental setup
Datasets: LAION ImageNet-R NICO++ MiniDomainNet LTLL CIRR FashionIQ CIRCO
Benchmarks: i-CIR ImageNet-R NICO++ MiniDN LTLL CIRR FashionIQ CIRCO
Metrics: mAP macro-mAP
Baselines: WeiCom Pic2Word CompoDiff CIReVL SEARLE MCL MagicLens CoVR-2 FreeDom
Builds on: CLIP Harris corner detector FAISS ChatGPT-generated corpora
Closest papers by meaning
- Generative Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.92)
- ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval (CVPR 2025 · similarity 0.92)
- CoLLM: A Large Language Model for Composed Image Retrieval (CVPR 2025 · similarity 0.92)
- Do Composed Image Retrieval Benchmarks Require Multimodal Composition? (NeurIPS 2026 · similarity 0.91)
- Vision-by-Language for Training-Free Compositional Image Retrieval (ICLR 2024 · similarity 0.91)
- Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.