Research map / Vision-language and multimodal / Human and embodied modeling
Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OSrCIR, a training-free one-stage reflective chain-of-thought prompting method where an MLLM (GPT-4o) reasons a target image description directly from the reference image and manipulation text, then retrieves with CLIP for zero-shot composed image retrieval.
Problem
Training-free zero-shot composed image retrieval (ZS-CIR) with two-stage captioning-then-LLM pipelines loses critical visual details and under-exploits LLM reasoning about implicit user intent.
Approach
Replace the two-stage caption-then-LLM pipeline with a single MLLM call that takes the reference image and manipulation text together and follows a reflective chain-of-thought (original description, thoughts, reflections, target description) with vision-by-language in-context examples, then performs CLIP text-to-image retrieval.
Key results
- On CIRCO with ViT-L/14, OSrCIR reaches 23.87 mAP@5 versus 18.92 for the best training-free baseline CIReVL* (GPT-4o two-stage).
- On GeneCIS with ViT-L/14, average R@1 improves by 1.8% over CIReVL*, and on FashionIQ with ViT-L/14 it outperforms CIReVL* by 4.74% on average.
- Ablation shows removing the one-stage reasoning drops CIRCO mAP@5 to 21.73 and removing reflective CoT drops it to 20.86 from 23.87.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use one-stage reasoning instead of the strongest two-stage baseline | ZS-CIR with ViT-L/14 · CIRCO test | mAP@5: 18.92 -> 23.87 | Helped |
| Use one-stage reasoning instead of the strongest baseline | ZS-CIR with ViT-L/14 · CIRR test | Recall@1: 25.60 -> 29.45 | Helped |
| Use reflective one-stage reasoning instead of the strongest training-free baseline | ZS-CIR with ViT-L/14 · GeneCIS test average | R@1: 16.1 -> 17.9 | Helped |
| Use reflective one-stage reasoning instead of the strongest training-free baseline | ZS-CIR with ViT-L/14 · FashionIQ validation average | R@10: 29.05 -> 33.26 | Helped |
| Use two-stage reasoning instead of one-stage reasoning (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 21.73 | Hurt |
| Remove Reflective CoT (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 20.86 | Hurt |
| Remove the Original Image Description step (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 22.56 | Hurt |
| Remove the Thoughts step (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 21.46 | Hurt |
| Remove the Reflections step (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 22.04 | Hurt |
| Remove vision-by-language in-context samples (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 22.97 | Hurt |
| Replace GPT-4o with LLaVA (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 20.89 | Hurt |
| Replace GPT-4o with MiniGPT-4 (ablation) | OSrCIR · CIRCO | mAP@5: 23.87 -> 19.85 | Hurt |
Surprising findings
- Simply swapping in a stronger MLLM for the two-stage pipeline (CIReVL*) yields only marginal gains over CIReVL, suggesting the two-stage structure is the bottleneck.
- On FashionIQ with ViT-G/14, OSrCIR falls short of the textual inversion method LinCIR, which the authors attribute to CLIP's limited fashion-domain knowledge.
Limitations
- Inference is about 30x slower than textual inversion methods due to MLLM API calls.
- Underperforms LinCIR on FashionIQ with ViT-G/14, likely due to CLIP's limited fashion-domain alignment.
- Relies on closed-source GPT-4o API; open-source MLLM variants lag behind.
Experimental setup
Datasets: CIRR CIRCO FashionIQ GeneCIS
Benchmarks: CIRCO CIRR GeneCIS FashionIQ
Metrics: mAP@k Recall@k RecallSubset@k
Baselines: Pic2Word SEARLE Context-I2W LinCIR CIReVL CIReVL*
Builds on: CIReVL Chain-of-Thought prompting CLIP
Closest papers by meaning
- Vision-by-Language for Training-Free Compositional Image Retrieval (ICLR 2024 · similarity 0.92)
- ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval (CVPR 2025 · similarity 0.92)
- Generative Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.92)
- Language-only Training of Zero-shot Composed Image Retrieval (CVPR 2024 · similarity 0.91)
- Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval (CVPR 2025 · similarity 0.91)
- Context-I2W: Mapping Images to Context-Dependent Words for Accurate Zero-Shot Composed Image Retrieval (AAAI 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.