Research map / Vision-language and multimodal / Multimodal LLMs
Retrieval-driven Reasoning for Deliberative Visual Classification
Jianye Xie, Lianyong Qi, Fan Wang, Anqi Wang, Wenjuan Gong, Danxin Wang, Wanchun Dou, Yang Cao, Shichao Pei, Xiaokang Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a retrieval-driven deliberative reasoning approach for visual classification with VLMs, aiming to improve generalization and provide explanations for category choices.
Problem
Existing methods for improving VLM visual classification rely heavily on direct category-to-image matching, which limits generalization and gives no rationale for why a category is chosen.
Approach
Introduces a deliberative reasoning framework for visual classification that is driven by retrieval rather than direct category-to-image matching, so the model reasons over retrieved evidence before choosing a category.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add descriptor-based deliberative reasoning pipeline to CLIP | CLIP ViT-B/32 · ImageNet | accuracy (%): 58.46 -> 70.28 | Improved |
| use image-descriptor matching instead of category-name prompts (RdR vs CBD) | RdR · ImageNet, ViT-B/32 | accuracy (%): 62.97 -> 70.28 | Improved |
| retrieve 2 descriptors rather than 1 (ablation) | RdR · ImageNet, ViT-B/16 | accuracy (%): 74.52 -> 75.05 | Improved |
| retrieve more descriptors (K=2 to K=5) (ablation) | RdR · ImageNet, ViT-B/16 | accuracy (%): 75.05 -> 73.36 | Hurt |
| retrieve more descriptors (K=2 to K=5) (ablation) | RdR · ImageNet, ResNet-50 | accuracy (%): 68.19 -> 63.09 | Hurt |
| separate visual and contextual descriptor sets instead of one unified set (ablation) | RdR · 11 datasets average, ViT-B/16 | accuracy (%) | Improved |
| add descriptor deliberation step (fine-tuned MLLM refines retrieved descriptors) (ablation) | RdR · 11 datasets average, ViT-B/16, K=2 | accuracy (%): 73.29 -> 76.89 | Improved |
| add explicit deliberative reasoning before the final category (ablation) | RdR · 11 datasets average, ViT-B/16, K=2 | accuracy (%): 73.95 -> 76.89 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- PromptKD — PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024) · better on 7 of 7 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- RMIR: A Benchmark Dataset for Reasoning-Intensive Multimodal Image Retrieval (CVPR 2026 · similarity 0.87)
- Democratizing Fine-grained Visual Recognition with Large Language Models (ICLR 2024 · similarity 0.87)
- VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs (CVPR 2026 · similarity 0.87)
- Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.87)
- Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models (ICLR 2025 · similarity 0.87)
- DiVE-k: DIFFERENTIAL VISUAL REASONING FOR FINE-GRAINED IMAGE RECOGNITION (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
