Research map / Vision-language and multimodal / Visual question answering
Provoking Multi-modal Few-Shot LVLM via Exploration-Exploitation In-Context Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an exploration-exploitation RL framework that learns an auto-regressive policy to select multi-modal demonstration combinations for few-shot LVLM in-context learning, evaluated on four VQA benchmarks with Qwen-VL and LLaVA.
Problem
Demonstration selection for in-context learning in large vision-language models is typically heuristic (fixed, similarity, or diversity based) and selects demonstrations independently, ignoring interactions that cause redundancy and suboptimal few-shot performance.
Approach
Reformulates demonstration selection as combinatorial optimization: a transformer policy over multi-modal interactive encodings autoregressively picks demonstration sets, explored via stochastic beam search and optimized with a GRPO-style policy gradient using answer-match rewards from the LVLM itself.
Key results
- On OKVQA with Qwen-VL, the method reaches 64.7 VQAScore vs 60.6 for similarity-based selection.
- Average over four VQA benchmarks with Qwen-VL improves from 44.3 (similarity) to 45.9.
- With LLaVA, average rises from 31.3 (similarity) to 40.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use exploration-exploitation demonstration selection instead of similarity retrieval | Qwen-VL · Vizwiz | VQAScore: 48.7 -> 53.6 | Helped |
| use exploration-exploitation demonstration selection instead of similarity retrieval | Qwen-VL · All benchmarks | Average VQAScore: 44.3 -> 45.9 | Helped |
| use exploration-exploitation demonstration selection instead of similarity retrieval | Qwen-VL · OKVQA | VQAScore: +4.1 | Helped |
| use exploration-exploitation demonstration selection instead of similarity retrieval | Qwen-VL · TextVQA | VQAScore: 67.4 -> 67.4 | No clear effect |
| remove stochastic beam search (ablation) | full method · OKVQA | VQAScore: 63.3 -> 64.7 | Helped |
| use autoregressive selection rather than independent selection (ablation) | full method · OKVQA | VQAScore: 63.3 -> 64.7 | Helped |
| use responsive reinforcement instead of preference loss (ablation) | full method · OKVQA | VQAScore: 62.9 -> 64.7 | Helped |
| reverse demonstration order | similarity retrieval · OKVQA | VQAScore: 60.63 -> 60.92 | Helped |
| reverse demonstration order (ablation) | proposed method · OKVQA | VQAScore: 64.66 -> 64.69 | No clear effect |
| increase the number of demonstrations (ablation) | proposed method · OKVQA | VQAScore | Mixed |
| increase stochastic beam-search temperature (ablation) | proposed method · OKVQA | VQAScore | Mixed |
Surprising findings
- Similarity-based demonstrations often mislead LVLMs; correct predictions were associated with lower-similarity demonstrations.
- The method underperforms on math-style tasks (MA) and some coarse perception/instance reasoning tasks compared with similarity selection.
Limitations
- Only explores vision-language multimodal ICL; video and audio not studied.
- Method underperforms on math tasks and some perception/reasoning categories versus similarity baseline.
- Performance declines when demonstration count grows too large, challenging the RL process.
Experimental setup
Datasets: OKVQA TextVQA VizWiz MMStar
Benchmarks: OKVQA TextVQA VizWiz MMStar
Metrics: VQAScore accuracy
Baselines: zero-shot random selection similarity-based selection BM25
Builds on: GRPO PPO BM25 ImageBind Qwen-VL
Closest papers by meaning
- What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration (NeurIPS 2024 · similarity 0.90)
- VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought (NeurIPS 2024 · similarity 0.89)
- VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning (ICLR 2025 · similarity 0.89)
- Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learning (AAAI 2026 · similarity 0.89)
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.88)
- In-Context Learning Can Help Vision Language Models Overcome Training Prior (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.