Research map / Vision-language and multimodal / Visual question answering

Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding

NeurIPS 2026 · Vision-language and multimodal › Visual question answering · approach: Multimodal question answering datasets · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Exemplar2VQA uses a four-agent multi-agent coding pipeline (with a geometric utility API and AI2-THOR simulator feedback) to turn user-given spatial QA exemplars into large synthetic indoor 3D VQA data, used to fine-tune Qwen2.5-VL-3B/7B for gains on MMSI-Bench, VSI-Bench, SpaCE-10, and ViewSpatial-Bench.

Problem

MLLMs need large, complex 3D spatial QA data, but manual annotation is costly and LLM-generated spatial QA suffers from hallucinated geometric reasoning, while rule-based generators are rigid.

Approach

Generate spatial QA by having LLM agents write deterministic Python code over simulator scene metadata and a geometric utility library, scaling user-provided exemplar QA templates into datasets, with simulator-error-driven self-correction; the LLM orchestrates rather than reasons spatially.

Key results

• On VSI-Bench multiple-choice subset, the 3B model fine-tuned on Exemplar2VQA data reaches 42.9% average vs 35.3% for base Qwen2.5-VL-3B.
• On MMSI-Bench, the LoRA 7B model reaches 28.2% overall vs 25.9% for base Qwen2.5-VL-7B.
• Zero-shot on SpaCE-10, the 7B model reaches 42.2% overall vs 33.3% for base; on ViewSpatial-Bench 45.4% overall vs 36.9% for base.

Surprising findings

• Direct Gemini3.5-Flash annotation of 10K QA pairs degrades Qwen2.5-VL-3B average accuracy (28.2% vs 35.3% base), with Appr. Order dropping to 6.3%.
• Replacing the 30B coder with Qwen3.5-9B drops QA generation success from 92% to 48%.

Limitations

• Fine-tuned models lose performance on dynamic temporal tasks (Obj.* in MMSI-Bench), biased toward static reasoning.
• Route Planning and Appearance Order-style tasks fall outside the static object-centric generative scope and were evaluated only zero-shot or partially.
• Scene Quantification drops on SpaCE-10 because the pipeline focuses on instance-level geometry rather than functional zones.

Experimental setup

Datasets: AI2-THOR simulated indoor scenes MMSI-Bench exemplars VSI-Bench exemplars OSR-Bench exemplars SpaCE-10 (eval) ViewSpatial-Bench (eval)

Benchmarks: MMSI-Bench VSI-Bench SpaCE-10 ViewSpatial-Bench OSR-Bench Spatial-MM Spatial-Obj

Metrics: accuracy

Baselines: GPT-4o Gemini-1.5 Pro Gemini-1.5 Flash LLaVA-Video-72B LLaVA-OneVision-72B InternVL2.5-38B Qwen2.5-VL-7B fine-tuned on MMSI-Bench Qwen2.5-VL-7B fine-tuned on SpaCE-10 Qwen2.5-VL-7B fine-tuned on ViewSpatial Gemini3.5-Flash direct annotation

Builds on: LoRA Qwen2.5-VL AI2-THOR Holodeck ProcTHOR

Closest papers by meaning

• OpenEQA: Embodied Question Answering in the Era of Foundation Models (CVPR 2024 · similarity 0.87)
• SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.86)
• Multi-modal Situated Reasoning in 3D Scenes (NeurIPS 2024 · similarity 0.86)
• SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligence (ICLR 2026 · similarity 0.86)
• SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.86)
• Divide-and-Conquer: Tree-structured Strategy with Answer Distribution Estimator for Goal-Oriented Visual Dialogue (AAAI 2025 · similarity 0.86)