Research map / Vision-language and multimodal / Multimodal LLMs

JRDB-Reasoning: A Difficulty-Graded Benchmark for Visual Reasoning in Robotics

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces JRDB-Reasoning, a difficulty-graded visual reasoning benchmark built by formalizing reasoning complexity, an adaptive query engine, and extending JRDB with human-object interaction and geometric annotations, then evaluating VLMs on VG and VQA across three difficulty levels.

Problem

Existing visual reasoning benchmarks lack a clear definition of reasoning complexity, cannot generate customizable questions of varying difficulty, and do not provide structured step-by-step reasoning annotations.

Approach

Defines reasoning complexity as D = S + R + T over a spatio-temporal scene graph (entities, spatial edges, temporal slots), uses an adaptive query engine to generate questions with intermediate workflow annotations, and enriches JRDB with HOI and geometric relation labels.

Key results

• InternVL 2.5 achieves the best image VQA accuracy among evaluated VLMs (49.9 / 49.2 / 44.6 across D1-D3), while human evaluation scores are 96.8 / 88.5 / 87.1.
• Florence-V2 achieves the best VG mIoU on images (37.6 / 35.7 / 25.2 across D1-D3), with performance declining as difficulty increases.
• On video VQA, InternVL 2.5 reaches 49.1 / 48.6 / 46.0 versus human 98.3 / 92.1 / 89.1.

Surprising findings

• Performance on video VQA is reported as better than on images for the evaluated models, attributed to multi-frame analysis.
• Reported 'Human Eval' scores are described as evaluator perceptions of model performance, making the human baseline ambiguous.

Limitations

• Only single time slot (T=1) evaluated; temporal complexity not tested in experiments.
• Difficulty levels stop at D3 because models reportedly struggle beyond it, so the benchmark's top range is not explored.
• Complexity formula validated only with a 20-participant user study on 500 questions; reporting of the study is brief.

Experimental setup

Datasets: JRDB

Benchmarks: JRDB-Reasoning

Metrics: accuracy mIoU

Baselines: InternVL 2.5 Paligemma Qwen2.5-VL LLaVA-NeXT LLaVA-NeXT-Video Florence-V2 GroundingDINO YOLO-World

Builds on: JRDB

Closest papers by meaning

• Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.89)
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.89)
• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.89)
• ClevrSkills: Compositional Language And Visual Reasoning in Robotics (NeurIPS 2024 · similarity 0.89)
• VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models (ICLR 2026 · similarity 0.88)
• Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale (ICML 2026 · similarity 0.88)