Research map / Vision-language and multimodal / Multimodal LLMs

STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces STAR-Bench, a multiple-choice benchmark for audio 4D intelligence (reasoning over sound dynamics in time and 3D space), evaluating 19 audio and omni models against humans and showing large gaps, especially for open-source models.

Problem

Existing audio benchmarks test semantics recoverable from text captions and do not measure fine-grained perceptual reasoning over temporal dynamics and 3D spatial audio cues.

Approach

Defines audio 4D intelligence and pairs procedurally synthesized foundational perception tests (pitch, loudness, duration, azimuth, elevation, distance; absolute and relative) with real-audio holistic temporal (segment reordering) and spatial (static localization, multi-source relations, trajectory tracking) reasoning tasks, plus a caption-only ablation showing drops of 31.5% temporal and 35.2% sp

Key results

• Gemini 2.5 Pro is the best model with 49.59% macro accuracy, versus 75.60% human perception and 88.00% human temporal accuracy.
• Gemini 2.5 Pro reaches 58.52% temporal OA, while Gemini 2.5 Pro's spatial OA is 43.62%, both far below humans on spatial tasks (73.72%).
• Caption-only answering drops accuracy by 31.5% on temporal and 35.2% on spatial tasks, versus only 5.9% and 9.0% for MMAU and MMAR.

Surprising findings

• Thinking variants of Audio Flamingo 3 and Xiaomi-MiMo-Audio perform worse than their non-thinking counterparts.
• Most models score near random on spatial tasks because mono downmixing discards stereo cues; BAT is the only model that keeps multi-channel information.

Limitations

• Open-source models show near-zero all-correct rates, indicating unstable predictions under prompt perturbations.
• Human baseline uses a subset of about 10% of data and non-expert students for perception tasks.

Experimental setup

Datasets: Clotho FSD50K STARSS23 Pyroomacoustics-synthesized audio MIT KEMAR HRTF

Benchmarks: STAR-Bench

Metrics: Average Accuracy All-Correct Rate macro accuracy overall accuracy

Baselines: Gemini 2.5 Pro Gemini 2.5 Flash GPT-4o Audio Qwen2-Audio-Instruct Audio Flamingo 3 MiDashengLM Qwen-2.5-Omni Kimi-Audio BAT SALMONN

Builds on: MMAU MMAR MMBench Pyroomacoustics

Closest papers by meaning

• The World is Not Mono: Enabling Spatial Understanding in Large Audio-Language Models (NeurIPS 2026 · similarity 0.92)
• SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing (NeurIPS 2025 · similarity 0.90)
• Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D World (CVPR 2026 · similarity 0.90)
• OWL : Geometry-Aware Spatial Reasoning for Audio Large Language Models (ICLR 2026 · similarity 0.90)
• Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs (NeurIPS 2026 · similarity 0.90)
• MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.90)