Research map / Vision-language and multimodal / Multimodal LLMs

OWL : Geometry-Aware Spatial Reasoning for Audio Large Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SAGE, a geometry-aware binaural audio encoder trained with depth-conditioned RIR reconstruction, and OWL, a spatial audio LLM with curriculum learning and geometry-aware chain-of-thought, plus the 1.1M-QA BiDepth dataset.

Problem

Audio LLMs lack geometric grounding and single-step inference, leaving direction and distance estimation coarse and spatial reasoning weak; prior spatial QA (BAT) uses only coarse four-region labels and audio-only encoders.

Approach

Train a binaural audio encoder with auxiliary binaural RIR prediction from panoramic depth (privileged supervision, audio-only at inference), feed it through a Q-Former into LLaMA-2-7B with a three-stage curriculum and clock-based CoT rationales.

Key results

• SAGE reduces mean angular error from 32.99 (Spatial-AST) to 26.32 on BiDepth (audio-only) and to 21.67 with depth supervision, while raising mAP on SpatialSoundQA to 49.94-49.93.
• OWL with CoT reaches 46.17% 12-bin DoA accuracy on BiDepth single-source and 77.89% Type III binary accuracy vs BAT's 69.46%.
• On SpatialSoundQA zero-shot, OWL improves reasoning average to 79.06 vs BAT's 76.89 and DoA accuracy from 75.54 to 78.31.

Surprising findings

• Gemini models (1.5 Pro, 2.5 Pro, 2.5 Flash) score only about 12% mAP on double-source detection, far below the spatial audio systems.

Limitations

• Dataset is simulation-based, so robustness to real-world acoustics is untested.
• Reasoning restricted to single-turn QA.
• Elevation distribution is skewed toward horizontal sources and RT60 range is limited.

Experimental setup

Datasets: BiDepth SpatialSoundQA AudioSet Matterport3D SoundSpaces v2.0

Benchmarks: BiDepth SpatialSoundQA

Metrics: mAP ER_20 MAE DER binary accuracy DoA accuracy

Baselines: SELDNet Spatial-AST BAT VideoLLaMA2 RAVEN AudioFlamingo2 Gemini-1.5-Pro Gemini-2.5-Pro Gemini-2.5-Flash

Builds on: BAT Q-Former LLaMA-2 LoRA SpatialSoundQA Spatial-AST

Closest papers by meaning

• The World is Not Mono: Enabling Spatial Understanding in Large Audio-Language Models (NeurIPS 2026 · similarity 0.92)
• BAT: Learning to Reason about Spatial Sounds with Large Language Models (ICML 2024 · similarity 0.91)
• Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs (NeurIPS 2026 · similarity 0.91)
• STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence (ICLR 2026 · similarity 0.90)
• SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing (NeurIPS 2025 · similarity 0.90)
• JAEGER: Joint 3D Audio-Visual Grounding and Reasoning in Simulated Physical Environments (ICML 2026 · similarity 0.89)