Research map / Vision-language and multimodal / Multimodal LLMs

RAIL: Rethinking Auditory Intelligence in Large Audio-Language Models with a CHC-Grounded Benchmark

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces RAIL, a CHC-grounded benchmark of 5,306 audio samples across 32 auditory cognitive subcapabilities, evaluating 26 LALMs against human performance under a unified protocol.

Problem

Existing audio benchmarks for large audio-language models are task- or domain-centric and do not reveal which underlying auditory cognitive abilities (perception, reasoning, memory, efficiency, knowledge) drive performance.

Approach

Organizes audio evaluation around the Cattell-Horn-Carroll (CHC) hierarchical cognitive theory, formalizing five broad abilities into 32 subcapabilities with curated/synthesized audio, human baselines, and a budgeted efficiency metric (B-AUC) that trades accuracy against reasoning-token budget.

Key results

• Knowledge (56.21) and Memory (55.05) score highest on average across 26 LALMs, while auditory processing is lowest (43.83).
• Closed-source models average 65.10 vs 46.27 for open-source; Gemini 3.1 Pro performs best overall.
• Six LALMs exceed human performance in general, but auditory processing and efficiency remain below human levels.

Surprising findings

• Memory for Sound Patterns (non-speech memory) stays below 60 for all models, while speech-based memory span is much stronger.
• Gemma3-E4B outperforms Gemma3-E2B in B-AUC only partly by size; model size does not determine efficiency, with similar-sized models varying widely (0.2-0.7 B-AUC).

Limitations

• Human evaluation uses only 640 samples with 24 participants, a small sample for human baselines.
• Strict accuracy and LLM-as-judge depend on GPT-5.4 judge, introducing judge dependence.
• Efficiency is measured via reasoning token counts, which is a model-internal proxy and not directly comparable to human time.

Experimental setup

Datasets: AudioBench-style existing audio corpora (selected samples)

Benchmarks: RAIL

Metrics: ACC LLM-as-Judge accuracy B-AUC

Baselines: AIR-Bench AudioBench MMAU MMAU-Pro MMAR MMSU SAKURA MUSE SonicBench STAR-Bench

Builds on: CHC theory AudioBench MMAU MMAR

Closest papers by meaning

• STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence (ICLR 2026 · similarity 0.88)
• Listen, Think, and Understand (ICLR 2024 · similarity 0.87)
• AudSemThinker: Enhancing Audio-Language Models Through Reasoning over Semantics of Sound (NeurIPS 2025 · similarity 0.87)
• MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark (ICLR 2025 · similarity 0.87)
• HAVE-Bench: Hierarchical Audio-Visual Evaluation from Perception to Interaction (CVPR 2026 · similarity 0.86)
• From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs (NeurIPS 2026 · similarity 0.86)