Research map / Vision-language and multimodal / Multimodal LLMs

Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AudioMCQ (571k audio MCQs with CoT), identifies zero audio-contribution in LALM benchmarks, and proposes Audio-Contribution Filtering to drive Weak-to-Strong and Mixed-to-Strong SFT-then-GRPO post-training of Qwen2.5-Omni.

Problem

LALMs often answer audio questions correctly from text alone (zero audio-contribution), and the allocation of data between SFT and RL stages for LALM post-training is unexplored, with few large high-quality datasets available.

Approach

Partition training data by whether models answer correctly with silent audio (Audio-Contribution Filtering), then use weak-contribution data for SFT and strong-contribution data for GRPO (or mixed data for SFT with strong data for GRPO), so RL focuses on questions that genuinely require audio.

Key results

• Weak-to-Strong achieves 78.2% on MMAU-test-mini and 75.6% on MMAU, exceeding prior SOTA (Audio-Thinker 78.0 / 75.4).
• Mixed-to-Strong reaches 67.0% on MMAR and 71.7% on MMSU, the first MMSU score above 70%.
• Silent-audio evaluation yields 49.8% on MMAU-test-mini versus a 25.5% random baseline, showing widespread zero audio-contribution.

Surprising findings

• Models answer about 49.8% of MMAU-test-mini and 37.8% of MMSU questions correctly with silent audio, well above chance.
• GRPO on mixed audio-contribution data yields minimal gains and can degrade performance on strong audio-contribution splits of MMAR and MMSU.

Limitations

• Audio-contribution labels depend on three specific LALMs and silent-audio replacement, which may not generalize.
• Synthetic MCQs and CoT are generated by Qwen3-235B with only spot-check human review.
• CoT is excluded from the main experiments, so the reasoning-based training benefits are not evaluated.

Experimental setup

Datasets: Clotho AudioCaps CompA-R MusicCaps LP-MusicCaps SpeechCraft TACOS AVQA VGGSound

Benchmarks: MMAU-test-mini MMAU MMAR MMSU DCASE 2025 Audio-Question-Answering

Metrics: accuracy

Baselines: Audio-Reasoner R1-AQA Kimi-Audio SARI Audio Flamingo 3 Omni-R1 Audio-Thinker GPT4o-Audio Gemini-2.0-Flash

Builds on: GRPO DeepSeekMath R1-AQA Omni-R1 Qwen2.5-Omni

Closest papers by meaning

• Audio-Thinker: Guiding Large Audio Language Model When and How to Think via Reinforcement Learning (AAAI 2026 · similarity 0.91)
• MATS: An Audio Language Model under Text-only Supervision (ICML 2025 · similarity 0.89)
• MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.89)
• Listen, Think, and Understand (ICLR 2024 · similarity 0.88)
• Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards (ICLR 2026 · similarity 0.88)
• Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities (ICML 2025 · similarity 0.88)