Research map / Large language models / Reasoning and chain of thought

Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CESAR, an online GRPO framework for audio LLMs that adds process-level rewards (consistency, keyword patterns, domain terms, overthinking penalty) to outcome rewards, reporting SOTA on MMAU Test-mini and MMSU reasoning.

Problem

Audio LLMs often perform worse when they reason at inference (test-time inverse scaling), and outcome-only RLVR training does not produce reliable reasoning processes.

Approach

Augment outcome-based GRPO rewards with reasoning-process rewards: reasoning-answer and reasoning-question concept-overlap consistency, rewards for structured analytical, logical and audio-domain keywords, and a length-based overthinking penalty, then sweep max thinking length to find a model-specific reasoning sweet spot.

Key results

• CESAR reaches 77.10% on MMAU Test-mini at its reasoning sweet spot, above the Ke-Omni-R-style outcome-only RL ablation (74.60%) and the untrained Qwen2.5-Omni base (65.20% with reasoning).
• On MMSU, CESAR scores 88.72% on semantic reasoning versus 82.16% for humans, while its perception average (48.45%) remains far below human (91.24%).
• Untrained Qwen2.5-Omni degrades under longer reasoning (68.60% direct answering vs. 65.20% with reasoning), while CESAR's test-time scaling curve peaks at a sweet spot of about 35-40 tokens.

Surprising findings

• Ablating RL training drops the base model to 65.20%, below its direct-answer score, showing reasoning hurts untrained audio LLMs.
• CESAR's direct-answering accuracy also rises to 73.70% from the base 68.60%, suggesting reasoning training transfers to non-reasoning mode.

Limitations

• Perception bottleneck: perception on MMSU lags far behind human performance.
• High computational cost of online GRPO with multiple samples per input.
• Reward weights require tuning; consistency reward uses simple concept-overlap similarity.

Experimental setup

Datasets: AVQA MMAU Test-mini MMSU MMAR

Benchmarks: MMAU Test-mini MMSU MMAR

Metrics: accuracy win rate (AI-as-judge)

Baselines: Qwen2.5-Omni-7B Ke-Omni-R Gemini 2.5 Pro GPT-4o Audio Gemini 1.5 Pro Audio-Reasoner

Builds on: GRPO Ke-Omni-R R1-AQA Qwen2.5-Omni

Closest papers by meaning

• Audio-Thinker: Guiding Large Audio Language Model When and How to Think via Reinforcement Learning (AAAI 2026 · similarity 0.93)
• MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.91)
• Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities (ICML 2025 · similarity 0.90)
• Mellow: a small audio language model for reasoning (NeurIPS 2025 · similarity 0.89)
• MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark (ICLR 2025 · similarity 0.89)
• PrismAudio: Decomposed Chain-of-Thought and Multi-dimensional Rewards for Video-to-Audio Generation (ICLR 2026 · similarity 0.88)