Research map / Large language models / Reasoning and chain of thought

Can Speech LLMs Think while Listening?

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Fine-tunes the multi-stream Moshi speech LLM to do text chain-of-thought with streaming user ASR, then adds a Question Completeness metric for early "thinking while listening" and DPO preference tuning to cut latency.

Problem

Speech LLMs lag behind text LLMs on complex reasoning, and adding chain-of-thought before responding adds latency that hurts spoken interaction.

Approach

Use text-only CoT in Moshi's text monologue stream, trained with spoken-rewritten CoT-Collection data and streaming ASR; start CoT early at a timestep where a KL-based Question Completeness score reaches 0.95, then apply DPO with correctness- and length-based rejection-sampled preferences to tune early reasoning.

Key results

• Moshi+CoT reaches 77.7 on ARC-E and 56.1 on SIQA versus 30.2 and 22.8 for the Moshi baseline, with an average absolute gain of about 29.1 points on SRQA tasks.
• Correctness-based DPO improved accuracy on all evaluation sets for the QC-early-CoT models, and length-based DPO cut latency by about 70% (e.g., ARC-E 49.2 to 12.0 tokens) without accuracy loss.
• On GSM8K, text CoT reached 17.5% accuracy versus 17.2% for speech CoT, while text CoT used about 119 tokens versus 369 for speech.

Surprising findings

• Direct fine-tuning without CoT degraded GSM8K accuracy below the Moshi baseline, suggesting the baseline already reasons implicitly.
• Word-count shift heuristics (WS-N) gave inconsistent latency reductions, with no clear learned early-reasoning pattern on PIQA and GSM8K.

Limitations

• Speech benchmarks are LLM-rewritten from text datasets rather than natural spoken data.
• Early reasoning causes accuracy drops; QC-based early CoT can start before key information is spoken and produce wrong reasoning.
• Evaluation is limited to single-turn tasks and relies on an LLM judge with ASR transcription.

Experimental setup

Datasets: CoT-Collection ARC PIQA SIQA GSM8K LLaMA-Questions

Benchmarks: SRQA (ARC-E, ARC-C, SIQA, PIQA, GSM8K) LLaMA-Questions

Metrics: accuracy latency (tokens between question end and response start) word error rate

Baselines: Moshi Qwen2-Audio-7B-Instruct Kimi-Audio-7B-Instruct LLaMA2-7b-Chat Gemma-7B-Instruct WS-N word-shift heuristic

Builds on: Moshi chain-of-thought prompting direct preference optimization STITCH

Closest papers by meaning

• STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Models (ICLR 2026 · similarity 0.89)
• MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.89)
• Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards (ICLR 2026 · similarity 0.88)
• Chain-of-Thought Reasoning Without Prompting (NeurIPS 2024 · similarity 0.88)
• Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs (NeurIPS 2025 · similarity 0.88)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.88)