Research map / Large language models / Instruction and fine-tuning

Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes full-duplex spoken LMs via logit lens, identifies "state inertia" delaying the switch from generative to perceptive state at interruptions, introduces the Zero-Buffer Benchmark, and mitigates it with a training-free perception steering vector.

Problem

Full-duplex spoken language models fail to perceive the beginning of abrupt user interruptions because their internal listening-speaking coordination lags behind conversational context changes.

Approach

Logit-lens affinity scores reveal stream-specific predictive focus and a generative/perceptive state modulation; a perception vector (mean-difference of perception- vs generation-dominant hidden states) is added at interruption onset to accelerate the transition into the perceptive state without fine-tuning.

Key results

• On PersonaPlex, interruption lowers correctness from 0.49 to 0.28 and IWOR from 0.74 to 0.40; steering raises them to 0.45 and 0.72.
• Steering improves correctness and IWOR for Moshi (0.22 to 0.34; 0.29 to 0.64) and Raon-SpeechChat (0.03 to 0.17; 0.16 to 0.24).
• Steering preserves Full-Duplex Bench interruption scores within uncertainty (PersonaPlex 3.34 to 3.41; Moshi 3.45 to 3.36).

Surprising findings

• Logit-lens decoding during listening predicts upcoming user-side words even though the model's output stream is silent (PAD).
• Steering gives little benefit on the standard FDB benchmark because its queries contain leading filler words, so state inertia is less visible there.

Limitations

• Relies on an energy-based onset detector that may be sensitive to noise or multi-speaker settings.
• Evaluation limited to few open-source FD-SLMs.
• Logit-lens affinity scores are diagnostic approximations and can be noisy per example.

Experimental setup

Datasets: turn-by-turn interaction dataset (100 queries) speech-inducing prompts Full-Duplex Bench

Benchmarks: Zero-Buffer Benchmark (ZBB) Full-Duplex Bench

Metrics: correctness IWOR (initial word occurrence rate) FDB score (1-5)

Baselines: Moshi Raon-SpeechChat Full-Duplex Bench

Builds on: logit lens activation steering representation engineering

Closest papers by meaning

• Language Model Can Listen While Speaking (AAAI 2025 · similarity 0.88)
• Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics (ICLR 2025 · similarity 0.86)
• Full-Duplex Speech-Motion Model for Dyadic Interaction (NeurIPS 2026 · similarity 0.86)
• Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
• MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversation (NeurIPS 2026 · similarity 0.85)
• The Sharp Directions Are Against You: Curvature Analysis of Activation Steering (NeurIPS 2026 · similarity 0.85)