Research map / Multi-agent systems and games / LLM agents

STEAMROLLER: A Multi-Agent System for Inclusive Automatic Speech Recognition for People Who Stutter

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Presents STEAMROLLER, a real-time three-stage pipeline (ASR transcription, multi-agent LLM text repair with a master agent, and zero-shot StyleTTS2 voice-cloned speech synthesis) that converts stuttered speech into fluent speech, evaluated on FluencyBank and a 23-participant user study.

Problem

Automatic speech recognition systems trained mostly on fluent speech fail on stuttered speech, producing semantically distorted transcripts and excluding people who stutter from voice-driven interfaces.

Approach

Decouple recognition, LLM-based text repair, and voice-cloned synthesis; the repair is done by several repair agents with chain-of-thought and few-shot prompting, coordinated by a master agent that evaluates outputs and gives feedback over up to three iterations.

Key results

• Average WER across four ASR systems drops from 31.60% to 18.69% on stuttered audio after STEAMROLLER repair (WER reduction reported as 10.46% in intro).
• Semantic similarity to ground truth rises from 0.81 (original ASR) to 0.91 (repaired) on average.
• In the 23-participant open-ended study, the system reports SUS 66.9, NPS 23.53, and average latency of 3.77 seconds per 50-100 word segment.

Surprising findings

• Zero-shot voice cloning yields low similarity to the original speaker (MOS-S 3.4), with participants noting only partial resemblance.

Limitations

• Zero-shot voice cloning produces limited speaker similarity and less natural voice.
• Removing disfluencies may alter paralinguistic cues such as tone and rhythm.
• Evaluation relies on a limited set of ASR systems and two languages; user study sample is 23 participants.

Experimental setup

Datasets: FluencyBank AS-70 SEP-28K

Benchmarks: FluencyBank AS-70

Metrics: WER MER WIL semantic similarity (Sentence-BERT) NPS SUS MOS Likert satisfaction latency

Baselines: data2vec wav2vec2 whisper-base SenseVoice-small zero-shot prompting

Builds on: Whisper StyleTTS2 Sentence-BERT

Closest papers by meaning

• $\tau$-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains (ICML 2026 · similarity 0.84)
• Heuristic-free Knowledge Distillation for Streaming ASR via Multi-modal Training (AAAI 2025 · similarity 0.82)
• Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.82)
• Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering (NeurIPS 2026 · similarity 0.81)
• MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark (ICLR 2025 · similarity 0.81)
• Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics (ICLR 2025 · similarity 0.81)