Research map / Image generation / Autoregressive and latent generation
SpeechOp: Inference-Time Task Composition for Generative Speech Processing
Justin Lovelace, Rithesh Kumar, Jiaqi Su, Ke Chen, Kilian Q Weinberger, Zeyu Jin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-task training (enhancement, separation) to the TTS model (ablation) | TTS Baseline · Zero-shot TTS | MOS-VS: 3.31 -> 3.67 | Improved |
| add multi-task training to the TTS model (ablation) | TTS Baseline · Zero-shot TTS | MOS-Q: 3.65 -> 3.86 | Improved |
| add multi-task training to the TTS model (ablation) | TTS Baseline · Zero-shot TTS | WER: 3.32 -> 3.57 | Hurt |
| add multi-task training to the TTS model (ablation) | TTS Baseline · Zero-shot TTS | speaker similarity: .48 -> .53 | Improved |
| add multi-task training to the TTS model (ablation) | TTS Baseline · Speech editing | MOS-Q: 4.18 -> 4.15 | No clear effect |
| add implicit task composition with Whisper transcripts | SpeechOp enhancement · Speech enhancement | WER: 8.1 -> 2.9 | Improved |
| add implicit task composition with Whisper transcripts | SpeechOp enhancement · Speech enhancement | PESQ: 2.00 -> 2.05 | Improved |
| add speaker personalization to ITC enhancement | SpeechOp-ITC · Speech enhancement | MCD: 4.85 -> 4.69 | Improved |
| use TC-CFG instead of score averaging (TC-Avg) (ablation) | SpeechOp enhancement · Speech enhancement (gold transcript) | WER: 3.4 -> 2.1 | Improved |
| use TC-CFG instead of score averaging (TC-Avg) (ablation) | SpeechOp enhancement · Speech enhancement (gold transcript) | MCD: 5.24 -> 4.83 | Improved |
| use TC-CFG instead of score averaging (TC-Avg) (ablation) | SpeechOp enhancement · Speech enhancement (gold transcript) | PESQ: 1.88 -> 2.06 | Improved |
| use framewise mixing instead of cross-attention for source audio (ablation) | SpeechOp-Small · Speech enhancement | WER: 9.6 -> 8.8 | Improved |
Closest papers by meaning
- Metis: A Foundation Speech Generation Model with Masked Generative Pre-training (NeurIPS 2025 · similarity 0.87)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.87)
- DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors (ICLR 2025 · similarity 0.86)
- Generative Pre-training for Speech with Flow Matching (ICLR 2024 · similarity 0.85)
- Predictive–Generative Drift Decomposition for Speech Enhancement and Separation (NeurIPS 2026 · similarity 0.85)
- Long-Form Speech Generation with Spoken Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
