Research map / Architectures and efficiency / Attention and transformers
Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment
Xiang Li, Yixuan Zhou, Jingran Xie, Zhiyong Wu, Hui Wang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-guidance feature mapping loss (ablation) | XCodec2 (65536) · LibriSpeech test-clean | PESQ: 2.28 -> 2.39 | Improved |
| add self-guidance feature mapping loss (ablation) | XCodec2 (8192) · LibriSpeech test-clean | PESQ: 2.03 -> 2.13 | Improved |
| add self-guidance feature mapping loss (ablation) | XCodec2 (65536) · LibriSpeech test-clean | WER: 3.2 -> 3.2 | No clear effect |
| add self-guidance feature mapping loss (ablation) | XCodec2 (16384) · LibriSpeech test-clean | WER: 3.47 -> 3.53 | Hurt |
| add self-guidance to BigCodec decoder (ablation) | BigCodec 40Hz · LibriSpeech test-clean | PESQ-WB: 1.6740 -> 1.7650 | Improved |
| add self-guidance to SimVQ quantizer codec (ablation) | XCodec2 SimVQ · LibriSpeech test-clean | PESQ: 2.10 -> 2.17 | Improved |
| add self-guidance to ResidualFSQ codec (ablation) | XCodec2 ResidualFSQ · LibriSpeech test-clean | PESQ: 1.75 -> 1.86 | Improved |
| add self-guidance loss (ablation) | XCodec2 decoder · Hidden feature alignment | Procrustes residuals: 0.265 -> 0.171 | Improved |
| add self-guidance loss (ablation) | XCodec2 · Quantization error mean (65536) | error mean: 0.858 -> 0.851 | No clear effect |
| add self-guidance loss (ablation) | XCodec2 TTS (16384) · LibriTTS test-clean TTS | WER: 28.78 -> 28.02 | Improved |
| add self-guidance loss (ablation) | XCodec2 TTS (65536) · LibriTTS test-clean TTS | WER: 33.03 -> 35.07 | Hurt |
| reduce codebook size from 65536 to 16384 (ablation) | XCodec2 TTS · LibriTTS test-clean TTS | WER: 33.03 -> 28.78 | Improved |
Closest papers by meaning
- WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling (ICLR 2025 · similarity 0.87)
- Scaling Transformers for Low-Bitrate High-Quality Speech Coding (ICLR 2025 · similarity 0.86)
- Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers (NeurIPS 2024 · similarity 0.86)
- ELLA-V: Stable Neural Codec Language Modeling with Alignment-Guided Sequence Reordering (AAAI 2025 · similarity 0.85)
- Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model (AAAI 2025 · similarity 0.85)
- CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
