Research map / Architectures and efficiency / Attention and transformers
Latent Sequence Decompositions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn input-conditioned latent word-piece decompositions instead of fixed character outputs | Seq2seq ASR · WSJ eval92 | WER: 14.8 -> 12.9 | Helped |
| Add CNN encoder to seq2seq + LSD4 | Seq2seq + LSD4 · WSJ eval92 | WER: 12.9 -> 9.6 | Helped |
| Use LSD rather than fixed maximum-extension decomposition | WSJ ASR with n=2 vocabulary · WSJ eval92 | WER: 15.56 -> 13.15 | Helped |
| Use LSD rather than fixed maximum-extension decomposition | WSJ ASR with n=3 vocabulary · WSJ eval92 | WER: 15.61 -> 13.08 | Helped |
| Use LSD rather than fixed maximum-extension decomposition | WSJ ASR with n=4 vocabulary · WSJ eval92 | WER: 14.96 -> 12.88 | Helped |
| Use LSD rather than fixed maximum-extension decomposition | WSJ ASR with n=5 vocabulary · WSJ eval92 | WER: 15.03 -> 13.52 | Helped |
| Use fixed maximum-extension word-piece decomposition instead of character outputs | WSJ ASR · WSJ eval92 | WER: 14.76 -> 14.96; 14.76 -> 15.56; 14.76 -> 15.61; 14.76 -> 15.03 | Hurt |
| Vary maximum n-gram length from n=2 to n=3 (ablation) | LSD · WSJ eval92 | WER: 13.15 -> 13.08 | No clear effect |
| Vary maximum n-gram length from n=3 to n=4 (ablation) | LSD · WSJ eval92 | WER: 13.08 -> 12.88 | Helped |
| Vary maximum n-gram length from n=4 to n=5 (ablation) | LSD · WSJ eval92 | WER: 12.88 -> 13.52 | Hurt |
| Add epsilon-greedy uniform exploration over valid next tokens during training (ablation) | LSD · WSJ ASR | WER | Helped |
| Remove epsilon-greedy exploration from training (ablation) | LSD · WSJ ASR | WER | Hurt |
Closest papers by meaning
- Deconvolutional Latent-Variable Model for Text Sequence Matching (AAAI 2018 · similarity 0.88)
- Sequence-to-Sequence Learning via Shared Latent Representation (AAAI 2018 · similarity 0.88)
- Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum (NeurIPS 2024 · similarity 0.86)
- Separate and Reconstruct: Asymmetric Encoder-Decoder for Speech Separation (NeurIPS 2024 · similarity 0.85)
- Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions (ICML 2024 · similarity 0.84)
- Non-autoregressive Sequence-to-Sequence Vision-Language Models (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.