Research map / Image generation / Autoregressive and latent generation
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a second-pass deliberation decoder on top of the first-pass draft | RNNSearch shallow NMT · WMT14 En→Fr newstest2014 | BLEU: +1.7 over Mbase (29.97 to 31.67) | Helped |
| add a second-pass deliberation decoder | RNNSearch shallow NMT · Zh→En NIST04-08 | BLEU: average gain of 1.26 over Mbase | Helped |
| add a second decoding layer without deliberation (Mdec×2) (ablation) | RNNSearch shallow NMT · WMT14 En→Fr newstest2014 | BLEU: +0.43 over Mbase (30.40 vs 29.97) | Helped |
| add a review network on the encoder side (4 reviewers) (ablation) | RNNSearch shallow NMT · WMT14 En→Fr newstest2014 | BLEU: +0.79 over Mbase (30.76 vs 29.97); deliberation is 0.91 better | Helped |
| add deliberation network on top of dual learning (ablation) | deep 4-layer LSTM NMT · WMT14 En→Fr | BLEU: +0.97 over dual learning (40.53 to 41.50) | Helped |
| add a second-pass deliberation decoder | RNNSearch summarization on Gigaword · Gigaword test | ROUGE-1: +3.45 over Mbase (27.45 to 30.90) | Helped |
| add a second-pass deliberation decoder | RNNSearch summarization on Gigaword · Gigaword test | ROUGE-2: +1.70 over Mbase (10.51 to 12.21) | Helped |
| add a second-pass deliberation decoder | RNNSearch summarization on Gigaword · Gigaword test | ROUGE-L: +3.02 over Mbase (26.07 to 29.09) | Helped |
| deliberation decoding roughly doubles decoding time | RNNSearch shallow NMT · 3003 En→Fr sentences decoding time | decoding time (s): 964 s to 1924 s | Hurt |
Closest papers by meaning
- Levenshtein Transformer (NeurIPS 2019 · similarity 0.86)
- MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models (NeurIPS 2018 · similarity 0.86)
- Syntactically Look-Ahead Attention Network for Sentence Compression (AAAI 2020 · similarity 0.86)
- MASS: Masked Sequence to Sequence Pre-training for Language Generation (ICML 2019 · similarity 0.86)
- DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models (ICLR 2023 · similarity 0.85)
- Multi-Agent Discussion Mechanism for Natural Language Generation (AAAI 2019 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.