Research map / Image generation / Fast diffusion sampling
Tailoring Language Generation Models under Total Variation Distance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MLE with TaiLr objective (token-level weighted log-likelihood) | LSTM trained on synthetic COCO data · Synthetic COCO oracle | PPL_oracle and SelfBLEU-4: PPL_oracle 31.64 -> 26.91; SelfBLEU-4 33.27 -> 28.91 | Helped |
| Replace MLE with TaiLr objective | Transformer on IWSLT14 De-En · IWSLT14 De-En test | Test BLEU: 34.27 -> 35.05 | Helped |
| Replace MLE with TaiLr objective | BART-base finetuned on Gigaword · Gigaword test | ROUGE-L: 35.70 -> 36.24 | Helped |
| Replace MLE with TaiLr objective | BART-base finetuned on WritingPrompts · WritingPrompts test | Mauve: 56.42 -> 64.64 | Helped |
| Replace MLE with TaiLr objective | BART-base finetuned on WritingPrompts · WritingPrompts test | rep-8 (lower better): 10.31 -> 9.73 | Helped |
| Replace MLE with TaiLr objective on BiBERT (stronger SOTA model) (ablation) | BiBERT on IWSLT14 De-En one-way training · IWSLT14 De-En test | Test BLEU: 38.01 -> 39.12 | Helped |
| Replace MLE with TaiLr objective on BiBERT in dual-directional setting (ablation) | BiBERT on IWSLT14 De-En · IWSLT14 De-En test (dual-directional + fine-tuning) | Test BLEU: 38.73 -> 39.23 | Helped |
| Replace MLE with TaiLr objective in a Transformer trained from scratch on GPT2-Medium synthetic data (ablation) | 6-layer Transformer synthetic data · GPT2-Medium oracle synthetic | PPL_oracle, PPL_test, BLEU-4, SelfBLEU-4: PPL_oracle 138.35 -> 137.59; PPL_test 260.09 -> 253.01; SelfBLEU-4 6.38 -> 5.94 | Helped |
| Replace MLE with unlikelihood training | Transformer on IWSLT14 De-En · IWSLT14 De-En test | Test BLEU: 34.27 -> 32.82 | Hurt |
| Replace MLE with loss truncation | Gigaword and WritingPrompts · WritingPrompts test | B-1 and Mauve: Mauve 56.42 -> 45.35; B-1 27.85 -> 19.49 | Hurt |
| Replace MLE with GOLD-delta | BART-base on WritingPrompts · WritingPrompts test | Mauve and D-4: Mauve 56.42 -> 15.44; D-4 84.28 -> 46.98 | Hurt |
| Tune gamma (proxy mixture) on WritingPrompts and IWSLT14 (ablation) | TaiLr · WritingPrompts and IWSLT14 De-En | Mauve and BLEU | Mixed |
Closest papers by meaning
- Language Generation with Strictly Proper Scoring Rules (ICML 2024 · similarity 0.89)
- Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Models (ICLR 2024 · similarity 0.89)
- MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods (ICLR 2024 · similarity 0.88)
- Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.88)
- The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation (ICLR 2025 · similarity 0.87)
- Closing the Curious Case of Neural Text Degeneration (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.