Research map / Label-efficient and robust learning / Semi-supervised learning
Revisiting Self-Training for Neural Sequence Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add noisy self-training with input perturbation and dropout | Base Transformer · WMT English-German 100K | BLEU: 15.6 -> 21.4 | Helped |
| add noisy self-training with input perturbation and dropout | Big Transformer · WMT English-German 3.9M | BLEU: 28.3 -> 29.3 | Helped |
| add noisy self-training with input perturbation and dropout | English-Nepali translation · FloRes, En-Origin | BLEU: 6.7 -> 8.9 | Helped |
| add noisy self-training with input perturbation and dropout | English-Nepali translation · FloRes, Ne-Origin | BLEU: 2.3 -> 3.5 | Helped |
| add noisy self-training with input perturbation and dropout | Base Transformer · Gigaword 100K | ROUGE-1: 30.4 -> 34.1 | Helped |
| use BT instead of noisy self-training | Gigaword summarization · Gigaword 640K | ROUGE-1: 36.6 -> 37.3 | Helped |
| use beam search instead of sampling to generate pseudo targets (ablation) | self-training with dropout · WMT100K | BLEU: 17.0 -> 17.5 | Helped |
| add dropout during pseudo-training (ablation) | self-training with beam search · WMT100K | BLEU: 16.3 -> 17.5 | Helped |
| add dropout during pseudo-training (ablation) | noisy self-training with beam search · WMT100K | BLEU: 17.9 -> 19.3 | Helped |
| add noisy self-training with input perturbation | self-training · WMT100K | BLEU: 17.5 -> 19.3 | Helped |
| add noisy self-training with input perturbation | LSTM · toy sum dataset | test error: 7.6 -> 4.5 | Helped |
| use separate pseudo-training and fine-tuning instead of joint training (ablation) | noisy self-training · WMT100K | BLEU: 18.8 -> 21.8 | Helped |
Closest papers by meaning
- Uncertainty-aware Self-training for Few-shot Text Classification (NeurIPS 2020 · similarity 0.88)
- Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data (ICLR 2021 · similarity 0.88)
- Generating Sequences by Learning to Self-Correct (ICLR 2023 · similarity 0.87)
- Towards Understanding Self-Pretraining for Sequence Classification (NeurIPS 2026 · similarity 0.87)
- How unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis (ICLR 2022 · similarity 0.87)
- Self Iterative Label Refinement via Robust Unlabeled Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.