Research map / Architectures and efficiency / Pruning and efficient tuning
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use winning-ticket initializations rather than random-ticket initializations | LSTM language model · Wikitext-2 | log perplexity | Helped |
| Prune up to 90% of parameters using winning tickets with late rewinding | LSTM language model · Wikitext-2 | log perplexity: 90% | No clear effect |
| Remove late rewinding (ablation) | LSTM language model · Wikitext-2 | log perplexity | Hurt |
| Replace iterative pruning with one-shot pruning (ablation) | LSTM language model · Wikitext-2 | log perplexity | Hurt |
| Use winning-ticket rather than random-ticket initializations | Transformer · WMT'14 Newstest'14 | BLEU | Helped |
| Add iterative pruning (ablation) | Transformer · WMT'14 Newstest'14 | BLEU | Helped |
| Add late rewinding (ablation) | Transformer · WMT'14 Newstest'14 | BLEU | Helped |
| Prune only Transformer layer weights while leaving embeddings intact (ablation) | Transformer Base winning tickets · WMT'14 Newstest'14 | BLEU | Mixed |
| Train a 67%-pruned winning-ticket model from scratch | Transformer Big · WMT'14 Newstest'14 | BLEU: 28.9 BLEU 67% pruned vs. 29.2 BLEU unpruned | No clear effect |
| Use winning-ticket rather than random-ticket initializations | Classic-control RL agents · CartPole-v0, Acrobot-v1, LunarLander-v2 | episodic reward | Helped |
| Use winning-ticket rather than random-ticket initializations | Atari RL agents · Atari games | episodic reward | Mixed |
| Add late rewinding (ablation) | RL agents · CartPole-v0, Acrobot-v1, LunarLander-v2, Assault, Breakout, Seaquest | episodic reward | Helped |
Closest papers by meaning
- One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers (NeurIPS 2019 · similarity 0.93)
- Winning Lottery Tickets in Deep Generative Models (AAAI 2021 · similarity 0.92)
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · similarity 0.91)
- Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks (NeurIPS 2021 · similarity 0.91)
- Playing Lottery Tickets with Vision and Language (AAAI 2022 · similarity 0.91)
- Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Win (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.