Research map / Image generation / GANs
Training Language GANs from Scratch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train from scratch with large batches, dense rewards, and discriminator regularization rather than using maximum-likelihood pretraining | ScratchGAN · EMNLP2017 News | FED: 0.084 -> 0.015 | Helped |
| Increase the batch size (ablation) | ScratchGAN · EMNLP2017 News | FED | Helped |
| Add discriminator regularization with L2 weight decay, dropout, and layer normalization (ablation) | ScratchGAN · EMNLP2017 News | FED | Helped |
| Add pretrained GloVe embeddings (ablation) | ScratchGAN · EMNLP2017 News | FED | No clear effect |
| Add a value baseline for REINFORCE (ablation) | ScratchGAN · EMNLP2017 News | FED | Mixed |
| Add sinusoidal positional information to the discriminator (ablation) | ScratchGAN · EMNLP2017 News | sentence-length distribution match | Helped |
| Use a Wasserstein loss on generator logits with a straight-through gradient (ablation) | ScratchGAN · Language generation | training performance | Hurt |
| Use ensembles of discriminators and generators instead of one pair (ablation) | ScratchGAN · Language generation | training performance | No clear effect |
| Train against past versions of generators and discriminators (ablation) | ScratchGAN · Language generation | training performance | No clear effect |
| Use a bidirectional discriminator (ablation) | ScratchGAN · Language generation | usefulness of generator feedback | Hurt |
| Use multiple discriminators with different architectures (ablation) | ScratchGAN · Language generation | training performance | No clear effect |
| Train on a small dataset such as Penn Tree Bank (ablation) | ScratchGAN · Penn Tree Bank | discriminator generalization | Hurt |
Closest papers by meaning
- Language GANs Falling Short (ICLR 2020 · similarity 0.90)
- Branch-GAN: Improving Text Generation with (not so) Large Language Models (ICLR 2024 · similarity 0.90)
- First Order Generative Adversarial Networks (ICML 2018 · similarity 0.90)
- Are GANs overkill for NLP? (NeurIPS 2022 · similarity 0.90)
- A Large-Scale Study on Regularization and Normalization in GANs (ICML 2019 · similarity 0.89)
- Are GANs Created Equal? A Large-Scale Study (NeurIPS 2018 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.