Research map / Large language models / Language, safety and interpretability

Branch-GAN: Improving Text Generation with (not so) Large Language Models

ICLR 2024 · Large language models › Language, safety and interpretability · approach: Text generation and evaluation metrics

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Branch-GAN, a GAN-style adversarial training approach for open-domain text generation that aims to improve on repetition and looping in smaller language models.

Problem

Open-domain text generation from Transformer LLMs suffers from repetitive text, looping, and lack of robustness; adversarial training for text has been hard to apply.

Approach

Uses adversarial (GAN-style) training to improve text generation quality, aiming to make smaller language models competitive with larger ones; the abstract is truncated so the exact mechanism is not stated.

Closest papers by meaning

• Long Text Generation via Adversarial Training with Leaked Information (AAAI 2018 · similarity 0.92)
• MaskGAN: Better Text Generation via Filling in the _______ (ICLR 2018 · similarity 0.92)
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation (AAAI 2020 · similarity 0.90)
• Training Language GANs from Scratch (NeurIPS 2019 · similarity 0.90)
• Adversarial Text Generation via Feature-Mover's Distance (NeurIPS 2018 · similarity 0.90)
• TextGAIL: Generative Adversarial Imitation Learning for Text Generation (AAAI 2021 · similarity 0.90)