Research map / Image generation / Text-to-image diffusion
Text Diffusion with Reinforced Conditioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TReC, a continuous text diffusion model that uses REINFORCE-based reinforced self-conditioning and time-aware variance scaling to fix self-conditioning degradation and train/sampling misalignment, evaluated on MT, paraphrase and question generation.
Problem
Continuous text diffusion models underperform because self-conditioning degrades during training (the denoiser ignores the noised latent) and training and sampling variances are misaligned, hurting non-autoregressive sequence generation.
Approach
Treat self-conditioned vs. non-self-conditioned denoising as two RL agents and reward the BLEU improvement of self-conditioning via a clipped REINFORCE advantage, plus a time-dependent variance scale lambda(t)=k1+k2t in the forward process so that the reverse-sampling variance is smaller than the training variance.
Key results
- On QQP, TReC (b=20) reaches 33.26 BLEU versus 30.82 for Difformer (b=20) and 30.86 for the Transformer AR baseline (b=5).
- On Quasar-T, TReC (b=20) reaches 21.37 BLEU versus 20.11 for Difformer and 17.45 for the beam-5 Transformer.
- On WMT14 En-De, TReC (b=20) reaches 27.55 BLEU, slightly above the beam-5 Transformer's 27.37 and CMLM's 27.03.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reinforced conditioning (ablation) | TReC with time-aware variance scaling · QQP | BLEU: 31.95 -> 33.19; 32.54 -> 33.26; 31.86 -> 32.11; 32.30 -> 32.60 | Helped |
| use time-aware rather than fixed variance scaling (ablation) | TReC without reinforced conditioning · QQP | BLEU: 30.48 -> 31.95; 30.71 -> 32.54; 29.90 -> 31.86; 30.49 -> 32.30 | Helped |
| add variance scaling (ablation) | TReC without reinforced conditioning or variance scaling · QQP | BLEU: 28.08 -> 31.95; 28.85 -> 32.54; 27.31 -> 31.86; 28.49 -> 32.30 | Helped |
| increase candidate set size from b=10 to b=20 (ablation) | TReC · QQP | BLEU: 33.19 -> 33.26; 32.11 -> 32.60 | Helped |
| use perplexity rather than BLEU for MBR reranking (ablation) | TReC · QQP | BLEU: 33.19 vs 32.11; 33.26 vs 32.60 | Helped |
| use TReC rather than Transformer (b=5) | sequence generation models · IWSLT14 De-En | BLEU: 33.59* -> 33.31 | Hurt |
| use TReC rather than Transformer (b=5) | sequence generation models · WMT14 En-De | BLEU: 27.37* -> 27.55 | Helped |
| use TReC rather than Transformer (b=5) | sequence generation models · QQP | BLEU: 30.86‡‡ -> 33.26 | Helped |
| use TReC rather than Difformer (b=20) | sequence generation models · Quasar-T | BLEU: 20.11‡‡ -> 21.37 | Helped |
Surprising findings
- Vanilla self-conditioning first improves and then degrades during training, with the self-conditioned advantage going below zero.
- Without reinforced conditioning, the denoiser is nearly insensitive to z_t, depending mostly on the previous prediction z_0-hat.
Limitations
- Reward is BLEU-based, a tractable but imperfect proxy for quality.
- Time-aware scaling hyperparameters k1, k2 chosen from preliminary experiments only.
Experimental setup
Datasets: IWSLT14 De-En WMT14 En-De QQP Quasar-T
Benchmarks: IWSLT14 De-En WMT14 En-De QQP Quasar-T
Metrics: BLEU
Baselines: Transformer GPVAE-Finetuned T5 Levenshtein CMLM DiffuSeq SeqDiffuSeq DiNoiSer Difformer
Builds on: Self-conditioning (Analog Bits) DiffuSeq Difformer DiNoiSer REINFORCE
Closest papers by meaning
- Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.89)
- Scaling Diffusion Language Models via Adaptation from Autoregressive Models (ICLR 2025 · similarity 0.89)
- The Diffusion Duality (ICML 2025 · similarity 0.89)
- Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.88)
- Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.