Research map / Image generation / Text-to-image diffusion
Moment- and Power-Spectrum-Based Gaussianity Regularization for Text-to-Image Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Gaussianity regularizer combining spatial moment matching with a power-spectrum (FFT-based) loss, applied to randomly permuted latents, and uses it to reduce reward hacking in one-step FLUX test-time reward alignment.
Problem
Latent noise optimization for text-to-image reward alignment drifts away from the standard Gaussian prior, causing reward hacking and degraded image quality; existing Gaussianity regularizers capture only marginal statistics or are computationally costly (covariance-based PRNO is O(Dk)).
Approach
Treat latent elements as i.i.d. N(0,1) samples and regularize both spatial moments (analytically known) and spectral power (DFT magnitudes follow chi_2/sqrt2), with batched frequency-subset means to preserve variance and random permutations for permutation invariance; the spectral term is O(D log D) and subsumes PRNO's covariance objective.
Key results
- On aesthetic reward optimization at 500 iterations, the proposed method reaches aesthetic score 8.4435 vs 8.1508 for ReNO and keeps the best ImageReward (0.8397) and HPSv2 (0.2927) among compared methods.
- On PickScore optimization at 500 iterations, the method reaches PickScore 0.2548 vs 0.2514 for PRNO and the best HPSv2 (0.3028), with ImageReward 0.5897 close to PRNO's 0.6061.
- The authors report about 50x less time than PRNO for the toy latent-optimization experiment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use joint moment- and power-spectrum-based Gaussianity regularization instead of ReNO regularization | FLUX reward alignment · Aesthetic image generation, 500 iterations | ImageReward: 0.7749 -> 0.8397 | Helped |
| Use joint moment- and power-spectrum-based Gaussianity regularization instead of Kurtosis regularization | FLUX reward alignment · Aesthetic image generation, 500 iterations | HPSv2: 0.2920 -> 0.2927 | Helped |
| Use joint moment- and power-spectrum-based Gaussianity regularization instead of ReNO regularization | FLUX reward alignment · Text-aligned image generation, 500 iterations | HPSv2: 0.2945 -> 0.3028 | Helped |
| Use joint moment- and power-spectrum-based Gaussianity regularization instead of PRNO regularization | FLUX reward alignment · Text-aligned image generation, 500 iterations | ImageReward: 0.6061 -> 0.5897 | Hurt |
| Add power-spectrum regularization to spatial-only regularization (ablation) | Latent optimization for FLUX · Toy experiment, checkerboard initialization | Gaussianity | Helped |
| Replace per-frequency spectral negative log-likelihood with subset-mean power-spectrum regularization (ablation) | Gaussianity regularization · Latent Gaussianity | Gaussianity | Helped |
Surprising findings
- Minimizing the regularization loss value alone does not reliably indicate how Gaussian a latent is, per the authors' limitations section.
- The naive spectral negative log-likelihood loss over individual frequency coefficients overly sharpens the spectrum and degrades Gaussianity, motivating the batched-mean variant.
Limitations
- A low regularization loss does not guarantee full Gaussianity; no reliable Gaussianity metric is provided.
- Inherits biases and artifacts of the pretrained FLUX base model and uncurated training data.
- Evaluation limited to one-step FLUX with a 500-iteration budget and a small prompt set (60 prompts for text alignment).
Experimental setup
Datasets: DDPO animal prompts T2I-CompBench++
Benchmarks: DDPO animal prompts T2I-CompBench++
Metrics: aesthetic score PickScore ImageReward HPSv2
Baselines: KL regularization Kurtosis regularization ReNO (Norm) PRNO No Reg. No Opt.
Builds on: PRNO ReNO FLUX
Closest papers by meaning
- Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective (NeurIPS 2024 · similarity 0.85)
- GARDO: Reinforcing Diffusion Models without Reward Hacking (NeurIPS 2026 · similarity 0.85)
- DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.84)
- Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation (ICLR 2024 · similarity 0.84)
- InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization (CVPR 2024 · similarity 0.84)
- PoGDiff: Product-of-Gaussians Diffusion Models for Imbalanced Text-to-Image Generation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.