Research map / Image generation / Text-to-image diffusion
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReNO, which improves one-step text-to-image models at inference by gradient-ascent optimization of the initial noise against an ensemble of human-preference reward models, without updating model weights.
Problem
Text-to-image models struggle with complex compositional prompts, and reward fine-tuning suffers from reward hacking and poor generalization to unseen prompts; per-prompt fine-tuning and multi-step noise optimization are too expensive.
Approach
Backpropagate reward-model gradients through a single-step distilled T2I generator to optimize the initial Gaussian noise at inference, using a weighted combination of HPSv2, PickScore, ImageReward, and CLIPScore plus a chi-distribution noise regularizer, with few iterations to limit reward hacking.
Key results
- On T2I-CompBench, SD-Turbo color attribute rises from 0.55 to 0.78 with ReNO, near DALL-E 3 at 0.81.
- On GenEval, FLUX-schnell + ReNO reaches mean 0.72, above FLUX-dev at 0.68, reported as strongest open-source GenEval result.
- In user study, HyperSDXL + ReNO narrowly beats SD3 (8B) with 54% win rate, and SD-Turbo + ReNO reaches up to 77% win rate against SD-Turbo base.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize initial noise with ReNO | SD-Turbo · GenEval | Mean: 0.49 to 0.62 | Helped |
| Optimize initial noise with ReNO | FLUX-schnell · GenEval | Mean: 0.64 to 0.72 | Helped |
| Optimize initial noise with ReNO | SD-Turbo · T2I-CompBench Color | Color: 55% to 78% | Helped |
| Use all four reward models rather than CLIPScore alone (ablation) | SD-Turbo + ReNO · T2I-CompBench attribute binding, first 50 prompts | Shape: 66.9 to 77.4 | Helped |
| Use all four reward models rather than CLIPScore alone (ablation) | SD-Turbo + ReNO · T2I-CompBench attribute binding, first 50 prompts | Texture: 79.6 to 82.8 | Helped |
| Optimize initial noise using CLIPScore alone (ablation) | SD-Turbo · LAION aesthetic score | Aesthetic: 5.647 -> 5.475 | Hurt |
| Optimize initial noise using ImageReward alone (ablation) | SD-Turbo · T2I-CompBench attribute binding | Color: 0.5513 -> 0.7720 | Helped |
| Optimize initial noise using HPSv2 alone (ablation) | SD-Turbo · LAION aesthetic score | Aesthetic: 5.647 -> 5.752 | Helped |
| Optimize initial noise with ReNO | SD-Turbo · Parti-Prompts, 50 seeds | LPIPS similarity: 0.382 -> 0.246 | Helped |
| Optimize initial noise with ReNO | SDXL-Turbo · Parti-Prompts, 50 seeds | LPIPS similarity: 0.391 -> 0.291 | Helped |
| Use one-step SD-Turbo with ReNO instead of 50-step SD2.1 + DOODL | Noise optimization · T2I-CompBench attribute binding, first 50 prompts | Color: 38.5 (+5.1) -> 70.1 (+9.7) | Helped |
Surprising findings
- Optimizing the initial noise increases output diversity across seeds (lower LPIPS/DINO similarity) for one-step models, unlike their multi-step counterparts.
- Optimizing the LAION aesthetic predictor directly increased scores but visual quality did not always improve, suggesting reward hacking.
Limitations
- Performance of different one-step generators converges to similar levels, suggesting limits from reward models' compositional reasoning.
- VRAM use is high and scales with image size (39GB for HyperSDXL at 1024x1024).
- Models still struggle with humans, text rendering, and complex compositions; ReNO does not fix these fully.
Experimental setup
Datasets: T2I-CompBench GenEval Parti-Prompts
Benchmarks: T2I-CompBench GenEval Parti-Prompts
Metrics: BLIP-VQA score object detection score GenEval mean LAION aesthetic score LPIPS DINO similarity user preference win rate CLIPScore
Baselines: SD-Turbo SDXL-Turbo PixArt-alpha DMD HyperSDXL SDXL SD3 DALL-E 3 DOODL DPO-SDXL FLUX-schnell
Builds on: DOODL Adversarial Diffusion Distillation Diffusion Matching Distillation ImageReward HPSv2
Closest papers by meaning
- Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generation (CVPR 2025 · similarity 0.88)
- Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models (ICLR 2024 · similarity 0.88)
- Subject-driven Text-to-Image Generation via Preference-based Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist Rewards (NeurIPS 2026 · similarity 0.87)
- STEPS: Sequential Probability Tensor Estimation for Text-to-Image Hard Prompt Search (CVPR 2025 · similarity 0.87)
- TLDR: Token-Level Detective Reward Model for Large Vision Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.