Research map / Image generation / Fast diffusion sampling

Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generation

NeurIPS 2025 · Image generation › Fast diffusion sampling · approach: Distillation for few-step generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reward-Instruct, a reward-centric method that converts pretrained diffusion models into 4-step text-to-image generators by SGLD-based reward maximization with weight regularization, random-eta sampling and gradient-normalized multi-reward training, without diffusion distillation losses or training images.

Problem

Fast few-step text-to-image generation aligned with human preferences normally requires expensive diffusion distillation losses alongside reward fine-tuning, which is costly and prone to reward hacking.

Approach

Treat few-step generator parameters as the optimization variable and sample from a reward-tilted parameter posterior via SGLD (reward gradient + Gaussian weight prior + injected noise), with random-eta sampling and gradient-normalized multi-reward weighting, replacing distillation as regularizer.

Key results

• Reward-Instruct with SD-v1.5 at 4 NFE reaches HPS 33.70 and ImageReward 1.22, versus 31.83 HPS for DI++ and 30.24 for Hyper-SD.
• Reward-Instruct+ achieves the best HPS (34.37), CLIP score (32.97) and ImageReward (1.27) among fast samplers on SD-v1.5.
• Reward-Instruct on SD3-Medium at 4 steps reaches HPS 34.04 and FID 31.97, compared to 31.37 HPS and FID 28.72 for the 56-NFE base model.

Measured effects reported in the paper

Surprising findings

• In re-implemented DI++, gradient norms show updates are dominated by the reward term, with distillation acting as a secondary regularizer.
• RG-LCM and DI++ collapse when trained with HPS v2.1 rather than v2.0 or ImageReward, while the proposed method remains stable.

Limitations

• Relies on differentiable reward functions; black-box rewards not supported.
• Single explicit reward leads to over-saturation, requiring multiple rewards.
• Not explored beyond diffusion-based few-step generators.

Experimental setup

Datasets: JourneyDB COCO-5k

Benchmarks: COCO-5k

Metrics: HPS v2.1 Aesthetic Score CLIP score FID ImageReward

Baselines: Hyper-SD RG-LCM DI++ ReFL DRaFT Base Model (Realistic-vision)

Builds on: Diff-Instruct++ RG-LCM DDIM classifier-free guidance SGLD

Closest papers by meaning

• Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL (NeurIPS 2026 · similarity 0.91)
• David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training (ICML 2025 · similarity 0.90)
• InstructVideo: Instructing Video Diffusion Models with Human Feedback (CVPR 2024 · similarity 0.89)
• Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation (CVPR 2025 · similarity 0.88)
• A Simple Approach to Unifying Diffusion-based Conditional Generation (ICLR 2025 · similarity 0.88)
• Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward (CVPR 2025 · similarity 0.88)