Research map / Image generation / Fast diffusion sampling

Stitched Value Model for Diffusion Alignment

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StitchVM, which stitches a frozen diffusion backbone head to a pretrained pixel-space reward model tail with a light stitching layer and short finetuning, yielding value functions on noisy latents used for DPS, FK steering, DRaFT, and DiffusionNFT.

Problem

Alignment of diffusion/flow models needs value estimates on noisy latents, but rewards are defined on clean images; existing Tweedie and Monte Carlo approximations are biased or costly, and directly trained noisy-latent value models lack generality.

Approach

Stitch a truncated frozen diffusion backbone (head) to a sliced pretrained clean-image reward model (tail) via a linear stitching layer selected in closed form and fine-tuned briefly on unlabeled images, so the combined model acts as a noisy-latent value function without Tweedie or rollouts.

Key results

• DPS + StitchVM on SD3.5-Medium with HPSv2 reward reduces sampling time from 52.8 to 16.5 s/sample (3.2x) and peak memory from 56.4 to 26.0 GB while raising HPSv2 from 0.335 to 0.348.
• FKS + StitchVM on SD3.5-Medium with HPSv2 reward raises ImageReward from 0.93 (FKS) to 1.10 and GenEval from 0.62 to 0.69.
• DiffusionNFT + StitchVM cuts GPU-hours from 191.5 to 84.7 while matching or improving HPSv2, DFN, ImageReward, PickScore and GenEval.

Surprising findings

• VAE-stitching baseline collapses at high noise, showing diffusion features are essential for noisy-latent transfer.
• StitchVM-trained value models outperform NoisyCLIP trained at LAION-400M scale despite using only unlabeled images.

Limitations

• Tweedie baseline and value accuracy depend on reward model quality; StitchVM inherits reward model biases
• Experiments limited to text-to-image with a few reward models and backbones

Experimental setup

Datasets: AVA HPDv2 LAION-400M MSCOCO Flickr30K ImageReward DrawBench GenEval

Benchmarks: MSCOCO Flickr30K HPDv2 ImageReward AVA DrawBench GenEval

Metrics: Recall@1 preference accuracy SRCC ImageReward Aesthetic Score HPSv2 PickScore CLIP score GenEval peak GPU memory

Baselines: Tweedie approximation VIST3A NoisyCLIP DiNa-LRM FKS BoN DPS Flow-GRPO-fast DRaFT-1 DRaFT-3 DiffusionNFT

Builds on: model stitching FK steering DPS DiffusionNFT DRaFT

Closest papers by meaning

• T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching (ICLR 2025 · similarity 0.90)
• Inference-Time Alignment of Diffusion Models with Direct Noise Optimization (ICML 2025 · similarity 0.89)
• Training-Free Diffusion Model Alignment with Sampling Demons (ICLR 2025 · similarity 0.89)
• Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.89)
• Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.89)
• Composition and Alignment of Diffusion Models using Constrained Learning (NeurIPS 2025 · similarity 0.89)