Research map / Reinforcement learning / RL from feedback

PRECISE: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: RL fine-tuning of generative models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PRECISE, an SDE-consistent stochastic sampler with a logSNR-based exploration schedule and a closed-form transition under a frozen clean-latent posterior mean, for online GRPO-style RL post-training of flow-matching models.

Problem

Converting the deterministic flow-matching ODE into a stochastic policy for online RL is coupled with an exploration-stability tradeoff and few-step discretization errors (excess noise in Euler samplers, marginal bias in CPS) at small NFE budgets.

Approach

Derive an exploration schedule eps_t = eta*sqrt(t/(1-t)) from logSNR progress balance, and an exact Gaussian transition under the assumption that the clean-latent posterior mean is frozen over one step, yielding a sampler that avoids Euler's excess noise and CPS's covariance contraction.

Key results

• On SD3.5-M 10-NFE, PRECISE reaches PickScore 23.745, CLIPScore 1.038, HPSv2.1 0.391, the best among samplers on all three training rewards.
• PRECISE reaches the best prior sampler's reward targets with 13.1%-53.2% fewer iterations in the 30-NFE protocol.
• On FLUX.2 Klein 4B at 20-NFE, PRECISE ranks first on five of six metrics, with PickScore 22.791 and ImageReward 1.410.

Surprising findings

• Dance-GRPO's constant noise schedule collapses training and corrupts fine details, and DiffusionNFT diverged when accelerated to 10-NFE training.
• CPS remains biased toward the inner ring of a double-ring toy distribution even at N=1280 NFE.

Limitations

• The frozen posterior-mean approximation is local and only holds approximately for non-degenerate data; validated mainly empirically via a stability plot.
• Experiments are limited to two flow-matching backbones and LoRA fine-tuning with GRPO-Guard.
• Default eta was tuned per method, and the logSNR argument for eta around sqrt(2) is heuristic.

Experimental setup

Datasets: Pick-a-Pic GenEval prompts COCO categories

Benchmarks: GenEval PickScore CLIPScore HPSv2.1 ImageReward UnifiedReward v2 Aesthetics

Metrics: PickScore CLIPScore HPSv2.1 ImageReward UnifiedReward v2 Aesthetics GenEval score wall-clock training time to reach target reward

Baselines: Flow-GRPO Dance-GRPO CPS DiffusionNFT

Builds on: Flow-GRPO Dance-GRPO CPS GRPO-Guard DDIM

Closest papers by meaning

• Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data (NeurIPS 2025 · similarity 0.91)
• Flow-GRPO: Training Flow Matching Models via Online RL (NeurIPS 2025 · similarity 0.91)
• Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints (NeurIPS 2026 · similarity 0.90)
• Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View (NeurIPS 2026 · similarity 0.90)
• Flow-DPPO: Divergence Proximal Policy Optimization for Flow Matching Models (NeurIPS 2026 · similarity 0.89)
• Elucidating Flow Matching ODE Dynamics via Data Geometry and Denoisers (ICML 2025 · similarity 0.89)