Research map / Probabilistic and scientific modeling / Variational and generative sampling
$\Psi$-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Ψ-Sampler, which initializes SMC particles from the reward-aware initial posterior using pCNL (preconditioned Crank–Nicolson Langevin) MCMC, improving inference-time reward alignment of FLUX on layout, counting, and aesthetic tasks.
Problem
Inference-time reward alignment with score-based generative models using SMC is limited because particles start from the reward-agnostic Gaussian prior, and standard Langevin MCMC (MALA) degenerates in very high-dimensional latent spaces.
Approach
Sample initial particles from the reward-tilted posterior at t=1 with pCNL, a dimension-robust MH-corrected Langevin sampler, then run TDS/DAS-style SMC over the denoising steps; the paper claims it is the first use of pCN in generative modeling.
Key results
- On layout-to-image, Ψ-Sampler reaches mIoU 0.471 (held-out) versus 0.427 for Top-K-of-N and 0.402 for TDS.
- On quantity-aware generation, the T2I-Count loss drops from 1.804 (TDS) to 1.077 (Top-K-of-N) and 0.850 (Ψ-Sampler); accuracy rises to 32.5%.
- MALA's acceptance probability falls to near zero as step size grows, while pCNL stays stable; pCNL at step 2.0 keeps diversity comparable to MALA at 0.05.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace prior initialization with Top-K-of-N posterior initialization | SMC · Layout-to-image generation | mIoU: 0.402 -> 0.425 | Helped |
| Replace prior initialization with Ψ-Sampler pCNL posterior initialization | SMC · Layout-to-image generation | mIoU: 0.417 -> 0.467 | Helped |
| Replace prior initialization with Top-K-of-N posterior initialization | SMC · Quantity-aware generation | T2I-Count: 1.804 -> 1.077 | Helped |
| Replace prior initialization with Ψ-Sampler pCNL posterior initialization | SMC · Quantity-aware generation | T2I-Count: 1.804 -> 0.850 | Helped |
| Use Ψ-Sampler instead of MALA for posterior initialization | SMC · Layout-to-image generation | GroundingDINO: 0.401 -> 0.467 | Helped |
| Use Ψ-Sampler instead of Top-K-of-N for posterior initialization | SMC · Layout-to-image generation | VQA: 0.855 -> 0.810 | Hurt |
| Use Ψ-Sampler instead of Top-K-of-N for posterior initialization | SMC · Quantity-aware generation | VQA: 0.960 -> 0.951 | Hurt |
| Use Ψ-Sampler instead of prior-initialized TDS | SMC · Quantity-aware generation | accuracy: 27.5 -> 32.5 | Helped |
| Increase the pCNL step size to 2.0 rather than using MALA at 0.05 (ablation) | Initial particle sampling · Layout-to-image generation | sample diversity (LPIPS MPD) | No clear effect |
| Increase the MALA step size (ablation) | Initial particle sampling · Layout-to-image generation | acceptance probability | Hurt |
| Increase the pCNL step size (ablation) | Initial particle sampling · Layout-to-image generation | acceptance probability | No clear effect |
Surprising findings
- ULA and MALA initializations give only marginal gains or even degrade performance relative to prior-based SMC in several tasks.
Limitations
- Requires differentiable reward models.
- Depends on accurate Tweedie approximations at early denoising steps.
- Evaluation limited to 50 layout cases and 40 quantity cases, with a fixed-step, non-adaptive sampler.
Experimental setup
Datasets: HRS-Spatial custom quantity-aware dataset built with GPT-4o (40 cases)
Benchmarks: HRS-Spatial (layout-to-image, 50 cases) custom quantity-aware set (40 cases) aesthetic preference prompts
Metrics: mIoU GroundingDINO score ImageReward VQAScore MAE counting accuracy aesthetic score LPIPS MPD acceptance probability
Baselines: DPS FreeDoM TDS DAS Top-K-of-N ULA MALA
Builds on: sequential monte carlo twisted diffusion sampler (TDS) divide-and-conquer sampler (DAS) preconditioned Crank-Nicolson metropolis-adjusted langevin algorithm
Closest papers by meaning
- Inference-Time Alignment of Diffusion Models with Direct Noise Optimization (ICML 2025 · similarity 0.88)
- Learnable Chernoff Baselines for Provable Inference-Time Alignment (NeurIPS 2026 · similarity 0.88)
- Test-time Alignment of Diffusion Models without Reward Over-optimization (ICLR 2025 · similarity 0.88)
- Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models (NeurIPS 2026 · similarity 0.87)
- VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editing (NeurIPS 2026 · similarity 0.87)
- Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.