Research map / Image generation / Fast diffusion sampling
SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SITCOM, an optimization-based diffusion sampler for inverse problems that optimizes the diffusion network input at each step under three step-wise consistency conditions (measurement, network-regularized backward, forward), enabling far fewer reverse steps.
Problem
Diffusion-model-based inverse problem solvers struggle with measurement noise, nonlinear tasks, error accumulation from early steps, and require many sampling and optimization steps.
Approach
Optimize over the input v_t of the pre-trained denoiser (rather than its output) with a measurement misfit plus a proximity term to x_t, so the estimate stays on the diffusion trajectory via Tweedie's formula, then resample forward; this enforces measurement, backward (network-regularized), and forward consistency at every step, allowing N << T sampling steps.
Key results
- On FFHQ random inpainting, SITCOM reaches 32.05 dB PSNR vs 31.02 for DAPS, at about 27 s vs 93.6 s run-time.
- SITCOM achieves best performance in 42 of 51 reported metric cases across 8 tasks on FFHQ and ImageNet (per the authors).
- Ablation: removing backward (network) consistency drops PSNR by more than 5 dB on FFHQ super-resolution (e.g., 31.34 to 25.97 at N=20).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SITCOM rather than the strongest PSNR baseline | Image restoration with diffusion priors · Most reported restoration tasks across FFHQ and ImageNet | PSNR: 30.68 vs 30.18 (SR, FFHQ); 24.68 vs 24.37 (BIP, FFHQ); 32.05 vs 31.02 (RIP, FFHQ); 30.25 vs 29.79 (GDB, FFHQ); 30.34 vs | Helped |
| Use SITCOM rather than the strongest PSNR baseline | Image restoration with diffusion priors · GDB, ImageNet | PSNR: 27.40 vs 28.06 | Hurt |
| Use SITCOM rather than the strongest PSNR baseline | Image restoration with diffusion priors · PR, FFHQ | PSNR: 30.67 vs 30.97 | Hurt |
| Use SITCOM rather than the strongest PSNR baseline | Image restoration with diffusion priors · PR, ImageNet; NDB, both datasets | PSNR: 25.45 vs 25.76; 30.12 vs 30.31; 28.78 vs 29.51 | Hurt |
| Remove step-wise backward consistency (ablation) | SITCOM · SR, FFHQ, N=20 | PSNR: 31.34 -> 25.97 | Hurt |
| Remove step-wise backward consistency (ablation) | SITCOM · SR, FFHQ, N=100 | PSNR: 31.39 -> 26.1 | Hurt |
| Use K=20 instead of K=1 gradient updates for step-wise measurement consistency (ablation) | SITCOM · SR, FFHQ, N=20 | PSNR: 11.41 -> 30.40 | Helped |
| Use K=20 instead of K=1 gradient updates for step-wise measurement consistency (ablation) | SITCOM · Non-linear deblurring, FFHQ, N=20 | PSNR: 12.30 -> 30.27 | Helped |
| Replace resampling with traditional sampling at each step (ablation) | SITCOM · SR, FFHQ, N=20 | PSNR: 30.40 -> 20.78 | Hurt |
| Replace resampling with traditional sampling at each step (ablation) | SITCOM · Non-linear deblurring, FFHQ, N=20 | PSNR: 30.27 -> 22.42 | Hurt |
Surprising findings
- For phase retrieval, DAPS obtains slightly higher PSNR than SITCOM, which the authors attribute to SITCOM's Tweedie-based default; SITCOM-ODE improves this at extra cost.
- SITCOM with K=1 reduces to DPS plus resampling and performs poorly at small N (about 11 dB PSNR at N=20 for super-resolution).
Limitations
- Limited to non-blind settings with known forward operator.
- Only 2D diffusion models; extension to 3D needs specialized regularization.
- Stopping threshold delta depends on noise level estimate, which may be inaccurate.
Experimental setup
Datasets: FFHQ ImageNet
Benchmarks: FFHQ ImageNet
Metrics: PSNR SSIM LPIPS run-time
Baselines: DPS DAPS DDNM RED-diff PGDM DCDP DMPlug ReSample Latent-DAPS
Builds on: DPS DDNM DAPS ReSample CSGM RED-diff
Closest papers by meaning
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.91)
- Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency (ICLR 2024 · similarity 0.91)
- Solving Video Inverse Problems Using Image Diffusion Models (ICLR 2025 · similarity 0.90)
- Coupled Data and Measurement Space Dynamics for Enhanced Diffusion Posterior Sampling (NeurIPS 2025 · similarity 0.90)
- Fast samplers for Inverse Problems in Iterative Refinement models (NeurIPS 2024 · similarity 0.90)
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.