Research map / Image generation / Autoregressive and latent generation
RecDiffusion: Rectangling for Image Stitching with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RecDiffusion, a two-stage diffusion framework (motion diffusion model producing warp fields, then content diffusion model refining results with a weighted sampling mask) for rectangling irregular boundaries of stitched images.
Problem
Stitched images have non-rectangular boundaries; existing fixes crop content, inpaint unrelated content, or warp and introduce distortion and artifacts.
Approach
Reformulates image rectangling as conditional diffusion: a Motion Diffusion Model generates rectangling motion fields to warp the stitched image, and a Content Diffusion Model refines it, with a confidence-weighted sampling mask (inspired by Rank-Nullity/DPS-style range-null decomposition) that keeps confident warped pixels and lets CDM fill the rest at each step.
Key results
- On DIR-D, RecDiffusion reaches SSIM 0.7733, PSNR 22.21 and FID 3.63, versus 0.7141 / 21.28 / 4.14 for Nie et al.
- Ablation: CDM plus weighted sampling mask raises SSIM from 0.3129 (MDM only) to 0.7733 on DIR-D.
- Against inpainting baselines, PSNR 22.21 vs 14.49 (Palette) and 14.23 (Stable Diffusion 2.1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RecDiffusion instead of Nie et al. | Image rectangling · DIR-D test set | SSIM: 0.7141 -> 0.7733 | Helped |
| Use RecDiffusion instead of Nie et al. | Image rectangling · DIR-D test set | PSNR: 21.28 -> 22.21 | Helped |
| Use RecDiffusion instead of Nie et al. | Image rectangling · DIR-D test set | FID: 4.14 -> 3.63 | Helped |
| Add stitched-image mask condition (ablation) | MDM at 256 × 192 · DIR-D test set | SSIM: 0.6125 -> 0.7337 | Helped |
| Increase resolution from 256 × 192 to 512 × 384 (ablation) | MDM with stitched-image mask condition · DIR-D test set | SSIM: 0.7337 -> 0.7580 | Helped |
| Add CDM after MDM (ablation) | RecDiffusion · DIR-D test set | SSIM: 0.3129 -> 0.7618 | Helped |
| Add weighted sampling mask (ablation) | MDM + CDM · DIR-D test set | SSIM: 0.7618 -> 0.7733 | Helped |
| Add weighted sampling mask (ablation) | MDM + CDM · DIR-D test set | PSNR: 22.03 -> 22.21 | Helped |
| Use CDM without MDM (ablation) | RecDiffusion · DIR-D test set | SSIM: 0.7733 -> 0.3129 | Hurt |
Surprising findings
- CDM alone is ineffective, generating images with different illuminations.
- Without the stitched-image mask condition, the MDM cannot outperform the pseudo-motion-field baseline.
Limitations
- Diffusion sampling is slow (200 CDM sampling steps), and compute cost is high.
- Zero-shot generalization evaluated only on one additional dataset (APAP-conssite).
Experimental setup
Datasets: DIR-D
Benchmarks: DIR-D APAP-conssite
Metrics: FID SSIM PSNR
Baselines: He et al. rectangling Nie et al. deep rectangling Palette Stable Diffusion 2.1 inpainting
Builds on: DDIM classifier-free guidance Rank-Nullity range-null space sampling Nie et al. deep rectangling
Closest papers by meaning
- Diffusing Differentiable Representations (NeurIPS 2024 · similarity 0.86)
- Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model (NeurIPS 2024 · similarity 0.86)
- Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow (ICLR 2025 · similarity 0.86)
- Move Anything with Layered Scene Diffusion (CVPR 2024 · similarity 0.85)
- RAD: Region-Aware Diffusion Models for Image Inpainting (CVPR 2025 · similarity 0.85)
- DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.