Research map / Image generation / Super-resolution
Diffusion Model for Dense Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffMatch, a conditional denoising diffusion model with a cascaded flow-upsampling stage that generates dense correspondence fields conditioned on initial flow and local matching cost, learning both data and prior terms.
Problem
Dense correspondence between image pairs struggles with textureless regions, repetitive patterns, large displacements, and noise, because discriminative learning-based methods only model the data term and not an explicit matching prior.
Approach
Reformulate dense matching as posterior estimation with a conditional diffusion model that generates the correspondence field, conditioned on a soft-argmax initial flow and a local matching cost, followed by a cascaded finetuned super-resolution diffusion model for flow upsampling.
Key results
- On HPatches original, DiffMatch achieves 18.84 average AEPE, the best among compared methods (PDCNet+ 18.91).
- On ETH3D, DiffMatch achieves 3.12 average AEPE, best overall, with largest gains at rates 13 and 15 (3.86 and 4.54).
- On ImageNet-C severity-5 corrupted HPatches, DiffMatch averages 34.86 AEPE versus 35.49 for PDCNet+; on corrupted ETH3D it reaches 26.45 versus 27.11 for PDCNet+.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiffMatch instead of PDCNet+ | Dense matching · HPatches Original | AEPE: 18.91 -> 18.84 | Helped |
| use DiffMatch instead of PDCNet+ | Dense matching · ETH3D | AEPE: 3.14 -> 3.12 | Helped |
| use DiffMatch instead of PDCNet+ | Dense matching under ImageNet-C corruptions · HPatches ImageNet-C severity 5 | AEPE (Avg.): 35.49 -> 34.86 | Helped |
| use DiffMatch instead of PDCNet+ | Dense matching under ImageNet-C corruptions · ETH3D ImageNet-C severity 5 | AEPE (Avg.): 27.11 -> 26.45 | Helped |
| use DiffMatch instead of PDC-Net+ | Dense matching · MegaDepth | AEPE: 63.97 -> 59.73 | Helped |
| add diffusion denoising rather than directly regress the matching field (ablation) | DiffMatch · HPatches | AEPE: 23.34 -> 18.82 | Helped |
| add diffusion denoising rather than directly regress the matching field (ablation) | DiffMatch · ETH3D | AEPE: 3.96 -> 3.12 | Helped |
| add local cost conditioning (ablation) | Conditional denoising diffusion · ETH3D | AEPE: 4.26 -> 3.44 | Helped |
| add local cost conditioning (ablation) | Conditional denoising diffusion · C-ETH3D | AEPE: 32.07 -> 26.89 | Helped |
| add initial flow conditioning (ablation) | Conditional denoising diffusion · ETH3D | AEPE: 10.28 -> 3.44 | Helped |
| add initial flow conditioning (ablation) | Conditional denoising diffusion · C-ETH3D | AEPE: 80.81 -> 26.89 | Helped |
| add flow upsampling diffusion (ablation) | Conditional denoising diffusion · ETH3D | AEPE: 3.44 -> 3.12 | Helped |
Surprising findings
- Conditioning on concatenated source and target features instead of the proposed initial flow and local cost gives very poor ETH3D AEPE (106.83), far worse than the proposed scheme.
- The model is slightly worse than baselines on ETH3D at small-displacement intervals, which the authors attribute to lower input resolution.
Limitations
- Slightly lower performance than prior methods on ETH3D at small displacement intervals, attributed to low input resolution (64 for the base model, 256 after upsampling).
- Diffusion inference is iterative and slower than feed-forward matchers, mitigated by using few steps.
- Uses frozen VGG-16 features, which limits accuracy.
Experimental setup
Datasets: DPED-CityScape-ADE COCO HPatches ETH3D ImageNet-C MegaDepth
Benchmarks: HPatches ETH3D ImageNet-C corrupted HPatches ImageNet-C corrupted ETH3D MegaDepth
Metrics: AEPE
Baselines: GLU-Net GLU-Net-GOCor PDCNet PDCNet+ COTR DGC-Net DMP DKM
Builds on: DDPM DDIM Palette Cascaded Diffusion Model PDC-Net+
Closest papers by meaning
- Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models (ICLR 2025 · similarity 0.88)
- SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching (CVPR 2024 · similarity 0.88)
- Exploiting Diffusion Prior for Generalizable Dense Prediction (CVPR 2024 · similarity 0.87)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.87)
- Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.87)
- Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.