Research map / Image generation / Fast diffusion sampling
FlowDiffuser: Advancing Optical Flow Estimation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reformulates optical flow estimation as conditional denoising diffusion with a RAFT-style recurrent denoising decoder and Hidden State Denoising, achieving SOTA generalization on Sintel and KITTI.
Problem
Optical flow estimation is usually treated as discriminative regression, which generalizes poorly when trained on synthetic or constrained data and struggles with occlusion, blur and brightness changes; existing diffusion-based flow (DDVM) is computationally expensive.
Approach
Generate flow via a noise-to-flow conditional diffusion process whose decoder is a RAFT-like recurrent denoising decoder (Conditional-RDD) with Hidden State Denoising, which predicts a latent intermediate flow from hidden states and uses DDIM-style reverse updates, plus an Embedding Enhancement module for time embeddings.
Key results
- FlowDiffuser achieves 0.86 clean and 2.19 final EPE on Sintel train and 3.61 EPE / 11.8% F1-all on KITTI train, with average rank 1.1 across 7 metrics in Table 1.
- Plugging Conditional-RDD into RAFT, GMA, KPA-Flow and SKFlow gives average gains of 3.1% to 9.3% on Sintel and KITTI train (Table 3).
- Ablation shows K=3 denoising steps is the default and HSD improves all metrics without extra computation (Table 4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FlowDiffuser instead of FlowFormer++ | Optical flow estimation · Sintel train clean | EPE: 0.90 -> 0.86 | Helped |
| Use FlowDiffuser instead of FlowFormer++ | Optical flow estimation · KITTI-15 train | EPE: 3.93 -> 3.61 | Helped |
| Add Hidden State Denoising (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.93 -> 0.86 | Helped |
| Set denoising steps K to 1 instead of 3 (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.86 -> 0.96 | Hurt |
| Set denoising steps K to 2 instead of 3 (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.86 -> 0.89 | Hurt |
| Set denoising steps K to 4 instead of 3 (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.86 -> 0.86 | No clear effect |
| Add Embedding Enhancement (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.92 -> 0.86 | Helped |
| Set scale factor b to 0.1 instead of 0.5 (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.86 -> 0.87 | Hurt |
| Set scale factor b to 1 instead of 0.5 (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.86 -> 0.89 | Hurt |
| Train on AF+T instead of C+T (ablation) | FlowDiffuser · Sintel train clean | EPE: 0.89 -> 0.86 | Helped |
| Add Conditional-RDD to RAFT (ablation) | RAFT · KITTI train | EPE: 5.04 -> 4.51 | Helped |
| Add Conditional-RDD to GMA (ablation) | GMA · KITTI train | EPE: 4.69 -> 4.32 | Helped |
Surprising findings
- Plugging the recurrent denoising decoder into RAFT, GMA, KPA-Flow and SKFlow improves all four, indicating the approach is decoder-agnostic.
- The model is insensitive to the scale factor b, unlike classification and detection diffusion tasks reported to be sensitive.
Limitations
- Relies on a fixed number of denoising steps (K=3) and training-time iteration tradeoffs.
- Gains on KITTI test over best prior methods are modest in some metrics; EPE on Sintel final is not the top rank in all cases.
Experimental setup
Datasets: FlyingChairs FlyingThings Sintel KITTI-2015 HD1K AutoFlow
Benchmarks: Sintel KITTI-2015
Metrics: EPE F1-all average rank
Baselines: RAFT GMA SeperableFlow CRAFT GMFlow GMFlowNet KPA-Flow OCTC SKFlow FlowFormer RAFT-it GMFlow+
Builds on: RAFT DDIM DDVM GMA KPA-Flow
Closest papers by meaning
- DiffSF: Diffusion Models for Scene Flow Estimation (NeurIPS 2024 · similarity 0.89)
- Recurrent Partial Kernel Network for Efficient Optical Flow Estimation (AAAI 2024 · similarity 0.87)
- DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework (CVPR 2025 · similarity 0.87)
- Asymmetric Flow Models (NeurIPS 2026 · similarity 0.87)
- StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · similarity 0.87)
- Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.