Research map / Video / Video segmentation and tracking
DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPFlow, a recurrent dual-pyramid optical flow network trained only on low-resolution data that generalizes to 8K inputs, and Kubric-NK, a multi-resolution (1K–8K) synthetic benchmark for evaluating resolution generalization.
Problem
Optical flow methods are designed for about 1K inputs and fail to generalize to high-resolution (up to 8K) inputs; existing high-resolution evaluations are only qualitative or sparse, so there is no quantitative benchmark for resolution generalization.
Approach
A recurrent bidirectional dual-pyramid encoder combines an image pyramid (downsampled inputs) with a feature pyramid (ConvGRU forward/backward passes), with a convolutional Cross-Gated Unit replacing attention, and the number of pyramid levels is chosen adaptively from input size at inference.
Key results
- On Spring, DPFlow outperforms SEA-RAFT and RPKNet by 6.5% and 10.8% in public zero-shot/fine-tuning benchmarks, and beats RAPIDFlow by 13% at 4K.
- On Kubric-NK at 8K, DPFlow outperforms RAPIDFlow by over 30% (EPE 4.11 vs 5.96) and FlowFormer++ by 83%.
- Achieves the best average rank on MPI-Sintel and KITTI 2015 public benchmarks among two-frame methods (average rank 2.7, Table 4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPFlow instead of RAPIDFlow | optical flow estimation · Kubric-NK 8K | EPE: 5.96 -> 4.11 | Helped |
| Use DPFlow instead of the strongest listed prior method | optical flow estimation · Spring train | 1px: 3.08 -> 3.06 | Helped |
| Use DPFlow instead of CroCo-Flow | optical flow estimation · VIPER 2K | WAUC: 76.1 -> 72.9 | Hurt |
| Use DPFlow instead of FlowDiffuser | optical flow estimation · MPI-Sintel clean | EPE: 0.86 -> 1.02 | Hurt |
| Add the dual-pyramid encoder (ablation) | DPFlow encoder · MPI-Sintel final | EPE: 2.58 -> 2.51 | Helped |
| Add bidirectional recurrence to the dual-pyramid encoder (ablation) | DPFlow encoder · KITTI 2015 | Fl-All: 13.7 -> 12.1 | Helped |
| Replace ResNet with CGU (ablation) | DPFlow encoder · KITTI 2015 | Fl-All: 13.9 -> 12.1 | Helped |
| Replace ConvNeXt with CGU (ablation) | DPFlow encoder · KITTI 2015 | Fl-All: 13.3 -> 12.1 | Helped |
| Replace Cross-Swin with CGU (ablation) | DPFlow encoder · KITTI 2015 | Fl-All: 14.1 -> 12.1 | Helped |
| Increase pyramid depth from 3 to 6 levels (ablation) | DPFlow · Kubric-NK 8K | EPE: 14.7 -> 4.11 | Helped |
| Increase pyramid depth from 3 to 6 levels (ablation) | DPFlow · KITTI 2015 (1K) | Fl-All: 11.1 -> 12.7 | Hurt |
Surprising findings
- RAPIDFlow, which handles high resolution well, underperforms on standard-resolution benchmarks like MPI-Sintel and KITTI.
- Single-resolution RAFT-style methods such as SEA-RAFT degrade by about two orders of magnitude in error from 1K to 8K.
Limitations
- Does not beat all methods on the MPI-Sintel clean pass.
- Kubric-NK does not cover adverse conditions such as lighting, fog, or fine details.
- Optimal pyramid depth must be tuned per resolution, and the chosen checkpoint differs per benchmark.
Experimental setup
Datasets: FlyingChairs FlyingThings3D KITTI 2015 HD1K Sintel Spring Kubric-NK
Benchmarks: Spring MPI-Sintel KITTI 2015 VIPER Middlebury-ST Kubric-NK
Metrics: 1px error EPE Fl-All WAUC
Baselines: RAFT SEA-RAFT RAPIDFlow FlowFormer++ RPKNet DIP MatchFlow CroCo-Flow FlowDiffuser GMA
Builds on: RAFT RAPIDFlow RPKNet SEA-RAFT Kubric
Closest papers by meaning
- StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · similarity 0.90)
- MemFlow: Optical Flow Estimation and Prediction with Memory (CVPR 2024 · similarity 0.88)
- OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation (CVPR 2024 · similarity 0.88)
- FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis (CVPR 2024 · similarity 0.87)
- FlowDiffuser: Advancing Optical Flow Estimation with Diffusion Models (CVPR 2024 · similarity 0.87)
- Recurrent Partial Kernel Network for Efficient Optical Flow Estimation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.