Research map / Probabilistic and scientific modeling / Molecules and proteins
Context-Aware Iteration Policy Network for Efficient Optical Flow Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Context-Aware Iteration Policy Network that learns per-sample iteration skipping for recurrent optical flow models (RAFT, GMA, FlowFormer, KPA-Flow), cutting FLOPs by about 40% on Sintel and 20% on KITTI while keeping accuracy.
Problem
Recurrent optical flow networks run a fixed large number of update iterations for every sample, which is computationally expensive at inference time.
Approach
A lightweight policy network, conditioned on the historical hidden cell, an iteration embedding, and a user resource preference r, decides per iteration whether to skip the update operator; an incremental loss forces it to predict the magnitude of the next iteration's flow improvement, so it implicitly anticipates future gains.
Key results
- DRAFT reduces FLOPs by 44%/43% on Sintel-Clean/Final test with EPE 1.93/3.20 vs RAFT 1.94/3.18, and 22% on KITTI test with F1-all 5.16 vs 5.10.
- DFlowFormer reduces FLOPs by 49% on Sintel-Final train (C+T) with EPE 2.43 vs baseline 2.33 at training-data setting (reported as DFlowFormer 496G vs 974G FLOPs, with EPE slightly worse).
- Ablation: removing previous/future information (DRAFT-B) increases KITTI EPE from 5.52 to 6.97 and F1-all from 19.0 to 25.7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the dynamic iteration policy to RAFT | RAFT · Sintel-Clean test | FLOPs(G): 730 -> 411(-44%) | Helped |
| apply the dynamic iteration policy to RAFT | RAFT · Sintel-Final test | FLOPs(G): 730 -> 412(-43%) | Helped |
| apply the dynamic iteration policy to RAFT | RAFT · KITTI-15 test | FLOPs(G): 595 -> 465(-22%) | Helped |
| replace max(0,x) resource preference loss with absolute-distance L1 loss (ablation) | DRAFT · Sintel-Clean, Sintel-Final, KITTI | EPE: 1.56 vs 1.73; 2.74 vs 2.97; 5.52 vs 5.84 | Hurt |
| remove previous- and future-iteration contextual information (ablation) | DRAFT · Sintel-Clean, Sintel-Final, KITTI | EPE: 1.56 vs 1.81; 2.74 vs 2.99; 5.52 vs 6.97 | Hurt |
| remove future-iteration information (ablation) | DRAFT · Sintel-Clean, Sintel-Final, KITTI | EPE: 1.56 vs 1.58; 2.74 vs 2.77; 5.52 vs 5.66 | Hurt |
| add future-iteration information using incremental loss (ablation) | DRAFT-P · KITTI examples | EPE: 1.32; 0.12 | Mixed |
| replace repeated iteration skipping with early exit after the first skip (ablation) | DRAFT · Sintel-Clean train | EPE: 1.57 vs 1.56 | Hurt |
| vary resource preference r from 0.5 to 0.8 (ablation) | DRAFT · KITTI | FLOPs(G): 225 -> 375 | Helped |
Surprising findings
- About 67.1% (Final) and 43.0% (KITTI) of samples reach near-best EPE within the first 15 iterations, indicating frequent bottlenecks.
- Simple exit-after-first-skip strategy (DRAFT-Exit) performs worse than the re-enter policy, suggesting early exit decisions are often wrong.
Limitations
- Dynamic FLOPs savings are smaller on KITTI (about 20%) than on Sintel (about 40%).
- On some settings, the dynamic model is slightly worse than the baseline (e.g., FlowFormer Sintel-Final test EPE 2.21 vs 2.18), and the paper reports gains only at comparable FLOPs.
Experimental setup
Datasets: FlyingChairs FlyingThings Sintel KITTI-2015 HD1K
Benchmarks: Sintel-Clean Sintel-Final KITTI-2015
Metrics: EPE F1-all FLOPs inference time
Baselines: RAFT GMA FlowFormer KPA-Flow DRAFT-Exit
Builds on: RAFT Gumbel softmax GMA KPA-Flow FlowFormer
Closest papers by meaning
- Recurrent Partial Kernel Network for Efficient Optical Flow Estimation (AAAI 2024 · similarity 0.88)
- Deep Equilibrium Optical Flow Estimation (CVPR 2022 · similarity 0.87)
- DDFlow: Learning Optical Flow with Unlabeled Data Distillation (AAAI 2019 · similarity 0.87)
- StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · similarity 0.87)
- DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework (CVPR 2025 · similarity 0.86)
- FlowDiffuser: Advancing Optical Flow Estimation with Diffusion Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.