Research map / Video / Video segmentation and tracking
Learned optical flow estimation
Estimates optical flow with multi-frame models, occlusion awareness and graph reasoning.
73 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Video segmentation and tracking, keyword cluster “interpolation · flow · event”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports None of the tracked benchmarks has a report from 2025–26.
15 papers in 2023–24 and 20 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 3.4% to 3.7%.
No longer reported by papers here: GOPRO (PSNR, last in 2024); GOPRO (SSIM, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Determining Occlusions From Space and Time Image Reconstructions (CVPR 2016 · 12 citations)
- 2017 · GMS: Grid-based Motion Statistics for Fast, Ultra-Robust Feature Correspondence (CVPR 2017 · 618 citations)
- 2018 · End-to-End Flow Correlation Tracking With Spatial-Temporal Attention (CVPR 2018 · 314 citations)
- 2019 · Unsupervised Event-Based Learning of Optical Flow, Depth, and Egomotion (CVPR 2019 · 574 citations)
- 2020 · FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation (CVPR 2020 · 283 citations)
- 2021 · SMURF: Self-Teaching Multi-Frame Unsupervised RAFT With Full-Image Warping (CVPR 2021 · 83 citations)
- 2022 · Towards an End-to-End Framework for Flow-Guided Video Inpainting (CVPR 2022 · 145 citations)
- 2023 · TransFlow: Transformer As Flow Learner (CVPR 2023 · 104 citations)
- 2024 · MemFlow: Optical Flow Estimation and Prediction with Memory (CVPR 2024 · 45 citations)
- 2025 · DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework (CVPR 2025 · 15 citations)
- 2026 · WorldTree: Towards 4D Dynamic Worlds from Monocular Video using Tree-Chains (ICLR 2026 · 0 citations)
Earliest papers here
- Determining Occlusions From Space and Time Image Reconstructions (CVPR 2016 · 12 citations)
- GMS: Grid-based Motion Statistics for Fast, Ultra-Robust Feature Correspondence (CVPR 2017 · 618 citations)
- Learning Motion Patterns in Videos (CVPR 2017 · 290 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GOPRO · PSNR (higher is better)
5 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 31.67 | Cascaded Deep Video Deblurring Using Temporal Sharpness Prior (CVPR 2020 · 165 citations) |
| 2022 | 33.32 | Deep Recurrent Neural Network with Multi-Scale Bi-directional Propagation for Video Deblurring (AAAI 2022 · 53 citations) |
| 2024 | 35.98 | Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · 25 citations) |
GOPRO · SSIM (higher is better)
5 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 0.928 | Cascaded Deep Video Deblurring Using Temporal Sharpness Prior (CVPR 2020 · 165 citations) |
| 2022 | 0.963 | Deep Recurrent Neural Network with Multi-Scale Bi-directional Propagation for Video Deblurring (AAAI 2022 · 53 citations) |
| 2024 | 0.979 | Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · 25 citations) |
What has been tried
60 papers here report 625 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 5%, hurt 18%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| remove 2-frame pretraining | Sintel clean · EPE | 1.00 -> 1.16 | Hurt | MemFlow: Optical Flow Estimation and Prediction with Memory (CVPR 2024 · 45 citations) |
| add track visibility information | CVO Final · EPE (all) | 1.46 -> 1.43 | Helped | Dense Optical Tracking: Connecting the Dots (CVPR 2024 · 30 citations) |
| remove BBFP | DVD · PSNR | 34.95 -> 34.74 | Hurt | Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · 25 citations) |
| Add the dual-pyramid encoder | MPI-Sintel final · EPE | 2.58 -> 2.51 | Helped | DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework (CVPR 2025 · 15 citations) |
| add motion prior from optical-flow trajectories | TAP-Vid DAVIS · AJ | 60.5 -> 63.2 | Helped | FlowTrack: Revisiting Optical Flow for Long-Range Dense Tracking (CVPR 2024 · 13 citations) |
| add the self-corrector branch to CDC | DSEC · EPE | 0.89 -> 0.81 | Helped | Efficient Meshflow and Optical Flow Estimation from Event Cameras (CVPR 2024 · 12 citations) |
| replace depth weighting with occlusion-aware weighting | KITTI VFI · SSIM | 0.733 -> 0.734 | No clear effect | OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation (CVPR 2024 · 12 citations) |
| Use BOFC instead of temporal-frame replacement or movement-speed correction | DNV real-world · NS (%) | 91.48; 91.88 -> 93.19 | Helped | DanceFix: An Exploration in Group Dance Neatness Assessment Through Fixing Abnormal Challenges of Human Pose (AAAI 2025 · 11 citations) |
| add video stabilization | CLEAR dataset · PSNR | 26.80 to 28.09 | Helped | Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence (CVPR 2024 · 11 citations) |
| Use the SIM pipeline instead of the recursive pipeline | Sintel · runtime (ms) | 122.18 -> 84.59 | Helped | StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · 5 citations) |
| add Motion Information Propagation (MIP) | KITTI-2015 train · F1-all | 8.19 -> 7.12 | Helped | M2Flow: A Motion Information Fusion Framework for Enhanced Unsupervised Optical Flow Estimation in Autonomous Driving (AAAI 2025 · 2 citations) |
| add difference motion encoding | DSEC test · EPE | 0.82 -> 0.72 | Helped | EDCFlow: Exploring Temporally Dense Difference Maps for Event-based Optical Flow Estimation (CVPR 2025 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 12 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- KITTI 2015 · 3 of 12 papers
- Sintel · 2 of 12 papers
- FlyingChairs · 2 of 12 papers
- DSEC · 2 of 12 papers
Baselines
- RAFT · 2 of 12 papers
- FlowFormer++ · 2 of 12 papers
- FlowDiffuser · 2 of 12 papers
- GMA · 2 of 12 papers
- E-RAFT · 2 of 12 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 35 papers; from OpenAlex and the papers' affiliations).
- Beijing Jiaotong University · 2 papers
- Sun Yat-sen University · 2 papers
- Meta · 2 papers
- Nanyang Technological University · 2 papers
- Xi’an Jiaotong University · 2 papers
- Johns Hopkins University · 2 papers
- Huazhong University of Science and Technology · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Video frame interpolation (60 papers)
- Event-based video reconstruction (63 papers)
- Spike and neuromorphic deblurring (57 papers)
Elsewhere in the topic
- Self-supervised video representation learning (167 papers)
- Video restoration with temporal priors (159 papers)
- Layered video decomposition and dynamics (140 papers)
- Stochastic video prediction (125 papers)
- Physics-inspired motion prediction (113 papers)
- Sign language and physiological video analysis (99 papers)
- Weakly supervised video anomaly detection (99 papers)
- Spatio-temporal predictive modeling (91 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).