Research map / 3D vision / Depth and stereo
Learned optical and scene flow
Estimates motion fields with unrolled, unsupervised or joint learning of flow, depth and ego-motion.
108 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Depth and stereo, keyword cluster “motion · flow · localization”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports None of the tracked benchmarks has a report from 2025–26.
19 papers in 2023–24 and 36 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 6.2% to 8.8%.
No longer reported by papers here: KITTI 2015 test (Fl-all, last in 2021).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation (CVPR 2016 · 2,292 citations)
- 2017 · DeMoN: Depth and Motion Network for Learning Monocular Stereo (CVPR 2017 · 770 citations)
- 2018 · GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose (CVPR 2018 · 1,301 citations)
- 2019 · Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation (CVPR 2019 · 674 citations)
- 2020 · EventCap: Monocular 3D Capture of High-Speed Human Motions Using an Event Camera (CVPR 2020 · 117 citations)
- 2021 · Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy (CVPR 2021 · 147 citations)
- 2022 · Event-Aided Direct Sparse Odometry (CVPR 2022 · 126 citations)
- 2023 · Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo (CVPR 2023 · 71 citations)
- 2024 · SpatialTracker: Tracking Any 2D Pixels in 3D Space (CVPR 2024 · 55 citations)
- 2025 · MegaSaM: Accurate, Fast and Robust Structure and Motion from Casual Dynamic Videos (CVPR 2025 · 41 citations)
- 2026 · Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation (ICML 2026 · 0 citations)
Earliest papers here
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation (CVPR 2016 · 2,292 citations)
- Simultaneous Optical Flow and Intensity Estimation From an Event Camera (CVPR 2016 · 306 citations)
- 3D Part-Based Sparse Tracker With Automatic Synchronization and Registration (CVPR 2016 · 69 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
KITTI 2015 test · Fl-all (lower is better)
4 papers, 2018–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 11.11 | UnFlow: Unsupervised Learning of Optical Flow With a Bidirectional Census Loss (AAAI 2018 · 569 citations) |
| 2021 | 9.7 | Feature-Level Collaboration: Joint Unsupervised Learning of Optical Flow, Stereo Depth and Camera Motion (CVPR 2021 · 29 citations) |
What has been tried
87 papers here report 798 measured changes (a component added, removed or swapped, compared with a baseline): helped 76%, no clear effect 4%, hurt 17%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add the ARAP loss | TAP-Vid DAVIS · AJ | 55.1 -> 61.1 | Helped | SpatialTracker: Tracking Any 2D Pixels in 3D Space (CVPR 2024 · 55 citations) |
| Add mono-depth initialization | Sintel · ATE | 0.038 -> 0.019 | Helped | MegaSaM: Accurate, Fast and Robust Structure and Motion from Casual Dynamic Videos (CVPR 2025 · 41 citations) |
| Add diffusion-based refinement to the coarse initialization | FlyingThings3D · EPE3D | 0.1559 -> 0.0114 | Helped | DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement (CVPR 2024 · 40 citations) |
| optimize camera poses rather than use MASt3R poses directly | DAVIS 2017 evaluation videos · PSNR | 29.21 -> 29.74 | Helped | GFlow: Recovering 4D World from Monocular Video (AAAI 2025 · 18 citations) |
| add BiHCM to the base model | E-3DTrack test · TFR (0.1m) | 0.5586 -> 0.6082 | Helped | 3D Feature Tracking via Event Camera (CVPR 2024 · 13 citations) |
| add Feature Denoiser | Sintel val flow clean · AEPE | 0.54 -> 0.51 | Helped | ProMotion: Prototypes As Motion Learners (CVPR 2024 · 11 citations) |
| replace the MonoGS representation with 2DGS | Replica average · Depth L1[cm]↓ | 4.52 -> 0.54 | Helped | 4DTAM: Non-Rigid Tracking and Mapping via Dynamic Surface Gaussians (CVPR 2025 · 6 citations) |
| Train on combined optical-flow and wide-baseline data instead of optical-flow data only | Sintel-C · EPE | 1.27 -> 1.02 | Helped | UFM: A Simple Path towards Unified Dense Correspondence with Flow (NeurIPS 2025 · 6 citations) |
| track in world coordinates rather than camera coordinates | LSFOdyssey, XYZ coordinates · AJ_3D | 67.1 -> 70.7 | Helped | TAPIP3D: Tracking Any Point in Persistent 3D Geometry (NeurIPS 2025 · 6 citations) |
| Use more scenes, ADF58 instead of ADF15 | KITTI evaluation · Flepe | Helped | ADFactory: An Effective Framework for Generalizing Optical Flow with NeRF (CVPR 2024 · 5 citations) | |
| Replace the translation field with an affine field | DeformingThings4D sequences · LCD | 1.401 -> 1.266 | Helped | Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories (CVPR 2024 · 5 citations) |
| Use cascade rotation parameterization instead of first-order approximated rotation | Synthetic, noise-free (M = 5, N = 100) · median εang | 1.4 × 10−2 -> 1.6 × 10−4 | Helped | Full-DoF Egomotion Estimation for Event Cameras Using Geometric Solvers (CVPR 2025 · 4 citations) |
What reviewers will expect
Datasets and baselines named in the 19 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Sintel · 3 of 19 papers
- FlyingThings3D · 2 of 19 papers
- TartanAir · 2 of 19 papers
- DAVIS · 2 of 19 papers
- HD1K · 2 of 19 papers
- PointOdyssey · 2 of 19 papers
- Bonn · 2 of 19 papers
- Dynamic Replica · 2 of 19 papers
Baselines
- MonST3R · 4 of 19 papers
- DROID-SLAM · 3 of 19 papers
- DUSt3R · 3 of 19 papers
- MASt3R · 3 of 19 papers
- RoDynRF · 2 of 19 papers
- COLMAP · 2 of 19 papers
- RAFT · 2 of 19 papers
- GMFlow · 2 of 19 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 55 papers; from OpenAlex and the papers' affiliations).
- Peking University · 3 papers
- Meta · 3 papers
- Nanyang Technological University · 2 papers
- Adobe · 2 papers
- Nanjing University of Science and Technology · 2 papers
- NVIDIA · 2 papers
- Zhejiang University · 2 papers
- Stanford University · 2 papers
- Delft University of Technology · 2 papers
- University of California, Berkeley · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Feed-forward 3D reconstruction (133 papers)
- Camera model and calibration for SfM (99 papers)
- Scene coordinate regression for localization (112 papers)
- Dense feature matching for correspondence (80 papers)
Elsewhere in the topic
- Self-supervised monocular depth (116 papers)
- Neural light field and illumination imaging (107 papers)
- Cost-volume multi-view stereo (107 papers)
- Non-line-of-sight imaging (93 papers)
- Dense depth prediction and completion (85 papers)
- Foundation models for monocular geometry (81 papers)
- Depth from light field and defocus (60 papers)
- Learned stereo with specialized sensors (53 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).