Research map / 3D vision / Depth and stereo
Zero-Shot Monocular Scene Flow Estimation in the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a feedforward monocular scene flow model that jointly predicts two pointmaps and 3D camera-space offsets, initialized from MASt3R and trained on a ~1M-sample multi-dataset recipe with scale-adaptive loss, showing zero-shot generalization.
Problem
Monocular scene flow estimation from two RGB frames generalizes poorly to out-of-domain data because geometry and motion are entangled, scene flow data is scarce, datasets mix metric and relative scales, and parameterization choice matters.
Approach
Extend pointmap models (DUSt3R/MASt3R) with a third DPT head that predicts 3D motion offsets jointly with geometry, train on a large mixed-scale dataset recipe with an Xor-style scale normalization, and parameterize scene flow as camera-space 3D offsets.
Key results
- On KITTI, the method reaches EPE 0.452 and AccS 0.398 versus 0.454 and 0.345 for Self-Mono-SF.
- The zero-shot Ours-exclude model on KITTI reaches EPE 0.641 versus 3.404 for DUSt3R and 3.708 for MASt3R finetuned on the same data recipe.
- Xor-based scale normalization gives EPE 0.452 versus 1.596 for Align, 0.899 for Always, and 0.758 for Never on KITTI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use camera-space 3D offsets instead of end-point parameterization (ablation) | Monocular scene flow model · KITTI | EPE: 0.618 -> 0.452 | Helped |
| Use camera-space 3D offsets instead of depth change plus optical flow (ablation) | Monocular scene flow model · KITTI | EPE: 11.905 -> 0.452 | Helped |
| Add projected optical flow loss (ablation) | CSO model with Xor scale handling · KITTI | EPE: 0.501 -> 0.452 | Helped |
| Add projected optical flow loss (ablation) | CSO model with Xor scale handling · KITTI | δ1-m: 0.319 -> 0.345 | Helped |
| Use Xor scale handling instead of Align (ablation) | CSO model · KITTI | EPE: 1.596 -> 0.452 | Helped |
| Use Xor scale handling instead of Always normalization (ablation) | CSO model · KITTI | EPE: 0.899 -> 0.452 | Helped |
| Use Xor scale handling instead of Never normalization (ablation) | CSO model · KITTI | EPE: 0.758 -> 0.452 | Helped |
| Train jointly for geometry and motion | Monocular scene flow model · Dynamic scenes | depth estimation | Helped |
Surprising findings
- Training with the optical-flow loss also improves depth estimation (delta1-m rises by 0.034), suggesting geometry and motion benefit from joint training.
- Depth-plus-optical-flow baselines built on DUSt3R or MASt3R perform far worse than the joint model even when the joint model is zero-shot.
Limitations
- Camera ego-motion and scene motion remain entangled in the output.
- Monocular evaluation protocol on Spring and VKITTI2 is nonstandard, defined by the authors.
Experimental setup
Datasets: SHIFT Dynamic Replica Virtual KITTI 2 MOVi-F PointOdyssey Spring KITTI Scene Flow
Benchmarks: KITTI Scene Flow Virtual KITTI 2 Spring DAVIS (qualitative) RoboTAP (qualitative)
Metrics: EPE AccS AccR Outliers AbsRel delta1
Baselines: Self-Mono-SF OpticalExpansion DUSt3R MASt3R MonST3R DepthAnythingV2 + RAFT SpaTracker
Builds on: DUSt3R MASt3R CroCo v2 MiDaS scale-invariant loss Self-Mono-SF
Closest papers by meaning
- ZeroFlow: Scalable Scene Flow via Distillation (ICLR 2024 · similarity 0.89)
- Flow-NeRF: Joint Learning of Geometry, Poses, and Dense Flow within Unified Neural Representations (CVPR 2025 · similarity 0.88)
- Taming generative video models for zero-shot optical flow extraction (NeurIPS 2025 · similarity 0.88)
- DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method (NeurIPS 2025 · similarity 0.87)
- DepthFM: Fast Generative Monocular Depth Estimation with Flow Matching (AAAI 2025 · similarity 0.87)
- DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.