Research map / 3D vision / Depth and stereo

UFM: A Simple Path towards Unified Dense Correspondence with Flow

NeurIPS 2025 · 3D vision › Depth and stereo · approach: Learned optical and scene flow · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a generic transformer (DINOv2 encoder + self-attention + DPT heads) to directly regress dense flow and covisibility on a unified optical flow and wide-baseline matching dataset, with a classification-based local refinement.

Problem

Dense correspondence has been split between optical flow (small baselines) and wide-baseline matching (large viewpoint changes), with specialized models failing to generalize across both; the paper asks whether unified training can help both.

Approach

A simple end-to-end transformer that directly regresses pixel-level flow (not coarse-to-fine cost volumes or patch-level matching), trained only on covisible pixels with a robust loss on 12 unified flow and wide-baseline datasets, plus a local feature-based refinement module.

Key results

• UFM 560 reaches 12.87 EPE on TA-WB covisible pixels versus 48.10 for RoMa, while running in 42.9 ms versus 387.4 ms for RoMa.
• Trained jointly on optical flow and wide-baseline data, UFM improves over its single-domain variants: OF+WB gives Sintel-Clean EPE 1.02 vs 1.27 (OF only) and KITTI EPE 2.62 vs 15.57.
• On zero-shot Sintel-Final and KITTI covisible EPE, UFM 560 refine achieves 1.40 and 1.69 versus 2.30 for RoMa on KITTI (RoMa is 390 ms).

Measured effects reported in the paper

Surprising findings

• Freezing the DINOv2 encoder substantially hurts UFM performance, contrary to RoMa's frozen-encoder design.
• Joint training on optical flow and wide-baseline data improves both domains, not just one.

Limitations

• Struggles with extreme seasonal changes and cross-spectrum matching (e.g., visible to thermal IR) where RoMA is more robust.
• Training and TA-WB evaluation are biased toward static scenes, potentially reducing accuracy on dynamic objects.
• DINOv2 features can drift during extended end-to-end training, reducing semantic matching ability.

Experimental setup

Datasets: BlendedMVS MegaDepth TartanAirV2 ScanNet++ V2 Habitat CAD StaticThings Kubric4D FlyingThings FlyingChairs Spring Monkaa HD1K

Benchmarks: ETH3D DTU TA-WB Sintel KITTI Scannet-1500 WxBS

Metrics: EPE 1px outlier rate 2px outlier rate 5px outlier rate AUC @ 5/10/15 degrees runtime ms

Baselines: SEA-RAFT FlowFormer UniMatch RoMa MASt3R GMFlow

Builds on: DINOv2 DPT RoMa MASt3R Charbonnier robust loss

Closest papers by meaning

• Dense Flow from Adaptive Correspondence (NeurIPS 2026 · similarity 0.90)
• Learning Accurate Dense Correspondences and When To Trust Them (CVPR 2021 · similarity 0.88)
• Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence (CVPR 2019 · similarity 0.88)
• Adapting Dense Matching for Homography Estimation with Grid-based Acceleration (CVPR 2025 · similarity 0.87)
• HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation (AAAI 2025 · similarity 0.87)
• Flow-NeRF: Joint Learning of Geometry, Poses, and Dense Flow within Unified Neural Representations (CVPR 2025 · similarity 0.85)