Research map / Probabilistic and scientific modeling / Variational and generative sampling
Marrying Optimal Transport and ODEs for Unified Continuous-Time 4D Reconstruction and Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uni4R learns a continuous velocity field via flow-matching-guided decoding and ODE integration to jointly do feed-forward 4D reconstruction and point tracking at arbitrary timestamps, trained with an integral-consistency loss from integer-frame pointmaps.
Problem
Existing unified 4D reconstruction and point tracking methods predict only at integer timestamps or use heuristic (B-spline) interpolation, lacking kinematic coherence; supervising continuous velocity fields from integer-time pointmaps is ill-posed.
Approach
Formulates an optimal-transport flow-matching path over anchor features (convex combination of adjacent-frame features) as a kinematic inductive bias for a local velocity decoder, and supervises it end-to-end by Euler-integrating predicted velocities from integer pointmaps to the next integer pointmap (integral-consistency loss).
Key results
- On world-coordinate 3D point tracking, Uni4R reports APD_3D of 81.59 on Point Odyssey (all points) vs 81.27 for V-DPM, and 79.43 on PStudio vs 78.76 for V-DPM.
- On world-coordinate 4D reconstruction, Uni4R reaches APD 80.71 and EPE 0.222 on Point Odyssey, and APD 84.36 on TUM-Dynamics, versus 78.73 and 83.42 for St4RTrack.
- Uni4R infers 80-frame input in 1.57 s on an H800 (vs 4.69 s for TraceAnything), with 43.67 GB VRAM, and achieves the best MAM (7.272) on the kinematics-aware benchmark.
Surprising findings
- Uni4R uses more VRAM (43.67 GB) than TraceAnything (35.71 GB) despite being faster.
Limitations
- Ill-posedness of continuous velocity supervision is addressed only heuristically; the OT path is a linear convex combination of adjacent-frame features rather than true distribution transport.
- Kinematics-aware benchmark is small (100 Kubric scenes, 24 integer frames) and derived from synthetic data.
- Higher VRAM footprint than TraceAnything.
Experimental setup
Datasets: Point Odyssey Dynamic Replica Kubric
Benchmarks: Point Odyssey Dynamic Replica ADT PStudio TUM-Dynamics DAVIS Kinematics-aware benchmark
Metrics: APD_3D EPE MAM ELV VLPD APD
Baselines: SpaTracker MonST3R POMATO St4RTrack TraceAnything V-DPM DUSt3R MASt3R SpaTracker+MonST3R SpaTracker+RANSAC-Procrustes
Builds on: DUSt3R flow matching St4RTrack TraceAnything V-DPM
Closest papers by meaning
- Flux4D: Flow-based Unsupervised 4D Reconstruction (NeurIPS 2025 · similarity 0.87)
- Optical Flow Matching: Reframing Optical Flow as Continuous Transport Dynamics (CVPR 2026 · similarity 0.87)
- Trace Anything: Representing Any Video in 4D via Trajectory Fields (ICLR 2026 · similarity 0.86)
- Stable Velocity: A Variance Perspective on Flow Matching (ICML 2026 · similarity 0.86)
- Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking (CVPR 2024 · similarity 0.86)
- ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.