Research map / 3D vision / 3D generation
ProMo3D: Probing Motion Cues in Frozen 3D Foundation Models
Xiaoang Zhang, Mert Kiray, Yordanka Velikova, Manoj Biswanath, Benjamin Busam
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Probes whether frozen feed-forward 3D foundation model tokens already encode motion cues that lightweight readouts can expose for dynamic 4D understanding.
Problem
Dynamic 4D understanding with feed-forward 3D foundation models often requires large-scale motion supervision or hand-crafted, architecture-specific heuristics.
Approach
Treats frozen 3D foundation model tokens as the source of motion information and exposes it through lightweight readout functions rather than re-training motion-specific features.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use low-rank quadratic probe instead of linear or diagonal probes (ablation) | PMD probes on pi3X, DA3, DUSt3R · DAVIS-all layer-wise probing | motion decodability | Improved |
| retrain probe with shuffled token-motion labels (ablation) | pi3X rank-8 probe · YouTube-VOS val | balanced accuracy (%): 91.67 -> 50.34 | Hurt |
| add SAM2 refinement to pi3X-PMD (ablation) | pi3X-PMD · DAVIS17 | J-M: 73.7 -> 81.9 | Improved |
| add SAM2 refinement to pi3-PMD (ablation) | pi3-PMD · DAVIS17 | J-M: 72.7 -> 80.2 | Improved |
| use pi3X backbone instead of DUSt3R (ablation) | PMD · DAVIS17 | J-M: 62.2 -> 73.7 | Improved |
| use pi3X backbone instead of DA3 (ablation) | PMD · DAVIS17 | J-M: 69.6 -> 73.7 | Improved |
| increase probe rank from 8 to 128 (validation) (ablation) | pi3X-PMD low-rank probe · YouTube-VOS val | normalized gain x100: 60.77 -> 60.38 | No clear effect |
| increase probe rank from 8 to 256 (diagnostic) (ablation) | pi3X-PMD low-rank probe · DAVIS-all | normalized gain x100: 66.70 -> 64.66 | Hurt |
| use low-rank probe (k=8) rather than full-rank probe with fewer training videos (ablation) | pi3X probe · reduced training-video count | normalized gain | Improved |
| reduce number of training videos (ablation) | pi3X-PMD · DAVIS-all | normalized gain, J-M | Hurt |
| use Otsu + 0.1 threshold instead of default Otsu + 0.2 (ablation) | coarse pi3X-PMD · DAVIS-all, SegTrackv2, FBMS-59 mean | mean J-M: 72.4 -> 73.0 | Improved |
| use extreme fixed thresholds (0.30 or 0.70) (ablation) | coarse pi3X-PMD · three-benchmark mean | mean J-M: 72.4 -> 66.8 (0.30), 66.4 (0.70) | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- GeoMotion — GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry (CVPR 2026) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding (NeurIPS 2025 · similarity 0.87)
- Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video (CVPR 2025 · similarity 0.86)
- HIMMEL: Hierarchical Interleaved Multi-stream Motion Encoding for Long Video Understanding (NeurIPS 2026 · similarity 0.86)
- Rethinking the State Update Gate for Long-Sequence Recurrent 3D Reconstruction (NeurIPS 2026 · similarity 0.86)
- Feat2GS: Probing Visual Foundation Models with Gaussian Splatting (CVPR 2025 · similarity 0.86)
- Cross-Modal Distillation for 2D/3D Multi-Object Discovery from 2D Motion (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
