Research map / 3D vision / Face and shape

Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video

NeurIPS 2026 · 3D vision › Face and shape · approach: Articulated and non-rigid shape learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts a pretrained multi-view geometric foundation model (MV-DUSt3R) to unlabeled Internet video using SfM sparse anchors, 3D Gaussian rendering consistency, and anchor-data regularization, improving zero-shot 3D reconstruction.

Problem

Geometric foundation models for 3D reconstruction are limited by scarce labeled 3D data; Internet videos are abundant but lack ground-truth geometry and contain noisy observations, and naive pseudo-depth fine-tuning degrades performance.

Approach

Combine sparse COLMAP/SfM point-cloud anchors (global structure) with dense photometric consistency from differentiable 3D Gaussian rendering of held-out target frames, plus a small replay fraction of supervised data to prevent forgetting, when fine-tuning feedforward 3D models on raw video.

Key results

• Relative to MV-DUSt3R, SAGE reduces Chamfer Distance by about 20% on 7Scenes, 29% on TUM-RGBD and 42% on MP3D (MP3D CD 43.23 to 24.90), with ScanNet CD improving about 6%.
• On MP3D with 8 views, SAGE-10K reaches DAc@0.5 of 48.0 versus 30.3 for MV-DUSt3R.
• Fine-tuning VGGT with SAGE reduces CD on all four benchmarks, e.g. 7Scenes CD 0.65 to 0.48.

Surprising findings

• Naive pseudo-depth video supervision (VDS) from Depth Anything Video often fails to improve or degrades performance, e.g. MP3D DAc@0.5 drops to 14.3.
• Freezing the pretrained Gaussian head works better than jointly updating it.

Limitations

• Relies on a static-scene pretrained model and filters out dynamic clips, so dynamic content in internet video is not exploited.
• Some metrics remain below VGGT, which was trained at higher 518 resolution, and comparison is at 224 resolution.
• Scaling claims rest on a limited range (100 to 10K scenes) with some results mixed across benchmarks.

Experimental setup

Datasets: RealEstate10K DL3DV ScanNet 7Scenes TUM RGB-D Matterport3D UASOL

Benchmarks: ScanNet 7Scenes TUM-RGBD Matterport3D UASOL

Metrics: Chamfer Distance DAc@0.2 DAc@0.5 RRE RTE mAE PSNR SSIM LPIPS

Baselines: Spann3R DUSt3R VGGT MV-DUSt3R Video depth supervision (Depth Anything Video)

Builds on: MV-DUSt3R DUSt3R 3D Gaussian Splatting COLMAP

Closest papers by meaning

• Emergent Extreme-View Geometry in 3D Foundation Models (CVPR 2026 · similarity 0.86)
• 3DPoV: Improving 3D understanding via Patch Ordering on Videos (ICML 2026 · similarity 0.86)
• Online3R: Online Learning for Consistent Sequential Reconstruction Based on Geometry Foundation Model (CVPR 2026 · similarity 0.86)
• Feat2GS: Probing Visual Foundation Models with Gaussian Splatting (CVPR 2025 · similarity 0.85)
• VideoMDM: Towards 3D Human Motion Generation From 2D Supervision (NeurIPS 2026 · similarity 0.84)
• GeoVideo: Introducing Geometric Regularization into Video Generation Model (NeurIPS 2025 · similarity 0.84)